Products Report Through Their API
Product statistics stay understandable when every system aggregates its own data and offers it through a small HTTP contract with uniform time semantics.
A statistics API is a product feature. That sentence sounds obvious, but while working on cross-product reporting, it only became clear after two worse designs. At first, a central collector was supposed to access several PostgreSQL databases directly. After that, special private reporting endpoints and dedicated keys were on the table. Both would have built the report deeper into the products than its purpose requires.
What remained was a smaller contract. A publishing system, a knowledge base and an orchestrator aggregate their own data themselves and offer their view through normal, authenticated HTTP endpoints. A downstream process combines the aggregates for explicitly bounded time windows – it sees neither tables nor Markdown, names, keys or internal IDs.
This split is not a special trick for dashboards. Every authorized consumer can use the same statistics resources: a capacity report, a quality check, a local analysis or a later export tool.
Data ownership is more than storage location#
Whoever owns a table knows its meaning. A publishing system knows when a revision was created and when a release was actually activated. A knowledge base distinguishes a started ingest from a verified state of knowledge. An orchestrator aggregates its execution data where the runs actually take place. A central collector would have to rebuild these meanings and keep up with every product change.
Direct SQL makes this rebuilding particularly easy – and therefore dangerous. A query works even though it may count an intermediate status that the product itself would never show as complete. A new migration can break the report without the public product API having changed. Database access also bypasses the normal permission check and pulls the credentials of several systems into one place.
With a product API, the domain aggregation lives where the state change is implemented:
Domain aggregation stays in the product
- Product action · creates a domain-defined state transition
- Own data store · keeps the product’s authoritative domain state
- Aggregation · translates internal states into bounded domain metrics
- Statistics API · delivers authenticated values with time range and meaning
- Collector · combines compatible responses without redefining them
- Report · explains the frozen shared state
The collector may combine the responses, but it may not reinterpret what a successful release or a verified claim is. These terms belong to the product.
Time is uniform, the domain is not#
A shared contract doesn’t mean that all products deliver the same metrics. That would be artificial leveling. The only shared properties are the ones a collector needs for reliable time windows:
bucketdetermines the resolution: hour, day, month or year.fromis the inclusive start of the window.tois the exclusive end.tzmakes the time zone explicit, UTC throughout here.bucketscontains a dense series, including empty intervals.totalssummarizes exactly the requested window.
The payload keeps its domain names. A publishing system can, for example, provide site-scoped aggregates for releases, content and reader access, a knowledge base space-scoped aggregates for usage, coverage and reviews, an orchestrator tenant-scoped runtime statistics for executions and connectors. No product has to adopt another product’s terms; all of them have to fulfill the same time contract.
{
"bucket": "hour",
"tz": "UTC",
"from": "2026-07-18T10:00:00.000Z",
"to": "2026-07-18T12:00:00.000Z",
"buckets": [
{ "ts": "2026-07-18T10:00:00.000Z", "items_created": 2 },
{ "ts": "2026-07-18T11:00:00.000Z", "items_created": 0 }
],
"totals": { "items_created": 2 }
}
The half-open notation [from,to) prevents double counting. The window from 10 to 11 o’clock contains an event at 10:59:59, but none at 11:00:00. The next window starts exactly there. A retry queries the same time window again and gets the same meaning.
Measured zero, missing observation, incomplete window#
A dense time series also contains empty buckets. That is practical, because the collector doesn’t have to guess or fill in missing hours. A zero is a measured value: the product evaluated the window and found no matching event.
Two other states have to be distinguished from this. Missing means that there is no observation for the metric – for example, because a source isn’t connected or a ratio cannot be calculated without a base. In the reporting process described here, the downstream report also determines incompleteness: an expected snapshot is missing, or the window had not yet closed at the time of collection. These three cases must not look the same in the report, because they carry different consequences: zero technical errors are reassuring, zero page views can be correct, a missing analytics connection is an open connection and not a measurement.
A product can make the distinction between measured and missing part of the response and label values, for example, with value_state and value_kind. A ratio without observations carries the state missing and is not output as a measured zero:
{
"metric": "engagement_rate",
"value_kind": "ratio",
"value_state": "missing"
}
For someone reading a report, this state matters more than another decimal place. Time window, data state and the meaning of a measurement therefore belong in the contract itself – not in a footnote on the report page.
Privacy starts with the shape of the response#
Aggregates should leave out information the purpose doesn’t need. A statistic can count successful, failed and rate-limited reader requests without storing user names, IP addresses, session IDs, Markdown content, URLs, job IDs or credentials in the statistics table. A knowledge base doesn’t have to output source texts through its statistics resources.
Such a negative list belongs in the documentation and in contract tests. “We currently don’t return any names” is weaker than a schema that has no field for names at all. The collector then cannot accidentally carry a sensitive value over into a JSON artifact or a report page.
Trace data also stays limited to its purpose. W3C Trace Context standardizes traceparent and tracestate so that a call can be correlated across several services. The W3C recommendation explicitly warns against transporting personal or other sensitive information in these headers. A trace ID connects technical steps; it doesn’t describe a person.
GET /v1/sites/example/stats/releases?bucket=hour&from=2026-07-18T10:00:00Z&to=2026-07-18T11:00:00Z
Authorization: Bearer <site-read-key>
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
Zero-config means using existing terms#
Telemetry is rarely completely configuration-free. A service needs at least durable storage and information about whether it runs in development, test or production. “Zero-config” is meant more narrowly here: for the reporting process, no special mode is activated in the product, no special key is created and no report target is entered.
The statistics endpoints can use existing product boundaries: a publishing system, for example, the site and a normal read key, a knowledge base the space and its read scope, an orchestrator the tenant.
OpenTelemetry provides a usable shared vocabulary for this. service.name names the logical service, deployment.environment.name distinguishes development, staging, production and test; as of August 2026, the older key deployment.environment is marked as deprecated in the OpenTelemetry conventions. Standardized fields like these prevent one service from treating prod, another live and a third production-eu as the same concept. The product sets these values once for its runtime; the reporting collector doesn’t have to repair labels.
The shared name alone is not enough, though. If an aggregator reads different field names than an older write path still delivers, successful calls can show up as errors. Such a transition can be made compatible in both directions: the reader understands old and new fields, the writer delivers both for the time being, and tests use real audit rows instead of hand-built ideal objects.
Bounded queries instead of hidden load#
A general statistics API needs hard limits, such as a maximum window length per resolution. Invalid or oversized windows return 422 instead of burdening the product database with an unexpected full evaluation.
These limits belong to the product contract, because only the product knows its costs. A collector should not try to get around them with many overlapping queries. For a long period, it uses the coarser resolution or works in distinct, non-overlapping pages.
The same applies to retention. Anonymous raw events can have their own retention period in the product. A report artifact can live longer if its aggregated content is meant for that. Raw event, product aggregate and published report have different lifecycles; they don’t have to end up in one store.
Existing keys instead of a back door#
A dedicated “reporting key” seems tidy at first, but it creates a second security logic. What may this key read? Does it apply to a site, a space or the whole product? Who rotates it? As soon as every product gives a different answer, the supposedly simple collector becomes a special case with far-reaching permissions.
The statistics APIs should therefore use existing scopes. A read-only key for a site may then read exactly the statistics of the site it was issued for. A read key for the knowledge base stays limited to its space. Without a matching key, each of these endpoints stays closed.
So the same rule applies to the collector as to other API users: a separate key per product, as narrow as possible; secrets only in the credential store; no keys in the workflow definition, JSON artifact, log or report. The workflow only knows a credential_ref; the actual value is inserted at the HTTP call and appears neither in the stored process nor in its result. This separation can be tested: a call without a key must return 401, a key without a matching scope 403, a correct key 200.
PostgreSQL is internal, not forbidden#
Rejecting direct database queries doesn’t mean that aggregates have to live outside the product database. For a publishing system or a knowledge base, PostgreSQL may be exactly the right place. Events and domain state can be recorded transactionally there, scoped by site or space and evolved with the same migrations as the rest of the product.
What matters is the direction of the dependency. The product may know its own database. An outside collector must not need to know it. Whether a response comes from a precomputed table, a bounded SQL aggregation or a cache remains an internal decision. What counts is that each product can design its internal storage according to its own operating model and, toward the outside, fulfills the same reliable time contract.
Documentation is part of the contract#
A new endpoint is only done when OpenAPI, llms.txt, llms-full.txt, the README and the detailed product documentation tell the same story. Otherwise, a person finds a scope in the README, a client generates from an older OpenAPI file and an agent reads a third variant.
That is why the repositories check their generated documents for drift. Contract tests send valid, invalid, oversized and unauthorized windows to the handlers. Integration tests use the real product storage. A binary test makes sure that the shipped file contains the same routes and the same version number as the source. These checks prevent the collector from depending on an outdated or internally known-only response shape.
APIs make exchange possible#
The immediate effect of this separation shows up in testing. Every product API can be checked against its own schema with a fixed window. The collector gets fixtures without database access. A product release can be rolled back independently at the API level, as long as the published contract is preserved.
A new consumer doesn’t need a new table in the product. Another reporting tool can call the same API just as well. A product may change its internal storage without the statistics resource having to be reinvented.
That also keeps the products understandable. Each product keeps its role – as a publishing system, knowledge base or orchestrator. Their statistics resources describe their own work as bounded aggregates with domain names. Everything else happens outside their boundaries. As long as the published contract is preserved, one part can be replaced without adjusting the others.
What has changed since then#
The post HTTP APIs: Small, Stable, Readable described an endpoint in 2014 as a long-term promise to unknown clients. Small resources, unambiguous timestamps, stable meanings and readable errors were meant to prevent every client from building its own defensive layer. These requirements still apply.
What’s new is the view of internal product statistics. A statistics path is not built for a specific dashboard but as a bounded projection of the product’s own data. It uses existing keys and scopes, delivers time boundaries and data state along with the values and publishes only aggregates. This keeps the API small, regardless of whether a person, a workflow or an agent calls it.
Further reading#
Revised and translated from German with the help of AI.