retro-footprint real calls

Run plays the chosen test for real on the stack, 10 times in two batches: 5 runs in a row, a pause of 20 seconds, 5 more runs (about a minute in all). A test is a few steps: pages of a front opened, API calls, SQL queries of its own. During each run, colleagues query some tables from their own connection (16, 9, 5, 2, 1 tables in the first batch, 6, 3, 2, 1, 1 in the second), and one colleague loops their own test on 2 tables during the whole first batch. The database side (performance_schema counters) sees every table read or written during a run, without knowing by whom. Keeping the tables seen in every run narrows the list: black, still candidate; light grey, ruled out. The looping colleague's tables survive the first batch; after the pause they are gone, and the second batch rules them out.

The counters are also read around each step, so each table is tied to the step that moved it: the path draws the test step by step, each front, API, query of the test and table joined by a dotted line, in the order they happened; a front the user was sent to by the previous one is reached by a redirect.

The trace of each API call of run 1 is read in Tempo while the next runs go on: the call starts its own trace (a traceparent header), so Tempo is asked for that trace id directly, the way you open a trace in Grafana from its id. A front sending traces (the Quality Console) is read the same way; a static front (the storefront, served by nginx) sends none. Every span carries that id, so the trace names for sure the APIs the call went through (the bank partner behind /api/hello, for instance) and the tables each one touched; the test's own SQL names its tables itself.

In short, two witnesses that do not see the same thing. The services write down what they do (the traces): they know who touched which table, but say nothing about what writes no notes. The database counts every table touched, by anyone, but cannot say who. The first gives the answer, the second checks that the first missed nothing.

So the page gives the answer from the traces (and the test's own SQL): what the test really went through, exact in one run. Under it, the coverage check compares it with what the database counters saw over the 10 runs: a table the database saw but no trace names means something untraced touched the database during the test. When no trace is found, the database's view becomes the answer, tables only. The Coverage check tab, under Run details, opens as soon as the runs end.

The number on each black table is the order in which the last run touched it; it is rewritten after each run, for the tables still candidate only. Under its name, each table says where it stands (candidate, ruled out or confirmed by the trace), whether the runs read it, wrote it or both, and in how many runs the database saw it move: a table seen in every run so far stays candidate, one missing from a single run is a colleague's.

Tables tagged live are written by the storefront (orders, stock, invoices, payments, audit); static ones are loaded once and only read.

The final answer merges the two witnesses once the runs are over and the traces read: the path the traces draw, each table marked named + counted (both agree: sure), named, not every run (the traces name it, some run did not see it move: a cache, a branch) or counted only, no trace (the database counted it in every run, no trace names it: a trigger, an old service), placed after the step it moved during. Each witness stays folded under it, one click away.

On a path, items of one kind in a row (3 or more: tables read by one API, fronts one after the other) fold into one stop, drawn as a stack, saying how many and what they are; click it to unfold, fold to fold back. In the grid, every front, API and table says in how many runs it was seen; when there are too many, the ones seen in the most runs stay and the others fold into a +N tile, click it to show them, Show fewer to fold them back. To keep the page fast, at most 150 tiles are drawn, the page draws again only when a run ends, and hidden items are never built.

Ownership: each service declares, in its telemetry, the team owning it (the team resource attribute, set in OTEL_RESOURCE_ATTRIBUTES); the observability platform files every span of the service under that team, and the tables its SQL spans name with it. Once the final answer is ready, each table of the test is owned (a span of a declared team names it) or an orphan: the database counted it, but its spans come from a service declaring no team, or there is no span at all. Searching Grafana by team never finds an orphan. APIs and message destinations get the same check: an API is owned when the service answering it declares a team, a destination when the services publishing or consuming it do. Owned tiles are green, with the team's name. The test Ownership: … a service whose team forgot to declare it shows it for real: the newsletter service is deployed without its team.

Resource optimization: once the traces are read, every endpoint answering during the test (a server span, with the service that called it) is counted one by one: an endpoint called more than once is listed as same call again (during one step: a loop, a missing cache), called by several services (the first could pass the answer along) or called again later (keep the first answer). The test Resource optimization: GET /api/hello, then the same greeting again shows it for real. Every SQL span of the test is counted the same way, with the step it ran during and the test's own SQL included. A table read more than once is listed with what to change: same query again (a service runs the same query twice during a step: a loop of queries or a missing cache), read by several services (the first could pass what it read along with its call), read again by one service (in another query or a later step: one read would do). Writes are left out. Messages too: a destination one service publishes to or consumes from more than once is listed as published again or consumed again. Each item shows the team owning the services using it. The ownership test is a good example: customers is read by the storefront API, the bank partner and the newsletter service, three times for one customer.

The two Demo tests are simulated: nothing is called, the runs are drawn from a fixed seed, so they look the same every time. Too many items goes through 5 fronts, a dozen APIs and almost every table, some steps played in only some runs. Hard to tell for 7 runs shares its minute with a nightly job reading 8 tables, 3 of them the test's own: until run 8 the job's tables are seen as often as the test's, then they drop out. Many of a kind sends the user through 10 fronts, then one export reads 29 tables while a trigger nobody traces writes the 30th: the stacks, and a table counted only.