Seeing Like a Dashboard

You never operate a system. You operate a representation of it: a metrics endpoint, a log stream, an SLO, a TPM quote, a spreadsheet, an org chart. The representation exists because the system is too large to hold in working memory. James Scott's Seeing Like a State is the long version of what happens next: you manage the representation, the system reorganises to look correct in the representation, and the two decouple. Most of this blog is that one failure mode at a different layer. Here is the mechanism, with the cases named.

A surface is a lossy projection

Every legibility instrument is a function from a high-dimensional system to a low-dimensional summary, and that function is not invertible. A mean discards the distribution. A p50 discards the tail. A daily rollup discards the 03:00 spike. Head-based trace sampling discards the rare path, which is the one you needed. A monotonic counter tells you a thing happened, never why, which is the whole content of monitoring is not understanding.

The losses are not edge cases; they are the point. A representation that preserved everything would be the system, illegible again. Concretely: a 99.9% availability SLO over a one-hour window stays green through a thirty-second total outage that dropped every request, because the window averaged it away. The number is honest about what it computes and silent about what it dropped, and you read the silence as health.

An aggregate is a hash of the system: cheap to compute, impossible to invert. You are steering by the hash.

Make the surface a target and the system optimises the surface

Goodhart's Law is the mechanical consequence: once a measure becomes a target, the optimiser moves the measure by whatever path is cheapest, and the cheapest path is rarely the goal you meant.

• Target line coverage and you get tests that execute lines without asserting on them, which is the core of who tests the AI's tests: green that means "ran," not "checked."
• Target a latency SLO and shedding or fast-failing the slow requests improves the metric while degrading the service. The dashboard recovers; the users do not.
• Target ticket close-time and tickets close and reopen.

None of this requires bad faith. It is a local optimisation against an observable, and the observable is the proxy, not the target. When every competitor optimises the same public proxy, the proxy stops discriminating and the field converges, which is why data-driven firms converge: institutional isomorphism is Goodhart run across an industry. Reward hacking in an RL loop is the same thing with a gradient: the model exploits everything the reward function cannot see.

The surface is self-authored

Now the move that turns a Scott problem into a security problem. The instrument is almost always emitted by the subject it describes. The health endpoint is code inside the service. The access log is written by the process being accessed. The agent's tool-call trace is produced by the agent.

GET /healthz  ->  200 {"status":"ok"}
              # returned by the process whose health is the question

The reporter and the reported share an address space and a failure domain. So the same defect that breaks the system breaks or forges its report. A kernel-resident rootkit filters the syscalls before the monitoring agent's eBPF probe sees them, and the alert never fires. An AI writes the code and the test that certifies it, so the certificate is circular. The audit log is writable by the service it audits, so it is a story its author can edit. A rooted node attests that it is healthy with a key the rooted OS holds.

The TPM quote exists precisely to break this: it is signed by hardware outside the OS's address space, so it is the one report in the stack the subject cannot author. That is the entire reason it is worth anything. The RATS architecture formalises the split: the attester, the verifier, and the relying party must be distinct, because a verifier inside the verified is not a verifier.

If the reporter and the reported share a failure domain, the report is not evidence. It is a claim from an interested party.

The bias runs toward the visible

Because the projection is lossy, the optimiser can only act on what survives it, and that creates a systematic bias, not just gaps. A cost cut is a line item; the foregone option has no field, so the organisation defends the measurable and starves the rest. Value that operates by being unnecessary, slack, redundancy, a costly signal, has no column to live in and gets cut as waste. A loss function penalises only what it observes and the model exploits everything else. Same shape, three domains.

The danger side is the mirror image. The failure lives in the path the instrument dropped: the unsampled trace, the unlogged agent-to-agent call, the request the SLO window averaged out. The surface is green right up to the incident because the incident was in the residual, the part that did not fit on the screen.

The same mechanism runs above the code, in how the work is tracked. A status board is a denormalised replica of state whose source of truth lives elsewhere: the commits, branches, PRs, CI runs, and deploys. Nothing keeps the replica in sync except people updating it by hand, so it goes stale the way any cache without invalidation does.

"In progress" decays into a union of active, blocked, waiting on review, and abandoned, which is to say it carries no information. A query the stale replica cannot answer falls back to the slow path: ask a person. Enough of those and engineering becomes the dashboard, and the cost (reload context, diff what changed, serialise it, resume) is accounted nowhere.

The fix is the standard one for a cache that drifts: stop maintaining a second copy by hand and derive the view from the source, status as a read-model projected over the work artifacts.

You cannot fix a projection with more projections

The reflex when a dashboard misleads you is to add a panel. That raises resolution on the same lossy, self-authored surface; it does not open a channel to ground truth. The fixes change the source of the signal, not its pixel count:

• Anchor outside the subject's failure domain. Hardware attestation, an external specification, mutation testing that grades the tests, provenance the actor cannot rewrite, a verifier on a different host and trust domain. Stop reading the self-report; get a report the subject did not write.
• Sample the territory directly. Replay real traffic and diff behaviour, fuzz off-distribution, run the thing instead of modelling it. When the answer is only available by running, make running cheap and let outcomes select.
• Keep the human in the path, not beside it. Vigilance over a green stream decays into rubber-stamping, so a watcher is theater; a participant stays in contact, and reversibility beats oversight.
• Own the instrument. Whoever emits the metric, keeps the log, and sets the SLO controls the story. The funding model is the governance model; a surface controlled by an interested party is that party's account of you.
• Shrink the system until you do not need the instrument. Keep it small enough to inspect directly, and publish the reasoning, not just the result, so what propagates is the territory and not a render of it.

None of these removes legibility. You cannot operate what you cannot see. They keep one channel to the system that does not pass through the system's own description of itself.

The discipline

Hold four facts about every surface at once: it is necessary, it is lossy, it is gameable, and it is usually self-authored. The instant the clean green panel invites you to forget any of them is the instant to distrust it. Read the dashboard, then go sample the part of the system it cannot resolve, because that is where the next failure and the next opportunity both already are.

Seeing is not understanding, and a system you can only see has not been understood.