Compliance season used to be a CSV-collection exercise. Each tool exported what it thought was the truth, the auditor reconciled by spreadsheet, and the answers from IAM did not match the answers from EDR for the same employee. We rebuilt the substrate.

What we measure

  • Coverage: for each control, which assets are covered by which tool, with what last-observation timestamp.
  • Quality signals: auditor pushback rates, reopen rates after remediation, time-to-evidence per finding.
  • Throughput: how long each finding spends in each stage of the pipeline, and where it queues.

The shape of the mess

IAM, EDR, vuln management, and code review each owned their own state and their own export format. Their disagreements were not bugs (they reflected which tool last observed which event) but to the auditor they read as contradictions.

Four tools, four truths, one employee. The contradiction was never in the data; it was in having no shared timeline to read it against.

One append-only event spine

Every producer appends events to a single, versioned, append-only spine. Consumers (audit dashboards, compliance reports, incident response, risk register) project what they need without ever rewriting source-of-truth. Replays are cheap; schema evolution is explicit.

A normalised event looks like this:

{
  "event_id":      "7a13...",
  "subject_id":    "U-429801",
  "control":       "AC-2",
  "asset":         "iam_account",
  "stage":         "review_passed",
  "ts":            "2026-01-07T12:44:05Z",
  "source_system": "iam_v3"
}
IAM EDR Vuln scanners Code review Append-only event spine Audit dashboards Compliance reports Incident response Risk register
Many security tools in, one spine, many compliance consumers out.
Before: per-tool exportsAfter: one spine
Source of trutheach tool's own stateone append-only log
Reconciliationauditor by spreadsheetconsumers project a view
Disagreementsread as contradictionsordered by timestamp
New reportnew export to wire upreplay the log
Audit prepweeksdays

What it enabled

The biggest change was organisational, not technical. When every team looked at the same numbers for the same controls, the weekly compliance review went from "whose data is right" to "what do we do next." Audit preparation dropped from weeks to days.

What it cost

Schema-evolution discipline is the bill. Adding a field is fine; removing one is a contract change with notice. Retention contracts are explicit and tier-by-tier. The spine itself is boring infrastructure; the politics of letting go of per-tool dashboards was the hard part.

The spine was easy. Persuading four teams to stop owning their own truth was the project.

Compliance used to mean collecting a CSV from every security tool and reconciling them by hand. IAM, EDR, vuln management, and code review each held their own state in their own format, so the same employee showed up four different ways. The auditor read those as contradictions; they were really just four tools observing different events at different times.

The rebuild. Every producer appends events to one versioned, append-only spine. Each event is normalised: an id, a subject, a control, an asset, a stage, a timestamp, and the source system. Nothing ever rewrites source-of-truth.

Consumers project, they do not own. Audit dashboards, compliance reports, incident response, and the risk register each read the spine and build the view they need:

  • Coverage: which assets each control covers, with the last-observation timestamp.
  • Quality: auditor pushback rates, reopen rates after remediation, time-to-evidence.
  • Throughput: how long a finding sits in each pipeline stage, and where it queues.

Replays are cheap; schema evolution is explicit. A new report is a new projection, not a new integration.

What changed

The biggest win was organisational, not technical. When every team read the same numbers for the same controls, the weekly review went from "whose data is right" to "what do we do next." Audit prep dropped from weeks to days.

The cost is discipline. Adding a field is free; removing one is a contract change with notice. Retention is explicit and tier-by-tier. The spine is boring infrastructure; the hard part was getting four teams to give up their own dashboards.

Sources