Skip to main content

Escaped-defect learning loop

A blameless 30-minute review for every production defect — not just the S1s. An escaped defect is the most expensive evidence the engine produces about where it's weak; this loop is the concrete mechanism by which Production → Flow-state deposits quality capital instead of just closing tickets.

The review​

  • Fix first, learn second. The fix ships on the rapid-fix SLAs; the review runs within a week of the fix, never on its critical path.
  • 30 minutes, timeboxed. Whoever fixed it, the tester for the product, and Support (who found or fielded it). SME joins only if domain correctness is in question.
  • Blameless, structurally. Names appear as owners of actions, never as causes of defects. The root cause is always a category of the engine — a defect blamed is the next defect hidden.
  • Scaled, not skipped. For an S3/S4 the review may run async on the ticket. The format scales down; the four deposits below do not.

The four deposits​

The review isn't done until all four exist — they are the exit criteria, checked when the ticket closes:

  1. Root cause against the taxonomy — classified per the defect taxonomy: spec gap, AI-generated code defect, human review miss, test gap, or environment/data. This turns one defect into process feedback about which part of the engine to tighten.
  2. A regression test, merged — the defect enters the regression pack so it can never escape silently again. Tagged Repaired for RCRCRC selection.
  3. A gotcha in agent memory — symptom → root cause → fix → verification, one fact per file per the repo context & memory standard, so no agent (or human) re-derives the fix.
  4. A runbook entry — the known-failure-modes table in the runbook gains a row linked to the gotcha, so the next occurrence is diagnosed in minutes, not rediscovered.

The AI fork​

If the taxonomy call is AI-generated code defect, this loop hands over to the AI-defect incident runbook: run the provenance blast-radius query (what else did this agent/prompt/skill build?) and patch the source so the defect class cannot recur. The four deposits still apply.

Why every defect​

One defect is an anecdote; the category trend is a diagnosis. Escaped-defect rate by root-cause category is a headline quality view on the metrics platform — if "test gap" dominates, the test strategy needs work; if "spec gap" dominates, Definition does. Reviewing only the dramatic defects biases the data toward outages and away from the quiet, chronic weaknesses that cost more over a year.

This is the same compounding pattern as Flow-state: the loop is cheap because agents draft the test, the gotcha and the runbook row — humans verify and merge. The product gets harder to break with every defect it ever had.

Standards referenced: blameless postmortems (Google SRE), Orthogonal Defect Classification (via the defect taxonomy).