Skip to main content

Testability & observability design checklist

A short checklist applied at the Design Review and Tech Selection gates so systems are cheap to test and observe before any code is generated. Retrofitting seams, determinism and instrumentation onto thousands of lines of agent-built code is expensive rework; specifying them costs one checklist pass. Agents then build inside a shape that is testable by construction.

The checklist​

  • Seams for isolation — every external dependency (payment, rating engine, document store, third-party API) sits behind a named interface, so components test in isolation and contract tests have a contract to test. The seams are visible in the design pack, not discovered during Build.
  • Deterministic time and IDs — no naked now() or random() in domain logic. Clocks and ID generation are injected, so tests control them. Non-negotiable in insurance logic: renewal dates, cooling-off periods, claim time-bars and MTA effective dates are all time arithmetic, and untestable time arithmetic is where silent defects live.
  • Feature flags — risky or incomplete behaviour ships behind flags with a named owner and an expiry; the flag mechanism comes from the scaffold, not per-product invention. A flag past its expiry is tech debt.
  • Health endpoints — liveness and readiness per service, wired into deployment and the demo environment, so "is it up" is a probe, not a Slack question.
  • Structured logging — per the observability standard: structured, correlated across service boundaries, no secrets or PII (UK GDPR — test data rules apply to log data too).
  • Golden-signal metrics — latency, traffic, errors, saturation instrumented from the scaffold on every service, so Refinement's performance pass and production SLOs read from instruments that already exist.

Where it's applied​

  • Design Review — the design pack answers each item per component: where the seams are, what is flagged, what states and failures are observable. A design that can't answer isn't buildable yet.
  • Tech Selection sign-off — the chosen stack and reference architecture must provide the mechanisms (dependency injection or equivalent seams, a flag provider, OpenTelemetry support, health-check conventions), so every product inherits them from the scaffold rather than hand-rolling them.
  • Downstream — the merge gate lint catches naked time/random calls in domain code; Production Readiness verifies the observability baseline is live. The gates check what this checklist designed in.

Why before code generation​

Agents amplify whatever shape they are given. Given seams, injected clocks and an instrumented scaffold, they produce testable, observable code at volume; given none, they produce untestable code at the same volume — and the test strategy inherits the bill.

Standards referenced: ISO/IEC 25010 (testability, analysability), Google SRE (golden signals), OpenTelemetry.