EPIC: run the full version matrix and publish the §9 report

[myselfsiddharth]

Context

This is the run that produces the company-deciding number. Track 2 is closed — two consecutive vertical FAILs (Week-0 observability in docs/research/census-week0/A8-DECISION.md, Track-2 counterparty in docs/research/vertical-search/DECISION.md, accepted as ADR-0004). docs/README-narrative.md section 6 states the consequence plainly: the company now depends on Track 1 proving the mechanism.

This issue is the epic — the coordination point. It is not implementation. Everything it needs is built by the issues below; this one executes the run, checks the output is complete and honest, and hands it to the memo.

Prerequisites (all must be closed)

Milestone	What it delivers
M1	A version matrix that provably boots and seeds deterministically
M2	A real 8–12 step gate task, recorded live and compiled with audited assertions
M3	Live replay per version, a real repair model with measured tokens, a measured fresh baseline, and a persistent confidence-tracking cache
M4	Repeat runs per version and the cache hit-rate metric

Do not start this until they are closed. A number produced from a partial harness is worse than no number, because it will be quoted.

What to do

1. Run the full matrix at the chosen repeat count. Record the exact command, the date, the model id, the effort setting, and the image digests used.
2. Generate the report: npm run gate:report. Commit the report artifacts to a dated directory so the number is permanently reproducible, even though experiments/gate-v1/out/ is normally gitignored — say in the PR why you are committing these specific files.
3. Check the report contains every PRD section 9 quantity, each either computed with a real denominator or explicitly no_data:
   • step-level replay-validity
   • task-level success with ≤2 repairs
   • repair cost vs fresh cost, tokens and wall-clock
   • self-heal success rate
   • time-to-repair
   • amortized tokens/task over N runs, as a plottable series — section 10 calls this plot the demo
   • cache hit-rate over N
   • canary-leak count, which must be 0
4. Check the totals against section 9's protocol: at least 42 runs and 400 step-executions, or an explicit statement of how far short the matrix substitution falls and why. Do not quietly under-deliver on the protocol.
5. State the proxy caveat prominently, quoting docs/gate/testbed.md: version-bump churn is a proxy for organic production churn — arguably harsher, and different in kind. The pivot brief section 4 requires this asterisk; a number presented without it is a misrepresentation.
6. Do not adjudicate PASS / EXTEND / FAIL here. That is the memo's job, and separating the data from the verdict is what makes the verdict credible.

Section 9 thresholds — for the memo, not for this issue

• Step-validity ≥ 80% and task success (≤2 repairs) ≥ 90% → thesis holds, build Phase 2
• Step-validity < ~50% or mean repair cost ≥ 70% of fresh → thesis is dead, stop before the raise
• In between → extend the window, add a second site, decide on data

Note that docs/INTEGRITY-AUDIT.md B-05 and D-03 flag these as proposed gates, not accepted results, and that the pivot brief's single "~50%" kill line and PRD section 9's fuller set are not yet reconciled — an open question in docs/prd/pivot-brief-v0.3.md. The memo has to resolve which document controls.

Constraints

• Every number comes from a measured metrics row. No hand-computation, no spreadsheet, no rounding a no_data into a figure. CONTRIBUTING: fabricating a number is a fireable error.
• If part of the matrix could not run, report the reduced denominator explicitly. A five-version result honestly labelled beats eight versions of assumption.
• Do not tune anything mid-run to improve the outcome. If a fix is needed, fix it, discard the partial run, and start over — and say that you did.

How to test

The output is the deliverable, so verify it rather than the code:

npm run gate:matrix -- --runs <n>
npm run gate:report

# every section accounted for, with its status
node -e "const r=require('./experiments/gate-v1/out/report/report.json'); \
  console.log(r.row_counts); \
  console.log(r.metrics.map(m=>[m.name,m.status,m.value,m.numerator+'/'+m.denominator].join(' | ')).join('\n'))"

npm run test:canary   # canary-leak count must be 0

Confirm every NDJSON line parses, the run count matches the intended runs × versions minus recorded skips, and no row carries a dry-run marker.

Before you open the PR

• All prerequisite issues closed
• Full matrix run completed; command, date, model id, effort, and image digests recorded
• Every section 9 quantity present, each computed or explicitly no_data
• Run and step-execution totals stated against section 9's ≥42 / ≥400
• Proxy caveat stated prominently
• Canary-leak count is 0
• Report artifacts committed to a dated directory, with justification
• No PASS/FAIL verdict in this PR — data only
• Branch track1/b4-gate-run