Skip to content

/qa can finish green having merged nothing, and the file that would explain why dies with the worktree #463

Description

@orioltf

What this is

Two findings from the review of PR #461, left out of it on purpose: neither is a defect that PR introduced, both are decisions rather than typos, and one of them asks /qa to do something it has never done.

They are filed together because they compound, and the compound is worse than either.

1. /qa writes its durable block and nothing commits it

uat-prep writes <artifacts_dir>/<slug>/qa-checks.md — the block naming every check that came back unresolved and whether an install ran. #430's criterion 10 requires that block to be durable, in those words:

the block is durable — it lands in a file under <artifacts_dir>/<slug>/, the repo-relative Session dir, and a block that exists only in a node's console output does not satisfy this criterion.

The letter is met. The purpose is not. Nothing in unic-dlc-qa.yaml ever commits that file: grep -c 'git add\|git commit\|git push' returns 0 for that Box and 8 for unic-dlc-build.yaml. The file is written into an Archon worktree, and the worktree is removed by archon complete or by /cleanup. The block dies with it.

/build does not have this problem for the same reason it never came up: open-pr stages SESSION/report.md onto the PR branch. /qa has no open-pr and no report node — that absence is exactly why criterion 10's amendment gave uat-prep the Write tool in the first place.

Why this was not fixed in #461: giving /qa a commit is new behaviour, not a correction. It puts a QA record on the feature branch and into the merge, which is a product decision about what a merged branch should carry. The criterion's author specified uat-prep writing the file, and #461 did exactly that.

2. /qa declares no evidence_policy, so a run that merges nothing still reports success

unic-dlc-build.yaml declares evidence_policy: { required: true }. /qa, /pr-review and /explore declare none — grep -c evidence_policy returns 4, 0, 0, 0 across the four Boxes. That is ADR-0034's explicit scope: "No other Box gains an evidence gate in this ADR."

Since #461, /qa's merge node is gated on $test.output.result == 'pass'. So with an unresolved test:

  1. merge's when: is false, so merge is skipped.
  2. merge is the last of the Box's ten nodes.
  3. A skipped node is terminal, and no node reports on it.
  4. The run reaches completed.

A /qa run therefore finishes green having merged nothing, and its exit status says so to nobody.

The compound, which is the actual reason to file this

Set gates.qa: afk, which is the mode /qa is built to support, and run it against a project whose config declares no test:

  • uat-gate and merge-gate are skipped — both are when: gate == 'hitl'.
  • uat-prep still runs and writes qa-checks.md, which explains exactly what went wrong.
  • merge is skipped on the floor.
  • The run reports completed.
  • archon complete removes the worktree, and qa-checks.md goes with it.

No human is at any gate, nothing is merged, no explanation survives, and the run's status is success. Finding 1 destroys the evidence and finding 2 removes the signal; either one alone leaves a way to notice.

This is the same shape #430 was opened about — a result that reads green while nothing was checked — arriving through the Box that merges rather than the Box that builds.

What would settle each

Not proposed as decided. Both need grilling, and the second may be a two-line change or may be out of scope entirely.

  • For 1: either /qa commits qa-checks.md (and then: onto which branch, at which node, and does it belong in the merge?), or the block goes somewhere already durable, or criterion 10's durability requirement is honestly narrowed to the two Boxes that can satisfy it. The third is a real option and should not be dismissed for looking like a retreat.
  • For 2: either /qa gains an evidence_policy of its own, which reopens what ADR-0034 deliberately closed, or merge stops being silent when its when: blocks — a node after it that reports the non-merge would cost less and change no engine behaviour.

Out of scope

  • Anything about /build's evidence gate. It works, and ADR-0034 §4 already records the one cost it accepts (a red build still opens a PR).
  • /pr-review and /explore having no evidence_policy. Neither merges nor certifies anything, so neither has a silent-success failure of this shape.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app:unic-archon-dlcbugSomething isn't workingneeds-specsEnough info from reporter; ready to write a spec

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions