Skip to content

bug(unic-archon-dlc): /pr-review produced eight findings, wrote none, and exited 0 #465

Description

@orioltf

What happened

unic-dlc-pr-review iteration 3 against ADO PR 5854 reviewed the diff, produced eight findings across both axes, wrote none of them to disk, and exited 0 reporting Workflow completed successfully. Run a7e55042fcfea725602ed4cc0787b260, 2026-09-05 19:16–19:23.

Reported by the DS-43028 session; every measurement below was re-taken here against ~/.archon/archon.db and the run's worktree before filing.

The evidence

The review node ran 332 s and returned. Its node_output is 14038 characters and holds the whole review — findings in YAML with per-finding confidence, a Spec verdict, and a positives block. Its last three lines are the defect:

Reconciled the 18 priors while waiting: 15 fixed, 1 still open (`83baac2a11f5`, …), 3 echoed contradictions carried.

Standards axis still running.

The node returned while it believed one of its two axes had not finished. It never wrote findings/, findings.json, comment.md or threads.json.

Everything downstream then degraded to well-formed zeros rather than erroring:

Node Output
review 332 s, 14 KB of findings in node_output, nothing on disk
synthesize {"critical":0,"important":0,"minor":0}
reconcile {"iteration":3,"verdicts":0,"unmatched_priors":18}
post "Nothing to post. The review nodes upstream produced no output." — refused, correctly

Run status completed, anyFailed: false, exit 0.

On disk, task-unic-dlc-pr-review-1788628547333/workflows/component-story-check/pr-review/ holds changed-files.txt, diff.patch, intent-brief.md, prep.json and an empty findings/. Iterations 1 and 2 of the same PR each hold one file there.

Reproduce the read:

sqlite3 ~/.archon/archon.db \
  "SELECT length(json_extract(data,'\$.node_output')) FROM remote_agent_workflow_events
   WHERE workflow_run_id LIKE 'a7e55042%' AND event_type='node_completed';"

Why this is p1 and not a flake

A Box that reviewed and lost its review is indistinguishable, at the exit code, from a Box that found nothing. Both report completed. Both leave findings.json absent or empty. Nothing between review and post treats "the upstream node produced no artefact" as different from "the upstream node found no defects".

The only thing that made it visible was post refusing — and a refusal reads as "nothing to say", not as "something broke". On this run a human happened to read the node output. Nobody would on an AFK run.

This is the same failure shape as #463 one Box over: a green run whose green means nothing. It also lands on run 3 directly. #457's positive-control comment says a mechanism that cannot be shown to fire scores inert rather than pass; /pr-review is one of the mechanisms run 3 will be read through, and it just failed that bar on its first observation.

What it cost, concretely

The eight findings were recovered by hand out of the event log and judged against the code. Six were real and are fixed in 12aa37d on feature/43028_component-story-check. The one that mattered: gap.excluded.length === 0 was reached by two causes and reported as one, so a story file that imports the component and exports no story printed "no story file imports X" while X.stories.tsx sat on disk importing it.

Without someone reading a 14 KB node output out of SQLite, six real findings including that one would have shipped, under a green run.

Not yet decided — this needs grilling before it has criteria

Three questions, and the second is the one that decides the shape:

  1. Why did review return with an axis unfinished? Whether it is a fan-out join that does not wait, a subagent whose completion is not awaited, or a turn budget reached mid-write is unmeasured. The event log has the node's own transcript; nobody has read it against the Box's node definition yet.
  2. Should the absence of an artefact be an error at all, and whose? synthesize reading an empty findings/ and emitting three zeros is defensible in isolation — an empty review directory is what a clean review also produces. The distinction has to be made by something that knows review ran, which is either review writing a sentinel or synthesize checking that the upstream node reported writing one. Those are different designs with different blast radii.
  3. Does /pr-review get an evidence_policy? ADR-0034 explicitly scopes the evidence gate to /build and says no other Box gains one. /qa can finish green having merged nothing, and the file that would explain why dies with the worktree #463 already reopens that question for /qa. If the answer there is yes, this is the second instance and the ADR is the thing to amend once rather than twice.

Out of scope

  • post's behaviour. It refused and it was right: publishing would have meant inventing 18 verdicts against a review that produced none.
  • Anything about /build's evidence gate. It works.
  • Retro-posting iteration 3's findings to PR 5854. They were recovered, judged and fixed by hand, and the record is on the PR's summary thread marked author-written. No iteration=3 Box marker exists and none should be inferred.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app:unic-archon-dlcbugSomething isn't workingneeds-specsEnough info from reporter; ready to write a specp1High priority

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions