feat(manifest): run manifest coverage contract for review (#367) - #520
Merged
Conversation
First slice of issue alibaba#367 (run manifest coverage contract): the data model and state machine only. Not yet wired into the agent or CLI, so existing review/scan output is unchanged. Introduce the versioned, immutable RunManifest (schema ocr.run-manifest/v1) and a concurrency-safe ManifestBuilder that tracks per-file coverage (selected/completed/reused/failed/waived) and freezes into a terminal state. - terminal state derived solely from coverage sets, never comments/warnings (complete/partial/failed/skipped) - Finalize sweeps any undecided selected item to failed/unknown so no item is silently dropped - single-mutex builder: first terminal state wins, frozen after Finalize, nil-receiver safe - fixed failure classification enum with an unknown catch-all - redaction floor on failure/waive reasons (strip secrets, cap length) as a single write entry so callers cannot bypass it - 22 unit tests, race-clean Refs: issue alibaba#367 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address findings from the concurrency / JSON-contract / PR#306-coupling adversarial review of the manifest data model (still slice 1; not wired to agent or CLI). - SetSweepClass: Finalize can classify undispatched items as cancelled/budget instead of a blanket unknown (the one real model gap the review found) - ItemID(fingerprint)=SHA-256 canonical mint helper; an item_id is never a raw fingerprint, keeping the resume cross-reference explicit and mix-ups caught - sanitizeReason: strip control/ANSI chars, coerce valid UTF-8, redact quoted secret values, guarantee single line - Finalize returns deep-copied coverage slices so the frozen snapshot is never aliased across the two outlets - RegisterSelected: nil-safe (lazy-init map) + documents that only the post-deletion/post-filter dispatchable set may be registered +7 unit tests (29 total), race-clean. Refs: issue alibaba#367 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ure (shard ②d) - Freeze per-mode input identity (mode + resolved_base/head + exact_range + source_artifact_sha256) via diff.ResolveInput/commitParents, and repository identity via RemoteIdentity/canonicalRemote (credential-free). - Add rule_config_sha256 and runtime_config_sha256 over an allowlist of non-secret fields using a length-prefixed SHA-256 framework (no tokens/URLs). - Replace SetRunLevelFailure(bool) with structured SetRunFailure(class, reason) and set ManifestInput.mode; fill execution.* (ocr version, provider, model, concurrency, config hashes). - Thread error returns through Finalize/WriteSessionEnd (main review path surfaces them; skip/all-failed/scan paths hardened in follow-up). - Tests: manifest_hash, canonical_config, git_resolve. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lassification Merged review themes A/B/E from the 07-22 consolidated assessment. Theme A — Finalize / session_end delivery errors no longer swallowed: - agent.go no-files path returns the Finalize error instead of nil (A1) - agent.go loadDiffs failure joins the Finalize error via errors.Join (A2) - session.Finalize uses sync.Once + cached finalizeErr: written exactly once, concurrency-safe, and every caller replays the same result so a retry cannot falsely report success (A3) - scan/agent.go wires both Finalize call sites to surface the error (A4) Theme B — canonicalRemote rewritten (internal/diff/git.go): - keep the port (u.Host, not u.Hostname) so endpoints differing only by port stay distinct (B1) - split scp syntax on the first ':' so an '@' inside the path survives (B2) - recognize local/file/Windows/UNC remotes and omit identity rather than misparsing a path as a host (B3; local-remote policy still open) Theme E — main_task-empty is now a sentinel (errMainTaskEmpty) classified via errors.Is instead of matching error text. Theme D (TOCTOU) deferred to shard 4 per issue alibaba#367 open-issues OI-12. Tests: go build ./... + go vet + go test ./... all green (23 pkgs). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mal path The success-path Finalize wiring used `ferr != nil && err == nil`, so when the review (or scan) failed AND session_end also failed to persist, the persistence error was dropped and only the dispatch error surfaced — the caller never learned the session/manifest was not saved. Join both with errors.Join when both occur (matching the loadDiffs path), so a persistence failure is always reported even alongside a dispatch failure. This closes the last gap in the OI-10 contract. - internal/agent/agent.go: review normal path - internal/scan/agent.go: scan normal path (+ errors import) Tests: go build ./... + go vet + go test ./... all green (23 pkgs). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 使用冻结 manifest 统一 review JSON、文本与退出状态\n- session CLI 和 viewer 展示五集合覆盖并兼容 legacy/aborted\n- 补充本地 mock、跨出口一致性及安全验收用例
验收用例:configuration 分类(run 级 sweep + item 级映射)、budget/timeout/panic 混合 partial 隔离、跨出口一致性改为规范化原始字节比对、flag 校验失败无产物断言。 代码修复:sanitizeReason 先剥控制字符再脱敏(堵控制字节绕过)、失败项异分类二次标记报冲突错误、source_artifact_sha256 按 item_id 去重并稳定排序、sortItems 改 SliceStable 对齐设计用词。 全仓 go test 23 包通过。
# Conflicts: # internal/llmloop/loop.go
覆盖 issue alibaba#367 验收标准 provider transition:resume 时 provider/model 改变后,子 manifest 记录当前值而非继承父运行,并经 parent_run_id 链接父会话以支持审计。用 mock client,不依赖真实 provider key。
Contributor
|
🔍 OpenCodeReview found 7 issue(s) in this PR.
|
| // fixed token ceiling. session_end embeds the complete run manifest and can | ||
| // legitimately exceed the former 10 MiB scanner limit on very large reviews. | ||
| func readJSONLLines(r io.Reader, visit func([]byte)) error { | ||
| reader := bufio.NewReader(r) |
Contributor
There was a problem hiding this comment.
Performance: The stated motivation for this refactor is to handle lines exceeding the former 10 MiB scanner limit. However, bufio.NewReader uses a default 4096-byte buffer. For very large JSONL lines (multi-megabyte manifest records), ReadBytes will repeatedly grow its internal buffer through many small allocations and copies, causing significant GC pressure. Consider using bufio.NewReaderSize with a larger initial buffer to reduce allocation overhead for the expected large-line use case.
Suggestion:
Suggested change
| reader := bufio.NewReader(r) | |
| reader := bufio.NewReaderSize(r, 64*1024) |
统一聚合预算停止时的 coverage、status 与退出码。传播 session writer 初始化错误,并补齐 merge first-parent 输入身份及回归测试。移除代码注释中的外部设计文档引用。
# Conflicts: # cmd/opencodereview/output.go # internal/agent/agent.go # internal/llmloop/loop.go # internal/scan/agent.go
冲突解决: - cmd/opencodereview/flags.go: 接受 upstream 删除(alibaba#625 迁移到 Cobra), 将 --max-tokens-budget 的说明文案迁移到 shared_flags.go 的 addConcurrencyFlags - internal/viewer/store.go: SessionSummary 取双方字段并集(manifest 终态 + CommentCount); LoadSession 同时保留 CommentCount 统计与 readErr 返回 - internal/viewer/templates/sessions.html: 同时保留 Status 与 Comments 两列 另修复两处自动合并后编译失败: - store.go ListSessions: 本分支已将 scanner 循环重构为 readJSONLLines 回调, alibaba#627 新增的 continue 落入闭包,改为 return - compat_test.go: 补 runReview 兼容包装(alibaba#625 已拆为 reviewCmd + executeReview) go build / go vet / go test ./... 全部通过。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #367.
Adds a versioned, immutable
RunManifesttoocr reviewthat records per-item coverage (selected/completed/reused/failed/waived) and an authoritative terminal state (complete/partial/failed/skipped). On successful persistence, CLI JSON andsession_end.run_manifestexpose the same frozen manifest, so a partial review can no longer be mistaken for ordinary success or a complete no-findings result.Built on top of PR #306's resumable sessions; this is a narrow output-contract follow-up, not a parallel persistence system.
internal/session/manifest.go:RunManifestvalue object (schemaocr.run-manifest/v1),ManifestBuilder(single state table, unified transition entry,RegisterSelected/MarkCompleted/MarkReused/MarkFailed/MarkWaived),SealSelected(closes the selected denominator before dispatch),Finalize(elapsed) (RunManifest, error)with hard validation, structuredRunFailure+RunFailureClassenum, andsanitizeReasonredaction floor.internal/agent/agent.go: a pre-dispatch pass (registerCoverage) registers all post-filter, non-deleted items and seals them before resume reuse or goroutine dispatch;markCompleted/markReused/markFailedrecord per-item outcomes. The live--max-tokens-budgetgate recordsFailureBudgetas the pending-item failure cause, so only undispatched/pending items are swept tofailed(budget);BudgetExceeded()continues to feed the summary and warning.SetRunFailureis reserved for run-level stop causes at the trigger source (inputfailure onloadDiffs,internalfailure onregisterCoverage).initManifestinitializes requested input and execution metadata; resolved input and repository identity are captured from the actual run and attached before finalization.internal/diff/git.go:ResolveInputfreezes resolved base/head/exact range per input mode, including first-parent merge input to matchGetDiff --diff-merges=first-parent;RemoteIdentity/canonicalRemoteproduce a credential-free repository identity by canonicalizing the origin URL tohost[:port]/path(dropping userinfo/query/fragment) and SHA-256ing the result; local remotes omit identity.internal/session/history.go:NewManifestBuildermounts the builder onSessionHistory;Finalizenow returnserror(was void) and usessync.Oncesosession_endis written exactly once while replaying the first error on every call.internal/session/persist.go:WriteSessionEndwritessession_endas the last physical JSONL record, embeds the frozen manifest underrun_manifest, and surfaces write/flush/close errors.cmd/opencodereview: review JSON exposesmanifest; top-levelstatususes the four terminal states (scanstays legacy); aggregate-budget reporting remains available throughsummary.budget_exceeded; failed runs include the sanitized run-failure cause or failed/selected counts; text output no longer showsLooks good to me.forpartial/failed;session list/showand the viewer prefer a validsession_end.run_manifestand otherwise display legacy/unknown state without fakingcomplete.Design (key invariants)
Finalize. On successful persistence, CLI JSON andsession_end.run_manifestserialize the same frozen manifest value. JSON output no longer derives review status from warnings.SealSelectedcloses the selected denominator after the pre-dispatch pass; resume-reused and to-be-dispatched items enter the same frozen set, soselected = completed ∪ reused ∪ failed ∪ waivedalways holds.run_failureis recorded at the trigger source (input= diff resolution,internal= scheduler/invariant). It is never inferred fromctx.Err(). Ordinary aggregate budget exhaustion is a controlled coverage stop wired to--max-tokens-budget: it sets a pendingFailureBudgetcause and sweeps only uncovered items, without creatingrun_failure. Contract-level recorders forcancelledand run-leveltimeout, plusrun_failure.classification=budgetfor a genuine counter/scheduler anomaly, remain available but have no live trigger in this release (no new SIGINT handler or global deadline).completemay have zero or many findings.sanitizeReasonprovides a second redaction floor for URL credentials, Bearer/Basic tokens, credential-like assignments, and control characters. CLI output suppresses rawsubtask_errorwarnings when a manifest is present. Existing session checkpoint and conversation persistence behavior is unchanged.Finalize/WriteSessionEndreturn errors up the stack across all exit paths—the no-files path, theloadDiffs-failure path, the normal dispatch path, and the scan path. On the normal path, dispatch and persistence errors are reported viaerrors.Join. A persistence failure does not rewrite the frozen manifest; it surfaces as a delivery error with a non-zero exit code.parent_run_id; immutable-ref drift rejection is unchanged and remains out of scope.Terminal state
completefailedempty, norun_failurepartial0 < failed < selected, norun_failurefailedrun_failurepresentskippedrun_failurewaiveditems count as covered, so a run containing onlycompleted/reused/waiveditems iscomplete. Findings count never affects the state.scanis unchanged and continues to emit the legacystatusvalues; the two commands do not yet share the same status contract.Compatibility
ocr review --format jsonintentionally migrates top-levelstatusfrom the legacysuccess/completed_with_errors/completed_with_warningsvalues (and feat(agent): add token/tool-call cost guardrails to the review path #508'sbudget_exceeded) to the manifest terminal statescomplete/partial/failed/skipped.summary.budget_exceededor inspectmanifest.coverage.failed[].classification == "budget"; the terminal state remains coverage-derived instead of being overwritten by the budget flag.partial), and exits non-zero when every selected item failed (failed).scanretains its legacy output contract.Upstream integration
This branch is synchronized through
upstream/main@c391892and preserves the upstream behavior merged while #367 was in progress:BudgetExceeded()signal; its formerstatus=budget_exceededoutput is adapted to the manifest terminal-state contract described above.task_done(status=DONE)marks an item completed;FAILED, provider errors, missing completion, and structuredMainLoopStopreasons keep the item failed and flow into manifest coverage.How to test
Checklist
go test -race -count=1 ./...greengo vet ./...cleangofmt -landgo mod tidyproduce no diffgo build ./...succeeds; CLI smoke checks passinput/internal), per-item timeout/budget/panic, skipped, resume (parent_run_id+ reused), provider transition,task_donecompletion semantics, structured main-loop stops (max_rounds/empty_rounds/compression), cancellation contract,SealSelectedlifecycle, failed-finalize rollback, conflicting/idempotent transitions, invalid failure class, empty waiver reason, cross-exit manifest consistency, malformed/legacy/aborted session display, large JSONL/non-EOF reader behavior, and security redactionmanifestandsession_end.run_manifestserialize the same frozen manifest valueOut of scope (deliberately)
scanis not wired to the v1 manifest (Non-Goal; it passes a nil manifest andemitRunResultis nil-safe) and keeps its legacystatusvalues.waivedoutput semantics.cancelledand run-leveltimeouthave contract coverage but no live source in this release—no new SIGINT handler or global deadline is added. Normal aggregate budget exhaustion is live through--max-tokens-budgetand is represented as per-itemfailed(budget)coverage;run_failure.classification=budgetis reserved for a separate run-level counter/scheduler anomaly and has no live trigger.parent_run_id; ref-drift rejection and "diff by immutable SHA" are deferred.complete.