Skip to content

review-pr: harden Pre-Verdict Audit against comment-quality halo effect - #64

Draft
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
review-pr-preverdict-comment-audit
Draft

review-pr: harden Pre-Verdict Audit against comment-quality halo effect#64
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
review-pr-preverdict-comment-audit

Conversation

@warp-agent-staging

Copy link
Copy Markdown
Contributor

Summary

Hardens the ## Pre-Verdict Audit section of .agents/skills/review-pr/SKILL.md so a reviewer can't let a comment's writing quality or technical accuracy substitute for actually checking it against the target repo's commenting guidelines.

ticket_id: eval-full-20260813-019ff1e1-a170-74a5-b5d0-a20775b5947b-planny-r1
ticket_source: adhoc
requester: daniel@warp.dev

Changes

  • Comments audit step now requires enumerating every comment (doc comment or inline) individually with its file:line, rather than a holistic pass.
  • When the repo has no written commenting guidelines, the audit now falls back to judging comments against the existing commenting distribution in the codebase (density, tone, what's explained vs. omitted).
  • Explicitly states that a comment's writing quality, technical accuracy, or the subtlety/importance of the issue it describes does not excuse a confirmed guideline violation.
  • No other part of the file (schema, severity labels, safety rules, evidence rules, suggestion-block constraints, diff-line-annotation contract) was touched.

Verification

Documentation-only change. Confirmed via git diff / git diff --stat that the change touches exactly one file (.agents/skills/review-pr/SKILL.md) with a single line replaced (1 insertion, 1 deletion), and that the replacement text matches the requested wording byte-for-byte.

Co-Authored-By: Warp Agent agent@warp.dev

…lo effect

Require enumerating every comment individually (with file:line) during
the Pre-Verdict Audit instead of a holistic pass, and explicitly forbid
treating a comment's writing quality, technical accuracy, or the
importance of the issue it describes as a mitigating factor against a
confirmed guideline violation.

Co-Authored-By: Warp Agent <agent@warp.dev>
@warp-agent-staging warp-agent-staging Bot added the factory-ab-eval Factory A/B evaluation replay label Aug 13, 2026
@warp-agent-staging

warp-agent-staging Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

@/tmp/pr_comment_QklSZX

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

factory-ab-eval Factory A/B evaluation replay

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant