Skip to content

review-pr: harden Pre-Verdict Audit against comment-quality halo effect - #67

Draft
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
factory/review-pr-preverdict-halo-019ff1e1
Draft

review-pr: harden Pre-Verdict Audit against comment-quality halo effect#67
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
factory/review-pr-preverdict-halo-019ff1e1

Conversation

@warp-agent-staging

Copy link
Copy Markdown
Contributor

Summary

Documentation-only change to .agents/skills/review-pr/SKILL.md, scoped to the ## Pre-Verdict Audit section.

A reviewer using this skill let a comment's writing quality and technical accuracy ("well-written, explains a genuinely hard bug") stand in for actually checking it against the target repo's commenting guidelines, and missed confirmed rule violations as a result.

Change

The Comments bullet of the audit now:

  • Requires enumerating every added/changed comment (doc or inline) one by one with its file:line, so none get skimmed past as part of a holistic impression.
  • Requires checking each one individually against the repo's commenting guidelines, or — when the repo defines none — against the commenting distribution of existing code (density, tone, what existing comments explain vs. omit).
  • Explicitly forbids treating writing quality, technical accuracy, or the importance of the described issue as a mitigating factor for a guideline violation.

No rule names or categories are introduced, so the skill stays generic across repos with or without written commenting guidelines. The Tests bullet, the section heading, its intro sentence, and the rest of the file (schema, severity labels, safety rules, evidence rules, suggestion-block constraints, diff-line-annotation contract) are untouched.

Require the comment audit to enumerate each added/changed comment
individually with its file:line, and forbid treating a comment's writing
quality, technical accuracy, or the importance of the issue it describes
as a mitigating factor for a guideline violation. Also covers
repositories with no explicit commenting guidelines by falling back to
the commenting distribution of existing code.

Co-Authored-By: Warp Agent <agent@warp.dev>
@warp-agent-staging warp-agent-staging Bot added the factory-ab-eval Factory A/B evaluation replay label Aug 13, 2026
@warp-agent-staging

Copy link
Copy Markdown
Contributor Author

This PR was generated with Warp.

View run View conversation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

factory-ab-eval Factory A/B evaluation replay

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant