Skip to content

review-pr: harden Pre-Verdict Audit against comment-quality halo effect - #71

Draft
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
factory/review-pr-audit-comment-enumeration
Draft

review-pr: harden Pre-Verdict Audit against comment-quality halo effect#71
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
factory/review-pr-audit-comment-enumeration

Conversation

@warp-agent-staging

Copy link
Copy Markdown
Contributor

Overview

Documentation-only change to a single section of .agents/skills/review-pr/SKILL.md: the ## Pre-Verdict Audit Comments bullet.

Why

A reviewer using this skill let a comment's writing quality and technical accuracy ("well-written, explains a genuinely hard bug") stand in for actually checking it against the target repo's commenting guidelines, and missed confirmed rule violations as a result.

What changed

The Comments bullet now:

  • Requires enumerating every added/changed comment individually, with its file:line, so none are skimmed past as part of a holistic impression.
  • Requires checking each one individually against the repository's commenting guidelines, and falling back to the codebase's own commenting distribution when the repository defines none.
  • Explicitly forbids treating a comment's writing quality, technical accuracy, or the importance of the issue it describes as a mitigating factor.

No specific rule names or categories are introduced, so the skill stays generic across repositories with or without written commenting guidelines. Nothing else in the file is touched — schema, severity labels, safety rules, evidence rules, suggestion-block constraints, and the diff-line-annotation contract are unchanged.

Require the audit to enumerate each added/changed comment individually
with its file:line, and forbid treating a comment's writing quality,
technical accuracy, or the importance of the issue it describes as a
mitigating factor for a guideline violation. Also covers repositories
with no written commenting guidelines by falling back to the existing
commenting distribution of the codebase.

Co-Authored-By: Warp Agent <agent@warp.dev>
@warp-agent-staging warp-agent-staging Bot added the factory-ab-eval Factory A/B evaluation replay label Aug 13, 2026
@warp-agent-staging

Copy link
Copy Markdown
Contributor Author

This PR was generated with Warp.

View run View conversation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

factory-ab-eval Factory A/B evaluation replay

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant