Skip to content

review-pr: harden Pre-Verdict Audit against comment-quality halo effect - #57

Draft
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
factory/review-pr-audit-halo-eval-019ff1e1
Draft

review-pr: harden Pre-Verdict Audit against comment-quality halo effect#57
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
factory/review-pr-audit-halo-eval-019ff1e1

Conversation

@warp-agent-staging

Copy link
Copy Markdown
Contributor

Overview

Documentation-only change to .agents/skills/review-pr/SKILL.md, limited to the ## Pre-Verdict Audit section's Comments bullet.

Why

A reviewer using this skill let a comment's writing quality and technical accuracy ("well-written, explains a genuinely hard bug") stand in for actually checking it against the target repo's commenting guidelines, and missed confirmed rule violations as a result.

What changed

The Comments bullet now:

  • Requires enumerating every added or changed comment (doc or inline) one by one with its file:line, so none get skimmed past as part of a holistic impression.
  • Falls back to the project's existing commenting distribution (density, tone, what existing comments explain vs. omit) when the repository defines no written guidelines.
  • Explicitly forbids treating a comment's writing quality, technical accuracy, or the importance of the issue it describes as a mitigating factor for a guideline violation or a mismatch with the codebase's norms.

No rule names or categories are introduced, so the skill stays generic across repos with and without explicit commenting guidelines. Nothing else in the file changed — the schema, severity labels, safety rules, evidence rules, suggestion-block constraints, and diff-line-annotation contract are untouched, as is the Tests bullet.

Conversation: https://staging.warp.dev/conversation/c1500141-7376-47dd-a966-91d76eb38fd4
Run: https://oz.staging.warp.dev/runs/019ff4a3-2387-705e-83bb-ef65542b64cb

This PR was generated with Oz.

Require enumerating every added or changed comment individually with its
file:line and checking each against the repository's commenting guidelines
(or, absent guidelines, the project's existing commenting norms). Forbid
treating a comment's writing quality, technical accuracy, or the importance
of the issue it describes as a mitigating factor for a guideline violation.

Co-Authored-By: Warp <agent@warp.dev>
@warp-agent-staging warp-agent-staging Bot added the factory-ab-eval Factory A/B evaluation replay label Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

factory-ab-eval Factory A/B evaluation replay

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant