Skip to content

Fix over-triggering description on respond-to-pr-comments-in-blocklist - #69

Merged
harryalbert merged 2 commits into
mainfrom
factory/respond-to-pr-comments-description
Aug 13, 2026
Merged

Fix over-triggering description on respond-to-pr-comments-in-blocklist#69
harryalbert merged 2 commits into
mainfrom
factory/respond-to-pr-comments-description

Conversation

@warp-agent-staging

@warp-agent-staging warp-agent-staging Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

A Warp engineer reported that the respond-to-pr-comments-in-blocklist skill was read every time PR comments were handed to the agent, even when the user only wanted the comments fetched/displayed or the underlying code fixed, with no intent to post anything to GitHub.

Root cause

The frontmatter description had three defects:

  1. It led with "Walk users through PR review comments, fetching and displaying them first when needed" — almost identical wording to the bundled pr-comments skill's own description ("Fetch and display GitHub PR review comments for the current branch"). Neither description stated a boundary, so this skill's description matched any context where PR comments were in play, including when they were already fetched. The body does state the correct boundary ("If comments are already visible in the conversation... continue from that context"), but that's only visible after the very read this was supposed to prevent — skill selection happens on name + description alone.
  2. "Use when responding to PR review comments" is ambiguous between replying on GitHub and acting on the feedback (e.g. just making the code changes). The skill is actually a heavy, multi-gate ask_user_question workflow whose terminal action is posting [Warp Agent]-prefixed replies via the GitHub REST API and resolving threads via the resolveReviewThread GraphQL mutation — none of that was visible in the description.
  3. There was no negative trigger / exclusion, unlike sibling skills in the same catalog (e.g. the bundled claude-api skill's TRIGGER when: ... SKIP: ... block).

Changes

  • Rewrote the frontmatter description to lead with the terminal action (posting replies / resolving threads), name the interactive nature of the workflow, and state explicit exclusions pointing at pr-comments and at code-only requests.
  • Added a short "When not to use this skill" section near the top of the body mirroring the same exclusions, for readers who do get past the description.
  • The body's workflow itself is unchanged — it was not the source of the bug.

This PR does not rename the skill directory or its name: frontmatter. A rename was considered (the -in-blocklist suffix is dead weight — it refers to Warp's terminal block list UI, not any filtering mechanism in this repo) but was dropped: this skill is redistributed to warp-server and warp via their checked-in skills-lock.json, and the regeneration tooling (scripts/update_common_skills_lock, via skills add ... -s '*') is additive rather than reconciling — a reproduction against a downstream lock confirmed a rename would leave both the old (now-removed) path and the new path in the regenerated lock, breaking lock-based installs. Renaming is left for separate follow-up work that also fixes the lock reconciliation gap. With no rename, this PR has no redistribution/lockfile impact.

Verification

This is a behavioral/prompt-engineering change, not something covered by CI. I attempted to use the bundled create-skill skill's trigger-eval tooling (scripts/run_eval.py) to quantitatively confirm the new description stops the misfires, but it shells out to the claude CLI (Claude Code) to run each eval query, and claude is not installed in this environment — so I could not run it. Verification here is a manual read-through:

  • The new description leads with the terminal action ("post agent-authored replies on GitHub and resolve the review threads"), so it no longer overlaps with pr-comments's "fetch and display" framing.
  • The new description explicitly says "Use only when the user wants to reply to or resolve review threads on GitHub" and "Skip when the user only wants comments fetched or displayed (use pr-comments), or only wants the code changes made without posting anything back to GitHub" — directly addressing the reported false-trigger cases.

Related

There is an existing stale draft PR, #34, which also renames this skill (as part of a broader "make common-skills Warp-agnostic" migration) but leaves the description unchanged. Per the requester, that PR is not being coordinated with here.

… fix over-triggering description

The description led with wording that overlapped almost entirely with
the bundled pr-comments skill (fetching/displaying PR comments), had
no negative trigger, and used the ambiguous verb "responding" without
naming the actual terminal action (posting GitHub replies and
resolving review threads via ask_user_question-gated approval). This
caused the skill to be read whenever PR comments were in play, even
when the user only wanted comments fetched/displayed or code changes
made without any GitHub posting.

Also drops the misleading -in-blocklist suffix, which referred to
Warp's terminal block list UI and had no connection to any filtering
mechanism in this repo.

Co-Authored-By: Warp Agent <agent@warp.dev>
@warp-agent-staging

Copy link
Copy Markdown
Contributor Author

This PR was generated with Warp.

View run View conversation

@harryalbert
harryalbert marked this pull request as ready for review August 13, 2026 14:44
@warp-for-oss

warp-for-oss Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

@harryalbert

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

The requester decided to drop the directory/name rename to avoid
downstream skills-lock.json reconciliation risk in warp-server and
warp (skills add is additive, not reconciling, so a rename would have
left a stale entry pointing at a removed path). Keeps the over-
triggering description rewrite and the new When not to use this
skill section, which is the actual fix and reviewed clean.

Co-Authored-By: Warp Agent <agent@warp.dev>

@warp-for-oss warp-for-oss Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR renames the PR-comment response skill and narrows its selection description so it is reserved for the interactive workflow that posts GitHub replies and resolves review threads. The added "When not to use this skill" section mirrors that trigger boundary in the body.

Concerns

  • No blocking concerns found.
  • No approved or repository spec context was provided, so there is no spec drift to flag.
  • Security pass found no security-sensitive behavior changes; this is documentation/skill metadata only.
  • Comment/prose audit found the changed skill description and added guidance aligned with the repository's skill format. No tests were added or changed.

Verdict

Found: 0 critical, 0 important, 0 suggestions

Approve

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@warp-agent-staging warp-agent-staging Bot changed the title Fix over-triggering description and rename respond-to-pr-comments-in-blocklist Fix over-triggering description on respond-to-pr-comments-in-blocklist Aug 13, 2026
@harryalbert
harryalbert merged commit d6ebdf7 into main Aug 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants