Skip to content

feat(agents): pin the four inherit toolkit reviewers to opus (−10.6% pr-reviewer spend) - #139

Open
DJRHails wants to merge 2 commits into
mainfrom
toolkit-agents-opus
Open

feat(agents): pin the four inherit toolkit reviewers to opus (−10.6% pr-reviewer spend)#139
DJRHails wants to merge 2 commits into
mainfrom
toolkit-agents-opus

Conversation

@DJRHails

Copy link
Copy Markdown
Owner

Operator decision materialized as a PR — merge or close; it is deliberately not self-merged because it changes which model runs four review agents.

The gantry pr-reviewer cost anatomy (gantry#942, N=109 runs / 18,815 requests, 2026-08-25) measured the §1 review battery at 28.6% of $/review. Within it, $847 of $1,142 bills at fable rates ($10/$50 per MTok) through these four agents' model: inherit, while their siblings code-reviewer and code-simplifier have pinned model: opus ($5/$25 — half price) all along.

This pins the remaining four:

  • pr-test-analyzer
  • silent-failure-hunter
  • comment-analyzer
  • type-design-analyzer

Projected saving: 10.6% of pr-reviewer spend ($4.2k/mo at the current ~$40k/mo run rate).

The quality case for parity rather than degradation: code-reviewer — the battery's highest-spend, highest-yield finder — has run on opus the whole time these numbers were gathered, and Opus 5 is the exact model every review already falls back to fleet-wide whenever fable is unavailable (whole review windows have run on it, verdicts accepted). Still a model change on finding-producing agents, so it ships only on your merge. Trivially reversible (four one-line frontmatter edits).

Note these files are the vendored copies consumed everywhere the dotfiles bake (gantry worker images pick it up via the djrhails-dev rebuildProbe on the next image freshen), so the pin applies to interactive sessions too, not only pr-reviewer.

via gantry cost-opt worker

The gantry pr-reviewer cost anatomy (DJRHails/gantry#942, N=109 runs /
18,815 requests) measured the review battery at 28.6% of $/review, of which
$847 of $1,142 bills at fable rates via these four agents' model: inherit —
while code-reviewer and code-simplifier have pinned model: opus all along.
Opus 5 is half fable's price and the same tier every review already falls
back to fleet-wide when fable is unavailable. Projected saving ~10.6% of
pr-reviewer spend (~$4.2k/mo at the current run rate).
Review of PR #139 found no P1-P3 issues; one informational hazard: the
README's re-sync instruction (copy upstream *.md wholesale) would silently
revert the deliberate model: opus pins back to upstream's model: inherit,
re-introducing the fable-rate billing the pins exist to stop. Record the
local modification next to the re-sync instruction so a future refresh
preserves it.
@DJRHails

Copy link
Copy Markdown
Owner Author

Review Summary

Direct single-pass review (4 changed lines of agent frontmatter — under the multi-agent thresholds).

Findings

No P1–P3 findings. The diff does exactly what the description says: pins model: opus on the four model: inherit toolkit reviewers, matching code-reviewer and code-simplifier.

# Severity Finding Resolution
1 P4 The subagents README's re-sync instruction ("copy the updated *.md files") would silently revert all six opus pins on the next upstream refresh, re-introducing the fable-rate billing this PR eliminates. Fixed: README now records the local modification next to the re-sync step (8f65558).
2 P4 silent-failure-hunter.md's description line is not strict-YAML (unquoted colon-space inside a plain scalar). Pre-existing, untouched by this PR, and Claude Code's lenient frontmatter parser loads it fine — noted only in case a strict-YAML tool ever consumes these files. Noted, no change.

Verified

  • Diff is exactly 4 one-line model: inheritmodel: opus edits; all six vendored agents now carry exactly one model: opus line (valid frontmatter value).
  • The "siblings pinned all along" claim holds: at the vendoring commit (7857935), code-reviewer/code-simplifier were already opus while the four changed agents were inherit.
  • No stale references: nothing else in the repo documents these agents' model, and the review-pr SKILL.md carries no model text to rot.
  • gantry#942 (the cited cost anatomy) exists and is open. The fable-vs-opus spend split and projected saving are the issue's own measurements — not re-derived here.
  • Quality pipeline: no CI workflows in this repo; the pre-commit hook suite (gitleaks, TruffleHog, glassine checks, symlink/behaviour suites) ran green on the fix commit.

Commit

8f65558 — docs(subagents): guard the opus pins against a plain re-vendor

The model-pin decision itself is untouched and remains yours to merge or close, as the PR intends.

Verdict: approve

via gantry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant