Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,12 @@ GITHUB_WEBHOOK_SECRET=your_webhook_secret
#REVIEW_LABELS_ALLOW_CREATE=false
#REVIEW_LABELS_MAX=3

# Optional: agentic /fix (off by default). When enabled, a write-access holder can reply /fix on a
# review finding thread and the bot commits a proposed fix to a thrillhousebot/* branch and opens
# a PR targeting the reviewed PR's branch. Requires the GitHub App to hold contents:write.
#REVIEW_FIX_ENABLED=true
#REVIEW_FIX_MAX_EDITED_FILES=10

# AI β€” any OpenAI-compatible API. Set AI_BASE_URL/AI_MODEL for your provider.
# Defaults below use DeepSeek; e.g. Alibaba Cloud Model Studio would be:
# AI_BASE_URL=https://dashscope-intl.aliyuncs.com/compatible-mode/v1
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ All notable changes to ThrillhouseBot.

## [Unreleased]

### Added

- **Agentic `/fix` command that opens a PR with the change** (opt-in, off by default): replying `/fix` on a review finding thread makes the bot draft the fix β€” across multiple files when the finding requires it β€” apply it as verbatim search/replace edits against the PR's current files, commit it to a bot-owned `thrillhousebot/fix-*` branch via the Git Data API, and open a clearly attributed PR targeting the reviewed PR's branch (merging the fix PR updates the original PR; a human always merges). Gated on `REVIEW_FIX_ENABLED=true`, the same write-access authorization as a manual `/review`, and the pause state; requires the GitHub App to hold `contents: write`. Fixes are all-or-nothing β€” an edit that no longer matches the files (e.g. after a new push) abandons the run with a thread reply instead of committing a partial change β€” capped at `REVIEW_FIX_MAX_EDITED_FILES` files (default 10), and unsupported on fork PRs. The fix PR and commit name the requesting user, link the finding thread, and carry an AI-generated disclaimer, keeping the project's "AI review is advisory" stance

## [0.5.0] β€” 2026-07-26

Review precision: confidence now decides where a finding lands, newly-added parsers and regexes are stress-tested for their own failure modes, and several classes of false positive are guarded at both the generator and the verifier. Operators gain configurable CI-gating and blocking strictness, structured skip reasons, and per-model generation parameters.
Expand Down
27 changes: 25 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ not a reaction.
| `/describe` | Suggest an improved PR title and description generated from the diff, as a comment to copy in (never overwrites the PR) | write |
| `/changelog` | Draft a CHANGELOG entry for the PR from the diff (Added/Changed/Fixed/Security…), as a comment to copy into `CHANGELOG.md` (never commits) | write |
| `/add-docs` | Generate docstrings/inline docs for the symbols changed in the PR, posted as committable suggestions (or a note with the drafted docs when a multi-line declaration can't be pinned to a single diff hunk) | write |
| `/fix` | Reply on a finding thread: draft the fix across the relevant files, commit it to a `thrillhousebot/*` branch, and open a PR targeting the reviewed PR's branch. Opt-in (`REVIEW_FIX_ENABLED=true`) and off by default | write |
| `/resolve` | Resolve ThrillhouseBot's outstanding finding threads on the PR | write |
| `/pause` | Silence the bot on the PR | write |
| `/resume` | Re-enable the bot on a paused PR | write |
Expand All @@ -100,7 +101,7 @@ the repository (or to be named in
AI budget.

**Pause** β€” while a PR is paused, ThrillhouseBot skips automatic reviews on new commits,
ignores `/review`, `/summary`, `/describe`, `/changelog`, and `/add-docs`, and does not answer
ignores `/review`, `/summary`, `/describe`, `/changelog`, `/add-docs`, and `/fix`, and does not answer
`@thrillhousebot` mentions (it replies once to say it is paused). `/resume` lifts the pause.
`/help` and `/resolve` keep working while paused.

Expand All @@ -111,6 +112,20 @@ declaration (spanning the whole signature when it wraps), so it only inserts doc
rewriting code. When a multi-line declaration can't be pinned to a single diff hunk, the bot
posts a note with the drafted docs to add manually instead of a committable suggestion. It
spends AI budget per run; operators can turn it off with `REVIEW_ADD_DOCS_ENABLED=false`.

**`/fix`** (opt-in, off by default) β€” replied on a review finding thread, the bot drafts the
change that resolves that finding β€” across multiple files when needed β€” commits it to a
bot-owned `thrillhousebot/fix-*` branch, and opens a pull request **targeting the reviewed
PR's branch**, so merging the fix PR updates the original PR. The fix PR and its commit are
clearly attributed to the bot, name the requesting user, link the finding thread, and carry
an AI-generated disclaimer; nothing ever lands without a human merging it, keeping the
[advisory stance](#responsible-use-and-security). Fixes are all-or-nothing (a fix whose
edits no longer match the files β€” e.g. after a new push β€” is abandoned with a reply, never
partially applied), capped at `REVIEW_FIX_MAX_EDITED_FILES` files, and unavailable on fork
PRs (the bot can only push branches to the repository it is installed on). Enabling it
requires the GitHub App to hold the **contents: write** permission and
`REVIEW_FIX_ENABLED=true`. Each run spends AI budget and is restricted to the same
write-access holders as `/review`.
<!-- docs:commands:end -->

## Quick start
Expand Down Expand Up @@ -203,7 +218,7 @@ Create a GitHub App before starting the bot; you'll need its credentials for `.e
|---|---|
| Webhook URL | `https://<your-host>/api/webhook` |
| Webhook Secret | Random string |
| Repository Permissions | Pull Requests: R/W, Checks: R/W, Contents: Read, Issues: R/W, Actions: Read, Commit Statuses: Read |
| Repository Permissions | Pull Requests: R/W, Checks: R/W, Contents: R/W (write only used by the opt-in `/fix`; Read suffices otherwise), Issues: R/W, Actions: Read, Commit Statuses: Read |
| Subscribe to Events | Pull Request, Issue comment, Pull request review comment |
| Identifying & authorizing users | Enabled (for dashboard login) |
| Callback URL | `https://<your-host>/api/auth/callback` |
Expand Down Expand Up @@ -247,6 +262,8 @@ will change per provider:
| `REVIEW_CONVERSATIONAL_REPLIES_ENABLED` | Answer `@thrillhousebot` mentions in PR threads (including finding replies) with an AI reply | `true` |
| `REVIEW_ADD_DOCS_ENABLED` | Allow the on-demand `/add-docs` command to generate docstrings as committable suggestions | `true` |
| `REVIEW_DIAGRAM_ENABLED` | Include an opt-in Mermaid control-flow diagram in the PR summary | `false` |
| `REVIEW_FIX_ENABLED` | Allow the opt-in `/fix` command on finding threads: the bot commits a proposed fix to a `thrillhousebot/*` branch and opens a PR targeting the reviewed PR's branch (requires the App to hold **contents: write**) | `false` |
| `REVIEW_FIX_MAX_EDITED_FILES` | Cap on how many files one `/fix` may edit or create; a broader fix is rejected with a reply instead of partially applied | `10` |
| `REVIEW_MAX_INPUT_TOKENS` | Per-call input-token budget for review calls; large PRs are split into batches that each fit it. Bounded by the active model's input cap (see [Per-model AI settings](#per-model-ai-settings)). `0` disables token budgeting | `48000` |
| `REVIEW_OUTPUT_BUFFER_TOKENS` | Tokens reserved out of the input budget for the model's response | `8192` |
| `REVIEW_MAX_AI_CALLS` | Cap on AI calls per review (batch calls plus the final summary call); files that still don't fit are reported by name as omitted | `6` |
Expand Down Expand Up @@ -495,6 +512,12 @@ AI review is advisory. The model can be wrong in both directions: it raises
false positives and misses real bugs. Treat its findings as suggestions and
confirm them yourself before acting.

The opt-in `/fix` command is held to the same stance: it is off by default, only
runs when a write-access holder explicitly requests it on a specific finding, only
writes to bot-owned `thrillhousebot/*` branches, and delivers its change as a
clearly attributed pull request that a human reviews and merges β€” the bot never
pushes to your branches or merges anything itself.

Pull request diffs are sent to whatever endpoint you configure, so use an HTTPS
endpoint with an API key, and read the provider's data-retention policy before
sending it private code.
Expand Down
11 changes: 7 additions & 4 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,10 +204,10 @@ sequenceDiagram

| Package | Responsibility | Notable classes |
|---|---|---|
| `webhook/` | Receives GitHub events, verifies the HMAC signature, decides whether an event triggers a review (trigger filters, per-PR pause state, auto-review rate limit), acks slash/mention commands with πŸ‘€, runs the comment commands (`/help`, `/summary`, `/describe`, `/changelog`, `/add-docs`, `/resolve`, `/pause`, `/resume`), and schedules finding-feedback capture on review-thread replies | `WebhookController`, `WebhookVerifier`, `TriggerDetector`, `ReviewTriggerFilter`, `AckReactionService`, `CommentCommandService`, `PrPauseService` |
| `review/` | Orchestrates a review: plans the token budget, calls the AI layer (single-call or map-reduce), maps findings to a risk level and review state, writes the summary comment, optionally labels the PR, answers maintainer replies/mentions in PR threads, and persists maintainer finding feedback (πŸ‘/πŸ‘Ž / reply heuristics) for a future learnings pipeline | `ReviewOrchestrator`, `ReviewDispatcher`, `DiffBudgetPlanner`, `FindingPipeline`, `AutoReviewRateLimiter`, `ReviewDiffFormatter`, `FollowUpAnalyzer`, `FindingFeedbackCaptureService`, `FindingFeedbackService`, `PrSummaryGenerator`, `PrLabeler`, `MaintainerReplyService`, `MaintainerReplyDispatcher` |
| `review/ai/` | The LangChain4j layer: streams or batches model responses, parses findings, runs a second pass to verify them, applies generation/reasoning customizers, and writes conversational replies | `PrReviewer`, `AiReviewService`, `ChatModelCustomizers`, `FindingVerifier`, `FindingVerificationService`, `ReviewResponseParser`, `ReplyAssistant` |
| `github/` | Talks to the GitHub REST and GraphQL APIs: app auth, pull requests, reviews, check runs, comments, labels, reactions (create + list), and reading the repo instructions file | `GitHubAuthClient`, `GitHubReviewClient`, `GitHubCheckRunClient`, `GitHubLabelClient`, `GitHubReactionClient`, `InstructionsResolver` |
| `webhook/` | Receives GitHub events, verifies the HMAC signature, decides whether an event triggers a review (trigger filters, per-PR pause state, auto-review rate limit), acks slash/mention commands with πŸ‘€, runs the comment commands (`/help`, `/summary`, `/describe`, `/changelog`, `/add-docs`, `/fix`, `/resolve`, `/pause`, `/resume`), and schedules finding-feedback capture on review-thread replies | `WebhookController`, `WebhookVerifier`, `TriggerDetector`, `ReviewTriggerFilter`, `AckReactionService`, `CommentCommandService`, `PrPauseService` |
| `review/` | Orchestrates a review: plans the token budget, calls the AI layer (single-call or map-reduce), maps findings to a risk level and review state, writes the summary comment, optionally labels the PR, answers maintainer replies/mentions in PR threads, authors opt-in `/fix` PRs on finding threads, and persists maintainer finding feedback (πŸ‘/πŸ‘Ž / reply heuristics) for a future learnings pipeline | `ReviewOrchestrator`, `ReviewDispatcher`, `DiffBudgetPlanner`, `FindingPipeline`, `AutoReviewRateLimiter`, `ReviewDiffFormatter`, `FollowUpAnalyzer`, `FindingFeedbackCaptureService`, `FindingFeedbackService`, `PrSummaryGenerator`, `PrLabeler`, `MaintainerReplyService`, `MaintainerReplyDispatcher`, `FixService` |
| `review/ai/` | The LangChain4j layer: streams or batches model responses, parses findings, runs a second pass to verify them, applies generation/reasoning customizers, writes conversational replies, and drafts `/fix` edits | `PrReviewer`, `AiReviewService`, `ChatModelCustomizers`, `FindingVerifier`, `FindingVerificationService`, `ReviewResponseParser`, `ReplyAssistant`, `FixGenerator` |
| `github/` | Talks to the GitHub REST and GraphQL APIs: app auth, pull requests, reviews, check runs, comments, labels, reactions (create + list), Git Data writes (blobs/trees/commits/refs, used only by `/fix`), and reading the repo instructions file | `GitHubAuthClient`, `GitHubReviewClient`, `GitHubCheckRunClient`, `GitHubLabelClient`, `GitHubReactionClient`, `GitHubGitDataClient`, `InstructionsResolver` |
| `dashboard/` | The live UI backend: OAuth login (in-memory sessions), WebSocket broadcaster (`review.stream` / `review.batch`), review session persistence, and finding-feedback aggregates | `AuthResource`, `DashboardSessionStore`, `SessionEventBroadcaster`, `ReviewSessionRepository`, `DashboardResource` |
| `config/` | Wiring: the outbound HTTP client, the review thread pool, typed config, active-model settings (caps, generation params), fail-fast startup validation, and the shared bot-identity used to recognize the bot's own activity | `HttpClientProducer`, `ReviewExecutorProducer`, `ThrillhouseConfig`, `ActiveModelSettings`, `StartupConfigValidator`, `BotIdentity` |
| `frontend/` | The Next.js dashboard, built to a static export and served by Quarkus | β€” |
Expand All @@ -216,6 +216,9 @@ sequenceDiagram

PR reviews carry inline comments and suggestions; check runs carry pass/fail
status for branch protection (no inline annotations on the check run itself).
The opt-in `/fix` command is the only path that writes repository content, and
it only pushes bot-owned `thrillhousebot/fix-*` branches and opens an attributed
PR targeting the reviewed PR's branch β€” a human always merges.

**AI call budget** β€” a review that reports findings makes **two** model calls
by default: the review call plus a skeptical verification pass
Expand Down
2 changes: 1 addition & 1 deletion manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"default_permissions": {
"checks": "write",
"pull_requests": "write",
"contents": "read",
"contents": "write",
"issues": "write",
"actions": "read",
"statuses": "read"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ void validate() {
"thrillhousebot.github.webhook-secret");
requirePresent(problems, aiApiKey, "AI_API_KEY", "quarkus.langchain4j.openai.api-key");
validateReviewBudget(problems, config.review());
validateFixSettings(problems, config.review().fix());
validateCiGating(problems, config.review());
validateBlockingStrictness(problems, config.review());
validateModelSettings(problems, config.ai().models());
Expand Down Expand Up @@ -159,6 +160,18 @@ private static void validateReviewBudget(
}
}

/**
* Validates the {@code /fix} file cap even while the feature is disabled, so an operator who
* mistyped the value discovers it at boot rather than on the first {@code /fix} after enabling.
*/
private static void validateFixSettings(List<String> problems, ThrillhouseConfig.FixConfig fix) {
if (fix.maxEditedFiles() < 1) {
problems.add(
"REVIEW_FIX_MAX_EDITED_FILES must be >= 1 (thrillhousebot.review.fix.max-edited-files): "
+ fix.maxEditedFiles());
}
}

/**
* Validates every per-model settings entry β€” not just the active model's β€” because an operator
* who wrote an invalid value has expressed clear intent, and rejecting the typo at boot beats
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,30 @@ static String normalizeCiGating(String raw) {
LabelsConfig labels();

DiagramConfig diagram();

FixConfig fix();
}

/**
* Opt-in agentic {@code /fix} command. When {@link #enabled()} a write-access holder can reply
* {@code /fix} on a review finding thread; the model drafts the change across the relevant files
* and the bot commits it to a bot-owned branch and opens a pull request targeting the reviewed
* PR's branch. Off by default: the bot never pushes code unless the operator opts in, keeping the
* "AI review is advisory" stance β€” the fix PR is a proposal a human reviews and merges.
*/
interface FixConfig {
/** Master switch β€” the whole feature is off unless this is {@code true}. */
@WithDefault("false")
boolean enabled();

/**
* Upper bound on how many files one {@code /fix} may edit or create. A fix that needs more is
* rejected with an explanatory reply instead of being partially applied β€” a suspiciously broad
* fix deserves a human, not a bigger commit.
*/
@WithName("max-edited-files")
@WithDefault("10")
int maxEditedFiles();
}

/**
Expand Down
Loading