feat: floor-test gate for BenchJack defense (Closes #1809) - #1821
Closed
Lexus2016 wants to merge 1 commit into
Closed
feat: floor-test gate for BenchJack defense (Closes #1809)#1821Lexus2016 wants to merge 1 commit into
Lexus2016 wants to merge 1 commit into
Conversation
Wires null-agent floor scores into the merge gate. A metric that a null-agent (which solves nothing) can pass is NOT trusted — if the PR's metrics fall at or below the floor, the merge is blocked with a diagnostic. No silent bypass. Closes #1809 Co-Authored-By: Hermes Evolution <evolution@hermes.ai>
Contributor
૮ >ﻌ< ა ci reviewran on 5ac5699 all good! |
Owner
Author
|
Blocked by code-review: dead code. |
Closed
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated evolution PR for issue #1809.
Summary
Implements the pre-merge floor-test gate for the BenchJack defense chain (parent #1267). Wires null-agent floor scores into the merge gate so a metric a null-agent can pass is NOT trusted.
Changes
scripts/evolution_floor_gate.py(new): gate logic that blocks merge if any PR metric is at or below floor * (1 + margin). Loads floor scores from JSONL (eval_baseline.py output) or uses pinned defaults. No silent bypass.tests/scripts/test_evolution_floor_gate.py(new): 10 tests covering block/pass logic, margin threshold, missing metrics, JSONL loading, CLI entry point.Integration
The module exports
check_floor_gate(pr_scores, floor_scores)which is designed to be called fromevolution_merge_gate.py'scheck_merge_policy_with_quality(). The actual wiring into merge_gate.py is a HIGH_RISK_PATH change that requires human review — this PR provides the gate logic ready for integration.Verification
Closes #1809