feat: SkillTrojan defense — multi-skill composition trace with turn-boundary reset (Closes #1802) - #1829
Merged
Merged
Conversation
…oundary reset #1802 (needs-work rework): Runtime composition tracer that monitors combined output when 2+ skills execute in the same turn, detecting payload reconstruction patterns. Rework fixes from PR #1818 review: 1. Wire reset_tracer() into the turn boundary (agent/turn_context.py reset_for_turn) so the singleton does NOT accumulate across the entire session — only skills loaded within the SAME turn are composed. 2. Tighten the cross-skill conditional heuristic: the regex now requires the conditional to be inside indented code or a fenced code block, NOT plain prose. This prevents false positives on skill documentation that mentions another skill's completion. Heuristics (all deterministic, no ML): - Base64 fragment assembly across skills - Cross-skill conditional references (code-block only) - URL assembly from cross-skill variable fragments Closes #1802 Co-Authored-By: Hermes Evolution <evolution@hermes.ai>
Contributor
૮ >ﻌ< ა ci reviewran on 5207547 all good! |
Closed
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rework of the SkillTrojan composition tracer (#1802, needs-work) addressing both issues from PR #1818 review.
Rework Fixes
1. Turn-boundary reset (the regression):
The previous PR used a module-level singleton (
_tracer = CompositionTracer()) that accumulated skills across turns — butreset_tracer()was NEVER called in production code (only in tests). This violated the "2+ skills in the SAME turn" invariant, causing spurious cross-turn blocks on legitimate skills.Fix: Wired
reset_tracer()intoagent/turn_context.pyat the turn boundary (reset_for_turn), right alongside_tool_guardrails.reset_for_turn(). Now the tracer only composes skills loaded within the current turn.2. Tightened conditional heuristic (false positives):
The previous regex fired on plain prose mentioning another skill's completion (e.g. "after skill_X completed"), hard-blocking skills whose documentation mentions other skills.
Fix: The conditional regex now requires the pattern to be inside indented code or a fenced code block (
```). Plain prose is NOT flagged. This is implemented via_INDENTED_CONDITIONAL_RE(matches leading whitespace) and_extract_fenced_code()(extracts text inside ``` blocks for checking).Heuristics (all deterministic, no ML)
if skill_X completedpatterns (code-block only)Files
tools/skill_composition_tracer.py— 345 lines (new module)tests/tools/test_skill_composition_tracer.py— 246 lines (20 tests)tools/skills_tool.py— +14 lines (check_composition wiring)agent/turn_context.py— +9 lines (reset_tracer at turn boundary)Tests
20 tests pass, including:
Line count
614 lines total — exceeds 200-line self-merge cap. This is a single coherent security feature that cannot be meaningfully split further (the tracer, call-site wiring, and turn-boundary reset are one atomic unit). Needs human review.
Closes #1802
Co-Authored-By: Hermes Evolution evolution@hermes.ai