Skip to content

SecurityEvaluator does not run at the tool-call gate — command/prompt-injection can't screen tool-call arguments #2

Description

@initializ-mk

SecurityEvaluator.SupportedGates() returns only {GateInput, GateContext} (security_evaluator.go). So the four detectors it owns — jailbreakDetection, promptInjection, sqlInjection, commandInjection — and its customPatterns run only on inbound user text and injected context. They never run at:

  • GateToolCall — the arguments an agent is about to pass to a tool, and
  • GateOutput — the model's generated response.

Why this matters

The highest-value screen for commandInjection / sqlInjection is arguably the tool-call gate: that's where a compromised or manipulated agent actually emits rm -rf, a shell metacharacter payload, or a '; DROP TABLE string into a tool it's about to execute. Today an operator who enables:

security:
  commandInjection: { enabled: true, confidenceThreshold: 15, action: block }

reasonably expects tool-call arguments to be screened — but they aren't. The detector only sees the user's inbound prompt, not what the agent decided to do with it. Prompt-injection that surfaces in a tool call (the classic "exfiltrate via a crafted API call" path) sails through.

By contrast CustomRulesEvaluator does support GateToolCall (and honors a per-rule gates list), so a custom regex can screen tool args — but the built-in injection detectors can't.

Ask

Add GateToolCall (and consider GateOutput) to SecurityEvaluator.SupportedGates(), so the injection/jailbreak detectors screen tool-call arguments and outputs — matching where the risk actually is. The evaluator already reads payload.GetContent(), and ToolCallPayload.GetContent() returns the request body, so the detection path should work at that gate with little change.

If input+context-only is a deliberate scope, please document it (and the recommended "use customRules with gates: [tool_call]" workaround) so operators don't assume commandInjection guards tool execution.

Context

Found while auditing which StructuredGuardrails sections are enforced, and at which gates, for the initializ/forge platform guardrails overlay work. Companion to #1 (hallucination has no evaluator) and initializ/forge#289 (skill-constraint / approval wiring). Version: v0.12.0.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions