[bug] llm_classifier: agentic tool-continuation turns silently fall through to the capable target because the judge is handed the newest tool result

[thossullivan]

Symptom

On agentic multi-turn sessions (tool-call and tool-result loops), every continuation turn bypasses classification and is served by the capable target. There is no warning above debug level. In one production session, request 1 classified weak and the following 45 requests all landed on the strong model.

Reproduction

Self-contained script attached below. It uses a stub upstream, needs no credentials, and nothing leaves localhost. It configures an llm_classifier route (judge plus weak and strong targets on one OpenAI-compatible client, base_threshold = 0.75, session_affinity = false) and sends:

• A: an ordinary user task.
• B: a tool-continuation turn (assistant tool_calls plus a role:"tool" result as the newest messages).

python3 repro_classifier_toolturns.py /path/to/switchyard-server

Output against current main:

request A served by: weak-model
request B served by: strong-model
switchyard_requests_total{model="strong-model",otel_scope_name="switchyard"} 1
switchyard_requests_total{model="weak-model",tier="weak",otel_scope_name="switchyard"} 1
A classified weak with tier label: True
B fell back to strong WITHOUT tier label: True

Expected vs. actual

• Expected: continuation turns of an already-classified task are routed by a decision that saw the task, or reuse the prior decision, and any fallback is observable.
• Actual: the following chain, traced against current main.
  1. Inbound role:"tool" messages are normalized to Role::User carrying ContentBlock::ToolResult (crates/switchyard-translation/src/codecs/openai_chat/buffered.rs:112-134).
  2. The newest-user reverse scan in CapabilityJudge::build_request (crates/libsy/src/algorithms/llm_class.rs:107-115) therefore selects the newest tool result as the task.
  3. Encoded back out, that message becomes an orphaned role:"tool" entry (openai_chat/buffered.rs:700-714). A strict upstream rejects it with 400 and the judge verdict is unavailable, logged at warn only (util/llm_judge.rs:77-97). A lenient upstream judges a contentless fragment and abstains, which the policy filters with no log at all (llm_class.rs:179-181).
  4. Either way the result is Ambiguous, so the cascade's DefaultTarget capable fallback decides (llm_class.rs:549-556).

Environment

• Switchyard version / commit: reproduced on current main, first observed on a v0.1.0 build (6ff491ac)
• Python: n/a for the Rust server path; the repro script needs only stdlib
• OS / arch: macOS ARM64 (native build)
• Install: source build (cargo build --release -p switchyard-server)
• Inbound format: Chat Completions
• Backend: OpenAI-compatible (stub in the repro; observed live via OpenRouter)

Additional context

Understood that "classify once per task" via session_affinity and message_hash_fallback is the designed cost answer (the classifier_config_reuses_message_hash_affinity_for_a_follow_up test pins it), and that recent_turn_window and the escalation variant are the designed context answers.

Two gaps survive those mitigations:

1. The no-user-anchor arm of trim_messages (llm_class.rs:80-82) returns instructions only, so the window cannot help on a turn that has no user message.
2. The window can slice mid-tool-pair, so the judge request opens with an orphaned role:"tool" or a dangling tool_calls. A strict upstream then rejects the judge call and the fallthrough comes back.

On observability: PR #205 counts fail-opens, but a valid abstain is not a fail-open by its definition and stays unlogged. That narrower gap is the observability ask here. #246 and #252 look like the Python-chain members of the same family, where the judge sees the wrong content.

Possible fix directions, in rough preference order:

1. reuse the previous decision on tool-continuation turns;
2. fall back to the first user message in build_request, mirroring first_user_message_hash; repair tool-pair integrity in trim_messages;
3. add a log or counter on the abstain path.

I can send a PR for any of these once maintainers confirm the direction.

repro_classifier_toolturns.py