fix(acp): deliver plain-text replies for self-hosted OpenAI-compatible agents - #4874
fix(acp): deliver plain-text replies for self-hosted OpenAI-compatible agents#4874repudi8or wants to merge 6 commits into
Conversation
buzz-agent's contract is that output is tool calls: streamed assistant text
is observability only and never published. Capable models honour that and
call `send_message`. Small local models served over shared compute do not
always — on multi-step turns they run their tools, then write the answer as
prose and end the turn, so the turn reports success with nothing in the
channel.
Retain the turn's streamed text in AcpClient (bounded at 8 KiB) alongside a
flag for whether a publish tool call was seen, and on EndTurn publish that
text as a threaded channel reply when the agent published nothing itself.
Gated on EndTurn only: MaxTokens / MaxTurnRequests mean the turn was
truncated, so the text is not a deliberate answer. Bare acknowledgements
("OK", "Done") are filtered, and taking the buffer clears it so the same
text can never post twice.
Off by default via --deliver-plain-replies / BUZZ_ACP_DELIVER_PLAIN_REPLIES:
for cloud models, which reliably publish their own replies, posting streamed
text would double-post. The desktop shared-compute preset opts in, so this
stays scoped to the case that needs it.
Verified end to end against a live relay with stub ACP agents that pin each
branch deterministically: a prose-only agent gets its answer delivered, and
an agent that calls send_message while also streaming prose posts exactly
once with the fallback silent.
Forward-ported onto main from the unmerged origin/micspiral/mesh-0-74-gemma
branch (bb4af41). This exact bug was hit in production by the "Local LLM
(llm1)" self-hosted mesh-llm/llama-server agent: turns ended with EndTurn
and correct prose visible in the ACP activity log, but nothing was published
to the channel because send_message was never called.
Signed-off-by: Michael Neale <michael.neale@gmail.com>
with a helping hand from Claude Code
Signed-off-by: Brett Meehan <repudi8or@gmail.com>
…ly delivery
The plain-reply-delivery fallback landed for the built-in relay-mesh
("Buzz shared compute") preset only, gated on provider == "relay-mesh"
in apply_relay_mesh_env. A self-hosted model reached the same way small
local models always are — the "OpenAI-compatible" provider pointed at a
user's own llama.cpp/vLLM/mesh-llm endpoint instead of the built-in
preset — never opted in, so the exact same silent-drop bug reproduced:
the model answers in prose, ACP's activity log shows a correct reply,
and nothing is ever published to the channel.
Default BUZZ_ACP_DELIVER_PLAIN_REPLIES to "true" for effective provider
"openai-compat" too, in the same effective-env assembly step, right
after the relay-mesh translation. Scoped narrowly to "openai-compat":
plain "openai" (the real OpenAI cloud API) is left untouched, since
cloud models reliably call send_message themselves and enabling the
fallback there would risk double-posting. An explicit user-set value
(including an intentional "false") is preserved, matching the emptyness
convention used elsewhere in this module.
Reproduced against the "Local LLM (llm1)" agent in production: provider
"openai-compat", OPENAI_COMPAT_BASE_URL pointed at a remote llama-server
instance — confirmed via the agent's own config screenshots that it is
not on the relay-mesh preset, so relay_mesh.rs's existing default never
applied to it.
with a helping hand from Claude Code
Signed-off-by: Brett Meehan <repudi8or@gmail.com>
|
@wesbillman this fixes a bug affecting self-hosted OpenAI-compatible agents (e.g. local llama.cpp/mesh-llm endpoints) silently dropping their replies instead of publishing them — full context and test results in the PR description. Would appreciate a review/merge when you have a moment. I only have read access to block/buzz directly (opened from a fork), so can't self-merge. |
Live end-to-end repro: confirmed ✅Independently verified the events directly on the relay (not just trusting the report):
This closes the one unchecked item in the test plan: a self-hosted Non-blocking follow-up spotted in the reply content (not the delivery mechanism, which is what this PR fixes): the model looped through many near-duplicate self-summaries for the full turn, and the delivered text is hard-cut mid-word — consistent with hitting the 8KiB |
rmcp/schemars-derived tool schemas (buzz-dev-mcp's shell, read_file, str_replace, etc.) routinely carry JSON Schema metadata and assertions like $schema, title, format, and additionalProperties. Anthropic and the official OpenAI API tolerate these, but some openai-compat backends reached via a self-hosted endpoint or a multi-provider proxy (observed via a litellm-fronted OpenRouter route) hard-reject the whole tool-call request with "unsupported assertions or reserved metadata" instead of ignoring the extra keys. Add sanitize_tool_schema, applied once at tool registration in mcp.rs alongside the existing cap_schema size cap, to strip a denylist of pure-metadata and format/range-assertion keywords from every object in the schema tree. Every stripped keyword only narrows or documents what the schema accepts, never widens it, so removal can loosen validation but never break a previously-valid tool call. Structural/semantic keywords (type, properties, required, items, description, enum) are left untouched. Signed-off-by: repudi8or <repudi8or@gmail.com>
…t-replies Signed-off-by: repudi8or <repudi8or@gmail.com>
An earlier commit on this branch grew readiness.rs past its grandfathered 1742-line ratchet allowance (to 1850 lines), blocking the pre-push desktop-check hook. Following this repo's existing convention for oversized test modules (migration.rs's migration_tests.rs, migration_command_tests.rs, etc., and readiness.rs's own readiness_goose_file_config_tests.rs), extract the inline `mod tests` block into sibling files declared via #[path]. The extracted module still exceeded the 1000-line cap for new files, so it's split further into readiness_tests.rs and readiness_provider_tests.rs. readiness.rs itself drops to 690 lines. Also includes a cargo fmt fix to relay_mesh.rs (a pre-existing line-wrap picked up by running fmt across the whole crate), and reformats the two new test files to match. Signed-off-by: repudi8or <repudi8or@gmail.com>
Resolves conflicts in buzz-acp's turn/usage tracking fields and the desktop relay-mesh env/readiness modules, restoring the branch's plain-text-reply delivery fallback (relay-mesh auto/model wiring, readiness test split) on top of main's newer usage-tracking and relay-mesh model-translation changes. with a helping hand from Claude Code Signed-off-by: Brett Meehan <repudi8or@gmail.com>
Summary
send_message— the turn reportsEndTurnsuccessfully, the ACP activity log shows the correct reply, but nothing is ever published to the channel. This is what we observed running a self-hosted local-model agent against a mesh-llm deployment.crates/buzz-acp: retain each turn's streamed text (bounded at 8 KiB) plus whether a publish tool call was seen; onEndTurn, if nothing was published, deliver that text as a threaded channel reply instead. Gated strictly onEndTurn(notMaxTokens/MaxTurnRequests, which mean the turn was truncated) and filters bare acknowledgements ("OK", "Done"). Off by default via--deliver-plain-replies/BUZZ_ACP_DELIVER_PLAIN_REPLIES, since cloud models reliably callsend_messagethemselves and enabling it there would risk double-posting.desktop/src-tauri: opt the built-in "Buzz shared compute" (relay-mesh) preset into this fallback, and separately default it on for the generalopenai-compatprovider too (a self-hosted endpoint configured directly, not through the built-in preset) — scoped narrowly so plainopenai(the real OpenAI cloud API) is untouched. An explicit user override (including an intentionalfalse) always survives.crates/buzz-agent: while validating theopenai-compatfallback above against a real self-hosted setup (a proxy fanning a single model alias out across several backend providers), we also hit a second, unrelated failure mode worth fixing alongside it — a 400 from a backend routed through OpenRouter:"auto tool schema uses unsupported assertions or reserved metadata". The root cause:buzz-dev-mcp's tool schemas are auto-derived viarmcp/schemars, which routinely emits JSON Schema metadata and assertion keywords ($schema,title,format,additionalProperties, range/length constraints, ...) that Anthropic and the official OpenAI API happily ignore, but that some openai-compat backends validate strictly and reject outright — breaking every tool call for that agent, not just the reply-delivery path. Addedsanitize_tool_schema, applied once at tool registration alongside the existing schema-size cap (cap_schema), which strips a denylist of those keywords from the whole schema tree before it's ever sent to a provider. Every stripped key only narrows or documents what a schema accepts, so this can only loosen a backend's validation — it can't change what a tool call actually does, and structural/semantic keywords (type,properties,required,items,description,enum) are left untouched. This is the same underlying theme as the reply-delivery fix above (self-hostedopenai-compatbackends don't share the reference providers' tolerances), so it seemed worth landing together — useful for anyone pointingbuzz-agentat vLLM, llama.cpp, a litellm/multi-provider proxy, or any OpenRouter-routed model with a stricter function-calling schema validator than Claude/GPT.desktop/src-tauri: mergedmainin to pick up two commits that had landed oncrates/buzz-acpsince this branch started, and splitreadiness.rs's test module (readiness_tests.rs/readiness_provider_tests.rs) to stay under the desktop file-size ratchet after that merge — no behavior change, same tests, same assertions.Test plan
cargo test --manifest-path desktop/src-tauri/Cargo.toml --lib— 2250 passed, 0 failed, 14 ignored (full crate, not scoped to touched files), run independently by two peoplepublish_detection_matches_tool_and_shell_shapes,bare_acknowledgements_are_not_worth_publishing,non_mesh_provider_does_not_opt_into_plain_reply_delivery,openai_compat_provider_opts_into_plain_reply_delivery,real_openai_provider_does_not_opt_into_plain_reply_delivery,explicit_deliver_plain_replies_override_is_preserved_for_openai_compatcargo test -p buzz-agent --lib— 401 passed, 0 failed, including 4 new schema-sanitization tests (sanitize_tool_schema_strips_root_metadata,sanitize_tool_schema_strips_nested_property_assertions,sanitize_tool_schema_preserves_structural_and_semantic_keywords,sanitize_tool_schema_recurses_into_arrays)cargo clippy -p buzz-agent -p buzz-acp --lib -- -D warningsandcargo fmt --manifest-path desktop/src-tauri/Cargo.toml --all -- --checkcleanbuzz-acp(bundled inside the Buzz Desktop app) anddesktop/src-tauri, and there's no way to swap just one agent's binary without patching the signed Desktop app bundle used by every other agent on that machine. Recommend validating live once this ships through the normal Desktop build/update path.Diagnosed and coordinated by the buzz-acp team (architect, dev, tester, conductor) in the team's working channel; core mechanism forward-ported from an unmerged branch (
micspiral/mesh-0-74-gemma, commitbb4af4189) by Michael Neale.with a helping hand from Claude Code