Skip to content

fix(server-utils): Deduplicate Google GenAI streaming tool calls - #23432

Draft
zkasuran wants to merge 1 commit into
getsentry:developfrom
zkasuran:fix/google-genai-streaming-toolcall-dedupe
Draft

fix(server-utils): Deduplicate Google GenAI streaming tool calls#23432
zkasuran wants to merge 1 commit into
getsentry:developfrom
zkasuran:fix/google-genai-streaming-toolcall-dedupe

Conversation

@zkasuran

Copy link
Copy Markdown

The streaming instrumentation for @google/genai records every tool call twice.

handleCandidateContent in packages/server-utils/src/ai/google-genai/streaming.ts pushes tool calls to the span from two sources on the same chunk:

  • chunk.functionCalls, spread straight into state.toolCalls
  • part.functionCall, mapped again while iterating candidate.content.parts

chunk.functionCalls is a getter on the @google/genai response that is itself derived from those same candidate parts (it filters parts for functionCall then maps them). So a single real tool call lands in gen_ai.response.tool_calls as two entries. The two do not even share a schema: the getter keeps the SDK-native { id, name, args } while the parts loop emits { type, id, name, arguments }. One value ends up keyed args, the other arguments.

The non-streaming path (addResponseAttributes) reads tool calls from response.functionCalls and records one entry per call, so streaming and non-streaming disagreed on both count and shape.

Fix: take streaming tool calls only from chunk.functionCalls, the same accessor the non-streaming path uses, then drop the second push. Each call is now recorded once, in one shape. The streaming output matches the non-streaming output.

I kept chunk.functionCalls rather than the parts loop so both code paths share one source of truth and emit the SDK-native shape. This mirrors the OpenAI and Anthropic integrations, whose streaming paths reconstruct the exact tool-call shape their non-streaming paths produce. The deprecated gen_ai.response.tool_calls example shows { name, arguments }, but no provider integration normalizes to that literally (Anthropic keeps input, OpenAI nests under function), so consistency between a provider's streaming and non-streaming output was the stronger property to preserve here.

Root cause: chunk.functionCalls and part.functionCall are two views of the same data. Both were being written to the span.

Verified against a real gemini-3.6-flash streaming response that returns one controlLight tool call (response id IuF-auTiDpW2g8UPnNDKsAI). The identical response was replayed through the instrumentation before and after the change.

before (gen_ai.response.tool_calls):

[{"id":"call_2079699","args":{"colorTemperature":"warm","brightness":30},"name":"controlLight"},{"type":"function","id":"call_2079699","name":"controlLight","arguments":{"colorTemperature":"warm","brightness":30}}]

after:

[{"id":"call_2079699","args":{"colorTemperature":"warm","brightness":30},"name":"controlLight"}]

A unit test in packages/server-utils/test/ai/lib/tracing/google-genai-streaming.test.ts covers the single-call case, multiple calls across chunks and the recordOutputs: false case, then asserts the non-streaming path still records one entry in the same shape. It fails on develop (two entries) and passes with this change.

  • If you've added code that should be tested, please add tests.
  • Ensure your code lints and the test suite passes (yarn lint) & (yarn test).
  • Link an issue if there is one related to your pull request. If no issue is linked, one will be auto-generated and linked.

AI assistance (Claude, Anthropic) was used in developing this change. The design, review and verification were done by the author. Verified locally before submitting: the new and existing @sentry/server-utils unit tests (377 passing), oxlint, oxfmt --check and the TypeScript type-check all pass, plus a real gemini-3.6-flash streaming run captured before and after.

The streaming handler recorded every tool call twice: once from `chunk.functionCalls` and again from the `functionCall` parts of each candidate. `functionCalls` is an SDK getter over those same parts, so one real tool call produced two span entries with mismatched shapes (one keyed by the non-spec `args`, one by `arguments`). Take tool calls only from `chunk.functionCalls`, the same source the non-streaming path uses, so each call is recorded once in one shape.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant