fix(server-utils): Deduplicate Google GenAI streaming tool calls - #23432
Draft
zkasuran wants to merge 1 commit into
Draft
fix(server-utils): Deduplicate Google GenAI streaming tool calls#23432zkasuran wants to merge 1 commit into
zkasuran wants to merge 1 commit into
Conversation
The streaming handler recorded every tool call twice: once from `chunk.functionCalls` and again from the `functionCall` parts of each candidate. `functionCalls` is an SDK getter over those same parts, so one real tool call produced two span entries with mismatched shapes (one keyed by the non-spec `args`, one by `arguments`). Take tool calls only from `chunk.functionCalls`, the same source the non-streaming path uses, so each call is recorded once in one shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The streaming instrumentation for
@google/genairecords every tool call twice.handleCandidateContentinpackages/server-utils/src/ai/google-genai/streaming.tspushes tool calls to the span from two sources on the same chunk:chunk.functionCalls, spread straight intostate.toolCallspart.functionCall, mapped again while iteratingcandidate.content.partschunk.functionCallsis a getter on the@google/genairesponse that is itself derived from those same candidate parts (it filterspartsforfunctionCallthen maps them). So a single real tool call lands ingen_ai.response.tool_callsas two entries. The two do not even share a schema: the getter keeps the SDK-native{ id, name, args }while the parts loop emits{ type, id, name, arguments }. One value ends up keyedargs, the otherarguments.The non-streaming path (
addResponseAttributes) reads tool calls fromresponse.functionCallsand records one entry per call, so streaming and non-streaming disagreed on both count and shape.Fix: take streaming tool calls only from
chunk.functionCalls, the same accessor the non-streaming path uses, then drop the second push. Each call is now recorded once, in one shape. The streaming output matches the non-streaming output.I kept
chunk.functionCallsrather than the parts loop so both code paths share one source of truth and emit the SDK-native shape. This mirrors the OpenAI and Anthropic integrations, whose streaming paths reconstruct the exact tool-call shape their non-streaming paths produce. The deprecatedgen_ai.response.tool_callsexample shows{ name, arguments }, but no provider integration normalizes to that literally (Anthropic keepsinput, OpenAI nests underfunction), so consistency between a provider's streaming and non-streaming output was the stronger property to preserve here.Root cause:
chunk.functionCallsandpart.functionCallare two views of the same data. Both were being written to the span.Verified against a real gemini-3.6-flash streaming response that returns one
controlLighttool call (response idIuF-auTiDpW2g8UPnNDKsAI). The identical response was replayed through the instrumentation before and after the change.before (
gen_ai.response.tool_calls):[{"id":"call_2079699","args":{"colorTemperature":"warm","brightness":30},"name":"controlLight"},{"type":"function","id":"call_2079699","name":"controlLight","arguments":{"colorTemperature":"warm","brightness":30}}]after:
[{"id":"call_2079699","args":{"colorTemperature":"warm","brightness":30},"name":"controlLight"}]A unit test in
packages/server-utils/test/ai/lib/tracing/google-genai-streaming.test.tscovers the single-call case, multiple calls across chunks and therecordOutputs: falsecase, then asserts the non-streaming path still records one entry in the same shape. It fails ondevelop(two entries) and passes with this change.yarn lint) & (yarn test).AI assistance (Claude, Anthropic) was used in developing this change. The design, review and verification were done by the author. Verified locally before submitting: the new and existing
@sentry/server-utilsunit tests (377 passing),oxlint,oxfmt --checkand the TypeScript type-check all pass, plus a real gemini-3.6-flash streaming run captured before and after.