fix(server-utils): Include Gemini reasoning tokens in Vercel AI token usage

[zkasuran]

Gemini reasoning models undercount their output tokens in the Vercel AI integration. Gemini reports its reasoning ("thoughts") tokens separately from the visible candidate output, so the AI SDK's outputTokens covers only the answer and exposes the reasoning count through providerMetadata.google.usageMetadata.thoughtsTokenCount. getProviderMetadataAttributes() handled OpenAI, Anthropic, Bedrock and DeepSeek metadata but never looked at the Google/Vertex block, so the reasoning tokens were dropped from gen_ai.usage.output_tokens and the total was computed as input + candidate-only output.

Per the gen_ai token usage conventions, gen_ai.usage.output_tokens includes reasoning tokens. The fix reads the google/vertex usageMetadata, derives output as candidatesTokenCount + thoughtsTokenCount, sets the total from the real totalTokenCount and records the reasoning breakdown under gen_ai.usage.reasoning.output_tokens. Both the OTel span path and the ai tracing-channel path go through this shared helper, so both emit the corrected shape.

Deriving output from the raw candidate + thoughts counts (rather than adding reasoning onto the existing SDK value) is deliberate: it stays correct even if a future AI SDK version folds reasoning into outputTokens itself, so it cannot double count. The change is gated on thoughtsTokenCount > 0, so non-reasoning Gemini responses are left exactly as they were.

Root cause

getProviderMetadataAttributes() in packages/server-utils/src/ai/vercel-ai/index.ts had no google/vertex branch. The total is also computed from input + output before provider metadata is applied. For a real Gemini reasoning response:

usageMetadata { promptTokenCount: 14, candidatesTokenCount: 1, thoughtsTokenCount: 100, totalTokenCount: 115 }
ai-sdk result.usage { inputTokens: 14, outputTokens: 1, totalTokens: 115, reasoningTokens: 100 }

the emitted span attributes were, before the fix:

{"gen_ai.usage.output_tokens":1,"gen_ai.usage.input_tokens":14,"gen_ai.usage.total_tokens":15}

and after the fix:

{"gen_ai.usage.output_tokens":101,"gen_ai.usage.input_tokens":14,"gen_ai.usage.total_tokens":115,"gen_ai.usage.reasoning.output_tokens":100}

A vitest covering the OTel processor path and the shared getProviderMetadataAttributes() helper (including the v6 vertex key and a non-reasoning regression case) is added in packages/server-utils/test/ai/lib/tracing/vercel-ai-reasoning-tokens.test.ts.

---

• If you've added code that should be tested, please add tests.
• Ensure your code lints and the test suite passes (yarn lint) & (yarn test).
• Link an issue if there is one related to your pull request. If no issue is linked, one will be auto-generated and linked.

---

AI assistance (Claude, Anthropic) was used in developing this change. The design, review and verification were done by the author. Verified locally before submitting: yarn test in packages/server-utils (377 passing, 4 new), yarn lint (oxlint, clean), oxfmt --check (clean) and yarn build:types (tsc, clean).