feat(vercel-ai-sdk): add AI SDK v7 integration#794
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
hassiebp
force-pushed
the
codex/add-vercel-ai-sdk-integration
branch
from
824ace2 to
53f0072
Compare
hassiebp
changed the title
|
@claude review |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 53f0072fe4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83394e9706
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 22c6d54403
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const base64Content = part["content"]; | ||
| const mediaType = part["mime_type"]; | ||
|
|
||
| if ( | ||
| part["type"] !== "blob" || | ||
| typeof base64Content !== "string" || | ||
| typeof mediaType !== "string" || | ||
| base64Content.startsWith("http") | ||
| ) { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
🔴 Empty blob content (content: "") in an AI SDK v7 gen_ai.input.messages / gen_ai.output.messages blob part bypasses the type guard at MediaService.ts:216-223 (typeof "" === "string") and reaches mediaReplacedValue.replaceAll(base64Content, langfuseMediaTag) at line 247. Per ECMA-262, String.prototype.replaceAll("", x) inserts x between every UTF-16 code unit, so the entire span attribute JSON is shredded with media-tag interleaving and becomes both unparseable and massively bloated. The pre-existing v6 branch at line 143 in the same file already guards against this with if (!base64Content) continue; — mirror that one line (or add || !base64Content to the OR chain at 216-222).
Extended reasoning...
Bug
In packages/otel/src/MediaService.ts, the new AI SDK v7 media handler (lines 213-247) only validates the content field with typeof base64Content !== "string" — which lets an empty string "" slip through (typeof "" === "string"). A few lines later, line 247 unconditionally calls mediaReplacedValue.replaceAll(base64Content, langfuseMediaTag). Per the ECMA-262 spec for String.prototype.replaceAll with an empty searchValue, the replacement is inserted between every UTF-16 code unit (and at start/end), so the entire gen_ai.input.messages / gen_ai.output.messages JSON attribute is corrupted.
Why existing guards don't catch it
Walking the four guards at lines 216-223 for part = { type: "blob", content: "", mime_type: "application/pdf" }:
part["type"] !== "blob"→ false (matches).typeof base64Content !== "string"→typeof "" === "string", so false.typeof mediaType !== "string"→ false.base64Content.startsWith("http")→"".startsWith("http")returns false.
All pass. Execution proceeds to base64ToBytes(""), which calls atob("") → "" → empty Uint8Array. The downstream media.getTag() is not short-circuited either: LangfuseMedia.getSha256Hash() checks if (!this._contentBytes) — but an empty Uint8Array is a truthy object, so it proceeds and crypto.subtle.digest returns the well-known SHA-256 of empty input. getId() yields a valid 22-char ID, the if (!langfuseMediaTag) continue guard at line 233 does not fire, and we reach replaceAll at line 247.
Step-by-step proof
Verified in Node: 'abc'.replaceAll('', 'X') === 'XaXbXcX'. Concretely, with value = '[{"role":"user","parts":[{"type":"blob","content":"","mime_type":"application/pdf"}]}]' and langfuseMediaTag = '@@@langfuseMedia:type=application/pdf|id=AAA…|source=bytes@@@':
value.replaceAll("", langfuseMediaTag)returns a string where the media tag is inserted between every single character of the original JSON —@@@…@@@[@@@…@@@{@@@…@@@"@@@…@@@r@@@…@@@, and so on for every code unit.- The resulting string is no longer valid JSON, is many times larger than the original, and is then written back to
span.attributes[attribute]and exported to Langfuse.
Downstream consumers that expect to parse gen_ai.input.messages see total corruption of the prompt history, and the payload is also bloated by roughly attribute.length * langfuseMediaTag.length bytes.
Asymmetry with the v6 block
The v6 branch immediately above, at line 143, already defends against this exact case:
if (!base64Content) continue;The v7 block was clearly modelled on the v6 pattern (same author, adjacent code, same replaceAll usage) but dropped this one defensive line during the port. The fix is trivial: mirror that guard, e.g. add if (!base64Content) continue; after the type check, or fold !base64Content into the existing OR chain at lines 216-222.
Trigger probability and impact
The trigger is narrow: @ai-sdk/otel would need to emit a blob part with content: "" — possible if an upstream serializer/redactor strips or defaults the field, or a future canary release changes the shape. The blast radius is catastrophic when it does fire (entire span attribute mangled, downstream parse failures, large payload inflation), and the v6 author considered it worth defending against; consistency alone justifies the one-line fix.
| const embeddingObservation = trace.observations.find( | ||
| (observation: any) => | ||
| observation.type === "EMBEDDING" || | ||
| observation.model === modelName || | ||
| observation.name?.includes(modelName), | ||
| ); | ||
|
|
||
| expect(embeddingObservation).toBeDefined(); | ||
| expect(embeddingObservation!.model ?? embeddingObservation!.name).toContain( | ||
| modelName, | ||
| ); |
There was a problem hiding this comment.
🟡 The embed() e2e assertion at tests/e2e/vercel-ai-sdk-v7.e2e.test.ts:551-561 is degenerate: the disjunctive find() includes observation.name?.includes(modelName), which matches the AI SDK v7 root span (named like embed text-embedding-3-small) before any EMBEDDING-type observation. The follow-up embeddingObservation!.model ?? embeddingObservation!.name then falls through to the root's name, which already contains modelName by construction — so both assertions trivially pass even if no EMBEDDING observation is emitted. Tighten to observation.type === "EMBEDDING" so the test fails when the typed observation is missing.
Extended reasoning...
What the bug is
In the new embed e2e test:
const embeddingObservation = trace.observations.find(
(observation: any) =>
observation.type === "EMBEDDING" ||
observation.model === modelName ||
observation.name?.includes(modelName),
);
expect(embeddingObservation).toBeDefined();
expect(embeddingObservation!.model ?? embeddingObservation!.name).toContain(modelName);An embed() call under @ai-sdk/otel produces multiple spans. The package's own unit tests (packages/vercel-ai-sdk/src/index.test.ts:50) confirm the root span is named invoke_agent <model-id>, and the model-call branch yields a span whose name also embeds the model id (e.g. embedding text-embedding-3-small). Array.prototype.find returns the first observation matching any disjunct; if the root appears before the EMBEDDING-type observation in iteration order, the root is picked even though it has neither type === "EMBEDDING" nor a model attribute.
Why this masks real failures
The follow-up assertion uses model ?? name. The root has no model, so the fallback selects name — and name already contains modelName because that's exactly what allowed it to match the disjunct. toContain(modelName) is therefore tautologically true: the same property that made find return the root is the property the assertion checks.
Step-by-step proof
modelName = "text-embedding-3-small".- AI SDK v7 emits observations in order
[{name: "embed text-embedding-3-small", type: "GENERATION" or "DEFAULT"}, {name: "embedding text-embedding-3-small", type: "EMBEDDING", model: "text-embedding-3-small"}]. findevaluates the root first:type === "EMBEDDING"→ false;model === modelName→ false (nomodel);name?.includes(modelName)→ true ⇒ return the root.embeddingObservation = root(nomodelfield).embeddingObservation!.model ?? embeddingObservation!.name→root.name="embed text-embedding-3-small".expect("embed text-embedding-3-small").toContain("text-embedding-3-small")⇒ passes.
Now imagine the upstream integration regresses and stops emitting the EMBEDDING-type observation altogether (or emits it with the wrong model field). The root still exists, its name still contains the model id, both assertions still pass. The test cannot detect the failure mode it claims to test.
Addressing the refutation
The refutation argues this is intentional permissiveness since AI SDK v7 is in canary and the exact observation shape is in flux, and that result.embedding plus expectTraceAttributes cover the rest. That's partially right — the test still verifies the e2e plumbing works end-to-end — but it concedes the specific assertion that the EMBEDDING-type observation carries the right model is unverified. That is the only thing this block exists to check; everything else is already covered by expectTraceAttributes and expect(result.embedding).toBeDefined() above. Tightening to observation.type === "EMBEDDING" keeps the same canary tolerance (no change to other assertions, the GENERATION-style matcher used elsewhere remains permissive) while restoring the assertion's stated purpose.
How to fix
Replace the disjunctive matcher with a single typed check:
const embeddingObservation = trace.observations.find(
(observation: any) => observation.type === "EMBEDDING",
);
expect(embeddingObservation).toBeDefined();
expect(embeddingObservation!.model).toContain(modelName);This is test-quality only; no production code is affected.
1 participant
What changed
@langfuse/vercel-ai-sdkworkspace package for AI SDK v7 telemetry integrationTelemetryintegration that mirrors the upstream v7 OpenTelemetry span shape while adding Langfuse trace and prompt attributesruntimeContext.langfuse, legacy metadata fallback, nested tool execution context, and older beta tool callback aliasesWhy
AI SDK v7 moved to a callback-based telemetry model, so Langfuse needs its own maintained integration package instead of relying on the older native integration path.
Impact
aiinstrumentation scope and AI SDK-style span attributesValidation
tsxsmoke test covering root spans, prompt linkage, per-call Langfuse override precedence, and nested tool-span contextNotes
vitestandtsuppackage commands in this environment because Rollup's native binary (@rollup/rollup-darwin-arm64) fails to load locally with a code-signing /ERR_DLOPEN_FAILEDissue.Greptile Summary
This PR adds a new
@langfuse/vercel-ai-sdkworkspace package that provides a first-class Langfuse integration for AI SDK v7's callback-based telemetry model, replacing the older native OTEL path. The implementation uses a clean delegate pattern over the upstream@ai-sdk/otelOpenTelemetryclass, injecting Langfuse-specific span attributes (prompt linkage, observation metadata) solely through theenrichSpancallback.LangfuseVercelAiSdkIntegration): allTelemetryinterface methods forward to the upstream delegate; Langfuse context is resolved by merging constructor-level config with per-callruntimeContext.langfuse, with runtime values winning on conflicts.generateText,streamText,streamObject,embed, prompt linking, and file/image attachments.packages/vercel-ai-sdk/node_modules/\u2026/dist/index.js, bypassing workspace module resolution and sensitive to pnpm hoisting configuration changes.Confidence Score: 4/5
The new package and its production code are solid; the only concerns are in test infrastructure and would not affect published package behavior.
The core integration logic is clean and well-tested at the unit level. The e2e and integration test files import AI SDK modules via hardcoded paths deep inside packages/vercel-ai-sdk/node_modules, which will silently break under pnpm hoisting or if the dist/ layout of those canary packages changes. TestContextManager.with() also restores context synchronously before async callbacks complete, making it fragile for future tests that read context after an await. Both concerns are confined to the test layer and do not affect the published package.
tests/e2e/vercel-ai-sdk-v7.e2e.test.ts and tests/integration/vercel-ai-sdk.integration.test.ts for the fragile node_modules path imports; packages/vercel-ai-sdk/src/testUtils.ts for the async-unsafe TestContextManager.
Sequence Diagram
sequenceDiagram participant User participant AISDK as AI SDK v7 participant LFVI as LangfuseVercelAiSdkIntegration participant OtelDelegate as OpenTelemetry (delegate) participant EnrichSpan as enrichSpan callback participant LFCtx as resolveLangfuseContext participant Tracer as OTel Tracer User->>AISDK: generateText / streamText / embed AISDK->>LFVI: onStart(event) LFVI->>OtelDelegate: onStart(event) OtelDelegate->>Tracer: startActiveSpan(invoke_agent) OtelDelegate->>EnrichSpan: "enrichSpan({ spanType, runtimeContext })" EnrichSpan->>LFCtx: resolveLangfuseContext(configured, runtime) LFCtx-->>EnrichSpan: merged LangfuseContext EnrichSpan-->>OtelDelegate: Langfuse span attributes AISDK->>LFVI: onLanguageModelCallStart(event) LFVI->>OtelDelegate: onLanguageModelCallStart(event) OtelDelegate->>Tracer: startActiveSpan(chat model-id) OtelDelegate->>EnrichSpan: "enrichSpan({ spanType: languageModel })" EnrichSpan-->>OtelDelegate: "OBSERVATION_PROMPT_NAME, OBSERVATION_METADATA.*" AISDK->>LFVI: onFinish(event) LFVI->>OtelDelegate: onFinish(event) OtelDelegate->>Tracer: span.end() Tracer-->>User: spans exported via OTel pipeline to LangfusePrompt To Fix All With AI
Reviews (1): Last reviewed commit: "fix(vercel-ai-sdk): align canary telemet..." | Re-trigger Greptile