feat(stock-watchlist-agent-js): accept image inputs and annotate them on spans

[cdfox]

What

Adds an image input path to the JS stock watchlist demo, and attaches the input image to the LLM Observability span that consumes it.

Inputs can now be ticker symbols, local image files, or a mix of both:

# a) all three inputs are ticker symbols
npm start -- AAPL GOOGL NVDA

# b) first two inputs are images, third is a ticker symbol
npm start -- logos/apple.png logos/google.png NVDA

When images are given, a vision step runs first: one LLM call per image identifies the public company shown and returns its ticker symbol. Resolved tickers are merged with any tickers passed directly, deduped, and handed to the existing orchestrator. Images that match no public company come back as UNKNOWN and are skipped.

Changes

• src/agents/vision.js (new) — resolveTickersFromImages() fans out one identify_ticker span per image. That span is kind: 'llm' and is annotated with a user message carrying imageParts: [{ mimeType, content }], so the image renders on the span. The image file is read as base64 so the same bytes are sent to OpenAI and attached to the trace.
• src/agents/orchestrator.js — image resolution happens inside the root analyze_portfolio agent span, so image translation and research share a single trace rather than producing two.
• src/main.js — argument parsing accepts local image files by extension, plus an explicit --image <path> override. Prints what each image resolved to. Evaluations run against the resolved ticker list.
• src/models.js — tickerFromImageSchema + validateTickerFromImage.
• logos/ — three small wordmark PNGs (Apple, Google, NVIDIA; ~900 bytes each, 12K total) so the image path is runnable without supplying your own files.
• Model defaults — all four call sites (agent loop, vision, web search, eval judges) now default to gpt-5.4-nano, matching the direction of Unpin openai, trim requirements, and update notebooks to gpt-5.4-nano #78. All remain overridable via OPENAI_MODEL, OPENAI_VISION_MODEL, OPENAI_SEARCH_MODEL, OPENAI_EVAL_MODEL.

Trace shape

analyze_portfolio (agent)                    ← evals attach here
├── resolve_tickers_from_images (workflow)   ← only when image inputs are given
│   └── identify_ticker (llm, one per image) ← image_parts annotated here
└── orchestrator (agent)
    └── delegate_research (tool) → stock_researcher ...

Testing

Tested against @joizddog's dd-trace-js branch for image support, dd-trace-js#9684 (jose/mlob-7916-llmobs-image-parts), installed as a local file dependency:

npm install ../../../dd-trace-js
DD_LLMOBS_ML_APP=stock-watchlist-agent-js-chris dd-auth -- npm start -- logos/apple.png

The Apple wordmark renders on the identify_ticker span in the LLM Observability trace view, alongside the user message and the assistant's structured {ticker: "AAPL", ...} output.

Two findings from that testing shaped this implementation:

1. Provider auto-instrumentation does not capture images. The dd-trace-js PR is manual-annotation only (auto-capture is a stated follow-up), so the auto-instrumented OpenAI.createResponse child span carries no image. The image has to be annotated by hand.
2. imageParts is only honored on llm-kind spans. In sdk.js, only spanKind === 'llm' routes to tagLLMIO → tagMessages, which is the sole reader of imageParts. On task / workflow / agent spans the input goes through tagTextIO and the whole message array is stringified into an opaque value — imageParts passes through as inert text with no warning. identify_ticker was originally a task span, which is why no image appeared at first.

Also verified end to end:

• npm start -- AAPL GOOGL NVDA — unchanged text-only path
• npm start -- logos/apple.png logos/google.png NVDA — mixed path; resolves AAPL + GOOGL, analyzes three tickers, evals submitted
• All three bundled logos resolve correctly at high confidence (AAPL, GOOGL, NVDA)
• Vision, research, and eval spans all share one trace_id

Note on dd-trace version

Image parts on LLMObs spans shipped in dd-trace 6.10.0, so package.json now requires ^6.10.0 and the lockfile is bumped to match.

The old lockfile pinned 6.4.0, which silently ignores imageParts and renders no image, so a plain npm install on the previous revision resolved a version that could not show the feature even though the "latest" spec suggested otherwise. Re-verified on the published 6.10.0 after removing the local-checkout swap: the identify_ticker span carries image_parts: [{mime_type: "image/png", content: "<base64>"}] and the image renders in the trace view. The local-checkout dependency swap used during development is deliberately not committed.

🤖 Generated with Claude Code