Skip to content

feat: deliver Logfire tracing as an explicit Instrumentation capability - #838

Open
mpfaffenberger wants to merge 2 commits into
mainfrom
feature/instrumentation-capability
Open

feat: deliver Logfire tracing as an explicit Instrumentation capability#838
mpfaffenberger wants to merge 2 commits into
mainfrom
feature/instrumentation-capability

Conversation

@mpfaffenberger

Copy link
Copy Markdown
Owner

What

Tenth in the capability-conversion series (#828#836). Promotes OpenTelemetry/Logfire tracing from the process-global mutable default (logfire.instrument_pydantic_ai()Agent.instrument_all(settings)) to a first-class, explicitly-declared Instrumentation capability on the agents code_puppy constructs.

pydantic-ai 2.31.0 itself treats the capability as the canonical form — instrument_all's own docstring reads "for all agents that don't explicitly add an Instrumentation capability", and the run layer skips its internal injection whenever an explicit one is present ("explicit-capability-wins").

How

  • New code_puppy/agents/_instrumentation.pybuild_instrumentation() resolves the effective global default at build time (mirroring Agent._resolve_instrumentation_settings) and returns a stock Instrumentation capability carrying the exact settings object logfire installed (True → default-constructed settings; unset/False → empty list).
  • Both construction sites converted — main builder (_builder.py) and sub-agent invoker (subagent_invocation.py) splat *build_instrumentation() into their capabilities=[...] blocks. List position is inert: the capability declares position='outermost' ordering — the same slot the run layer's internal prepend targets.
  • observability.configure_logfire() unchanged in behaviour; docstring now documents the two delivery paths.

Why the global default is deliberately retained

  1. Out-of-tree agents. Core plugins construct their own raw pydantic-ai agents (wiggum's judge, btw's side-query) and receive spans purely from Agent.instrument_all. Removing it would silently kill their telemetry — a parity break.
  2. ctx.tracer consistency. pydantic-ai resolves the run context's tracer from global/instance settings independently of explicit capabilities; keeping both in sync means capabilities/toolsets observing ctx.tracer never see a NoOpTracer while spans exist.
  3. Ordering robustness. An agent built before configure_logfire runs carries no capability and degrades to exactly the old behaviour via the run-layer fallback.

Explicit-when-ours, fallback-for-guests.

Divergence (documented + pinned)

The capability is a build-time snapshot; the global default is read per run. If Agent.instrument_all(False) were called after a build, that agent would keep tracing until rebuilt (previously it would go quiet on the next run). Nothing in code_puppy or its plugins flips the default after startup. Pinned by test_snapshot_survives_later_global_disable.

Tests

12 contract tests in tests/agents/test_instrumentation_capability.py:

  • builder contract (empty/verbatim/True-normalisation/fresh-per-call)
  • private-seam pin on Agent._instrument_default (a pydantic-ai rename fails loudly here; runtime degrades soft to the fallback)
  • position='outermost' ordering pin
  • span parity: identical span name + gen_ai.operation.name sequences between the old global-only path and the new explicit path (in-memory OTel exporter)
  • no double instrumentation with both the explicit capability and the global default live
  • both real construction paths (main builder + _invoke_agent_impl) deliver the verbatim settings object

Full suite: 7609 passed, 28 skipped, 1 xpassed, 0 failed — including the previously-eternal test_render_version_check_current, healed on main by #837.

Promote OpenTelemetry/Logfire tracing from the process-global
Agent.instrument_all default (installed by logfire.instrument_pydantic_ai)
to a first-class pydantic-ai Instrumentation capability on the agents
code_puppy constructs itself.

- New code_puppy/agents/_instrumentation.py: build_instrumentation()
  snapshots the effective global settings at build time and returns a
  stock Instrumentation capability carrying the exact settings object
  logfire installed; empty list when uninstrumented.
- Both construction sites (main builder + sub-agent invoker) declare it
  explicitly; pydantic-ai's run layer skips its own injection when an
  explicit Instrumentation capability is present, so spans are identical.
- The global default is deliberately retained: plugin-built agents
  (wiggum judge, btw side-query) get their spans from it, and ctx.tracer
  resolution stays consistent. Explicit-when-ours, fallback-for-guests.
- One documented divergence, pinned by test: the capability is a
  build-time snapshot, so clearing instrument_all after a build keeps an
  already-built agent traced until rebuild. Nothing flips the default
  after startup.

12 contract tests including span-parity (old global-only path vs new
explicit path produce identical span name/op sequences) and a
no-double-instrumentation guarantee. Full suite: 7609 passed.
- Span-parity test now compares fully normalized spans: status codes,
  parent topology (by span name), complete attribute key sets, and
  stable attribute values — equality also rules out duplicated
  request/tool spans wholesale (findings 1+2).
- New ctx.tracer contract test: a probe capability observes the run
  context tracer and pins identity with settings.tracer, locking the
  documented rationale for retaining the global default (finding 3).
- Reworded fresh-capability test rationale and added a direct for_run
  isolation test pinning upstream's replace-copy contract (finding 5).
- Module docstring: ctx.tracer resolution claim scoped to classic
  run/iter, with the realtime-session nuance noted (finding 3 nuance).

14 contract tests; full suite 7611 passed.
@mpfaffenberger

Copy link
Copy Markdown
Owner Author

Review trail (code-puppy clone, session pr-838-review):

  • Pass 1: APPROVE — 4 non-blocking findings + 2 nits. Applied in 58a3847: span-parity test upgraded to fully-normalized comparison (status codes, parent topology, attribute key sets, stable values — equality also rules out duplicated request/tool spans), new ctx.tracer identity contract test, for_run isolation test, docstring scoped to classic run/iter with realtime nuance. DBOS wrapper test declined (DBOS not installed; capability attaches before on_wrap_pydantic_agent, wrapper transparency established by feat: deliver MCP toolsets via a pydantic-ai capability (McpToolsets) #834/Deliver stream rendering via a StreamRendering capability #835) — reviewer accepted: 'Mocking a fake DBOS-like wrapper would mostly test the mock. YAGNI wins.'
  • Pass 2: APPROVE — zero blocking findings.

Final state: 14 contract tests, full suite 7611 passed, 0 failed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant