Skip to content

Promote cancellation trace-context capture to a CancellationTraceCapture capability - #848

Open
mpfaffenberger wants to merge 2 commits into
mainfrom
feature/cancellation-trace-capability
Open

Promote cancellation trace-context capture to a CancellationTraceCapture capability#848
mpfaffenberger wants to merge 2 commits into
mainfrom
feature/cancellation-trace-capability

Conversation

@mpfaffenberger

Copy link
Copy Markdown
Owner

What

Eighteenth in the capability-conversion series (#828-#836, #838-#842, #844, #845, #847). Promotes the cancellation trace-context capture -- the newest feature on main (#827, "Link cancellation event to agent trace context") -- from an eager _observed_event_stream_handler closure in _runtime._do_run to a first-class pydantic-ai capability, CancellationTraceCapture, on the wrap_run_event_stream seam.

This is a disjoint second claim of the seam (#835 StreamRendering claims it for rendering; this claims it for trace capture -- same precedent as #847's second claim of after_run). Both are pass-through observers, so order between them is inert.

Why this feature maps to this seam

emit_cancellation links the "Agent run cancelled" Logfire warning to the run's live trace by attaching the context captured during the most recent streamed model request. That capture is per-request, run-path work that main smuggled into the event_stream_handler wrapper -- the exact surface wrap_run_event_stream formalizes.

The other two thirds of the feature deliberately stay eager:

  • clear_agent_context runs in the turn task's finally (must cover cancel/crash exits; a cancelled run never reaches after_run).
  • emit_cancellation fires from the await-site except* handlers via on_agent_run_cancel (cancellation can land between runs of a turn, where no seam is active; and round-15 scouting proved the CancelledError at wrap_run carries no usable snapshot).

Design

  • code_puppy/agents/_cancellation_trace.py -- the static entry CancellationTraceCapture does NOT override the seam. Its for_run resolves a per-turn CancellationTraceObservation (group_id + streaming gate) from a ContextVar installed by _do_run, returning a per-run _ActiveCancellationTraceCapture only when enabled. Disabled/absent observations return self, so pydantic-ai's "overriding this seam auto-enables streaming" behavior never fires for gated-off runs -- byte-identical to the eager wrapper existing only when get_enable_streaming() was true (the Deliver stream rendering via a StreamRendering capability #835 inert-resolution trick).
  • The active capture yields events through unchanged and calls capture_agent_context(group_id) once per wrapped node stream -- the same cadence as the old per-handler-invocation capture (pinned by a counting test), in the identical OTel span context (pinned under a real SDK TracerProvider: capture and handler record the same span id).
  • ContextVar custody: plain set, no reset -- _do_run executes inside the turn's agent task, whose context dies with the task. Nested run_with_mcp turns run in their own task and install their own observation (shadowing pinned). None installs shadow (the feat: InterruptedSubagentNotes capability (capability series, round 12) #840 lesson).
  • Injectable capture seam defaulting to late-bound code_puppy.observability.capture_agent_context, so existing module patches keep intercepting (feat(agents): SubagentSessionPersistence capability on the wrap_run seam #842's call-time-resolution pattern).
  • Sub-agent invoker untouched: the eager capture never covered temp agents (source-pinned).

Bounded divergences (documented)

  1. Capture moment moves from handler-invocation to first event pull -- one frame apart in the same await chain, same task, same span context; observable only if a cancel lands in that microscopic window, or if a handler never iterates its events (the production handler always does).
  2. A direct pydantic_agent.run() call issued inside a turn's task (bypassing run_with_mcp) would inherit the turn's observation and capture under its group id, where the eager wrapper (bound to the kwarg) did not. No such caller exists in the codebase; production installs are turn-task-scoped.
  3. Under DBOS durable exec, the capture now rides the workflow-side capability application rather than the per-run kwarg -- the Deliver stream rendering via a StreamRendering capability #835 carve-out analysis applies unchanged.

Tests

21 contract tests in tests/agents/test_cancellation_trace_capability.py: for_run resolution, no-forced-streaming (request-only FunctionModel proves inertness; stream-only proves streaming -- the #835 asymmetric-model trick), capture cadence == handler invocations, span-context parity under real OTel instrumentation, gate-off silence, event pass-through equality, sequential-run accumulation, ContextVar shadowing/task-death custody, late-binding patches, builder wiring via the public apply visitor, production-shaped run_with_mcp drives (gate on/off, group id == agent_run_start session id), and source pins for the retired wrapper + eager emit/clear custody.

Full suite: 7618 passed, 0 failed (28 skipped, 1 xpassed).

Merge notes

Shares the main capabilities=[...] block with the seventeen open siblings -- whichever lands last eats a trivial rebase. Conflicts with #835's _do_run changes are mechanical (both touch the handler setup; both preserve capture_agent_context semantics).

The Logfire cancellation-linking feature (#827) captured the live trace
context by wrapping the run's event_stream_handler in an eager
_observed_event_stream_handler closure inside _runtime._do_run. Promote
the capture to a first-class pydantic-ai capability,
CancellationTraceCapture, on the wrap_run_event_stream seam.

- code_puppy/agents/_cancellation_trace.py: static entry capability with
  no seam override; for_run resolves the turn's
  CancellationTraceObservation (group_id + streaming gate) from a
  ContextVar installed by _do_run and returns a per-run
  _ActiveCancellationTraceCapture that captures the trace context once
  per streamed node event stream and passes events through unchanged.
  Disabled/absent observations resolve to the seam-less static entry, so
  non-streamed runs are not forced into streaming mode (gate parity).
- The observation install is a plain ContextVar.set with no reset: it
  runs inside the turn's agent task, whose context dies with the task;
  nested run_with_mcp turns install their own in their own task.
- clear_agent_context (turn-end finally) and emit_cancellation (await-
  site except* handlers via on_agent_run_cancel) stay eager: both run
  outside the run boundary where no capability seam fires.
- Span-context parity pinned under real OTel instrumentation: the
  capture observes the identical span context the event stream handler
  is invoked in.

21 contract tests in tests/agents/test_cancellation_trace_capability.py.
@mpfaffenberger

Copy link
Copy Markdown
Owner Author

Review pass 1 (code-puppy-clone-1): APPROVE, zero blocking. Findings: 2 NON-BLOCKING (capture-timing divergence and guest-wrapper caveat — both already documented in the PR body), 1 NIT (docstring overstated how small the invocation-to-first-pull window must be for arbitrary handlers). NIT applied: docs now state first-pull semantics explicitly, noting the production StreamingTextDetector consumes immediately so the moments coincide in practice.

@mpfaffenberger

Copy link
Copy Markdown
Owner Author

Review pass 2 (code-puppy-clone-1): APPROVE, zero remaining findings. Both NON-BLOCKING dispositions (documented divergences, no code change) explicitly confirmed. Not merging per instructions — awaiting human eyeballs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant