Skip to content

otel/10: assert span timestamp relationships - #30

Merged
zhongkechen merged 1 commit into
codex/execution-otel-plugin-testsfrom
codex/span-timestamp-assertions
Jul 24, 2026
Merged

otel/10: assert span timestamp relationships#30
zhongkechen merged 1 commit into
codex/execution-otel-plugin-testsfrom
codex/span-timestamp-assertions

Conversation

@zhongkechen

@zhongkechen zhongkechen commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • reject every normalized span whose start timestamp is after its end timestamp
  • require a selected child span's complete timespan to remain within its asserted parent
  • add before and after span relationships with inclusive boundary semantics
  • assert deterministic ordering across resumed invocations, retries, polling attempts, handled-failure recovery, callbacks, and chained invokes in both telemetry views
  • document the timestamp rules and cover valid, invalid, missing, ambiguous, and parent-containment cases

Stack

Validation

  • hatch run test:all (203 passed)
  • hatch run types:check
  • hatch run yaml:lint
  • hatch fmt --check packages scripts
  • hatch run dist:all

@zhongkechen zhongkechen changed the title otel/8: assert span timestamp relationships otel/10: assert span timestamp relationships Jul 24, 2026
@zhongkechen
zhongkechen force-pushed the codex/span-timestamp-assertions branch from f75f5de to d04f133 Compare July 24, 2026 20:57
@zhongkechen
zhongkechen force-pushed the codex/execution-otel-plugin-tests branch 2 times, most recently from 9fce7a0 to bada378 Compare July 24, 2026 22:42
@zhongkechen
zhongkechen force-pushed the codex/span-timestamp-assertions branch from d04f133 to 240ad9c Compare July 24, 2026 23:35
@zhongkechen
zhongkechen marked this pull request as ready for review July 24, 2026 23:45
@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime July 24, 2026 23:45 — with GitHub Actions Inactive
@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime July 24, 2026 23:45 — with GitHub Actions Inactive
@zhongkechen
zhongkechen merged commit 520a05b into codex/execution-otel-plugin-tests Jul 24, 2026
9 of 23 checks passed
@zhongkechen
zhongkechen deleted the codex/span-timestamp-assertions branch July 24, 2026 23:45
@zhongkechen zhongkechen self-assigned this Jul 24, 2026
@github-actions

Copy link
Copy Markdown

Codex AI review

No actionable findings.

Residual risk: live backend timestamp precision and cross-invocation ordering are not integration-tested here. Tests were not run per review constraints.

Reviewed commit 240ad9c1475fd7109e1e1cf5f65d0c629e1f2d18. Workflow run

@github-actions

Copy link
Copy Markdown

Claude AI review

No actionable findings.

I reviewed the diff (validator changes in validators.py, model import, docs, the 12 requirement YAMLs, and the two new test modules) for correctness, edge cases, typing, schema/semantic compatibility, and test adequacy.

Verified as correct:

  • Boundary semantics (_temporal_relation_errors, validators.py:293): before errors iff selected.end_time > related.start_time and after errors iff selected.start_time < related.end_time, matching the documented inclusive semantics. Self-comparison is correctly excluded via span_index != selected_span_index, and the "exactly one" selector is enforced (0 → "matched no spans", >1 → "matched N spans").
  • Parent containment (_parent_expectation_errors, validators.py:171): property matching now runs first to compute matching_parents, then timespan containment (span.start_time < parent.start_time / span.end_time > parent.end_time) is checked against those; the ambiguous-parent and single-parent error paths are preserved. Containment correctly uses the Span model timestamps while property matching uses the serialized dict.
  • Type/dispatch split: _parent_expectation_errors receives span_models_by_id (list[tuple[Span, Mapping]]) while _link_expectation_errors still receives the serialized-only spans_by_id; both call sites (validators.py:422, :363) pass the matching structure, so types:check is unaffected.
  • Global start≤end check (validate_trace, validators.py:497): runs per span; instantaneous spans (start == end) pass. Running inside the polling accept predicate is benign (keeps polling on partial data).
  • Requirement YAMLs: every placeholder used in the new before/after selectors (${RETRY_STEP}, ${CONDITION1}, ${RECOVERY_STEP}, ${FAILED_STEP}, ${EXECUTION_ARN}, ${TARGET_EXECUTION_ARN}, etc.) is bound by the case's ExpectedExecutionHistory, and selectors disambiguate via status/attempt attributes so each before/after resolves to exactly one span.
  • Tests: test_otel_catalog_asserts_every_deterministic_span_order's hardcoded ordered-case sets ({2,3,8,9} and {2,3,8,9,10,11,17,18}) exactly match the files that gained before/after, and its relationships == {"before","after"} invariant holds for each. Validator tests cover valid, invalid-selector, missing, ambiguous, and containment-violation cases.

Residual test risk: the new before/after ordering and parent-timespan-containment assertions are only structurally unit-tested. Their behavior against real backend telemetry — where clock skew or exact boundary equality can differ across the ADOT/X-Ray, Datadog, Dash0, and S3-collector backends — is not exercised in CI. The parent-containment check is a deliberate strengthening (per the PR description) applied to all 36 existing parent: assertions; if any conformant SDK/backend emits a child span whose start precedes its parent's start (e.g. a terminal Workflow root in the execution view constructed with a later start), those previously-passing cases would now fail only when run against live telemetry.

Reviewed commit 240ad9c1475fd7109e1e1cf5f65d0c629e1f2d18. Workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant