From e26224dff9df88c99ab6103e821ab9b95f7b0c1e Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 08:21:19 +0100 Subject: [PATCH 01/37] chore: update packages --- Directory.Packages.props | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Directory.Packages.props b/Directory.Packages.props index 2c7ecbb3..6eb14c9c 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -11,9 +11,9 @@ - - - + + + From 4824f096a5ee8fcda151914f1604e11f8b6d0380 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 19:51:43 +0100 Subject: [PATCH 02/37] docs: Add streaming query results spec and ADR 0019 Add requirements, design review, and ADR 0019 for the streaming query pipeline feature (spec 012-streaming_results). Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/adr/0019-streaming-query-pipeline.md | 598 ++++++++++++++++++ specs/.current-spec | 2 +- specs/012-streaming_results/.adr-list | 1 + specs/012-streaming_results/.design-approved | 0 specs/012-streaming_results/.issue-number | 1 + .../.requirements-approved | 0 specs/012-streaming_results/README.md | 29 + specs/012-streaming_results/requirements.md | 241 +++++++ specs/012-streaming_results/review-design.md | 215 +++++++ 9 files changed, 1086 insertions(+), 1 deletion(-) create mode 100644 docs/adr/0019-streaming-query-pipeline.md create mode 100644 specs/012-streaming_results/.adr-list create mode 100644 specs/012-streaming_results/.design-approved create mode 100644 specs/012-streaming_results/.issue-number create mode 100644 specs/012-streaming_results/.requirements-approved create mode 100644 specs/012-streaming_results/README.md create mode 100644 specs/012-streaming_results/requirements.md create mode 100644 specs/012-streaming_results/review-design.md diff --git a/docs/adr/0019-streaming-query-pipeline.md b/docs/adr/0019-streaming-query-pipeline.md new file mode 100644 index 00000000..96e7ba77 --- /dev/null +++ b/docs/adr/0019-streaming-query-pipeline.md @@ -0,0 +1,598 @@ +# 19. Streaming Query Pipeline + +Date: 2026-07-09 + +## Status + +Accepted + +## Context + +Darker today supports only request/response queries: a query returns a single, fully-materialised +`TResult` through `IQueryProcessor.Execute` / `ExecuteAsync`. For large result sets, paged data, or +real-time feeds this forces the whole result into memory before the caller sees the first item. + +**Parent Requirement**: [specs/012-streaming_results/requirements.md](../../specs/012-streaming_results/requirements.md) + +**Scope**: This ADR decides the **public API shape and pipeline mechanics** for streaming queries +that yield results incrementally as `IAsyncEnumerable` — a single, cohesive architectural +decision covering the query/handler/decorator contracts, the processor entry point, how the +existing `PipelineBuilder` is extended, handler resolution, span/lifetime management, and +target-framework support. It resolves the five open questions recorded in the requirements. + +### Forces at play + +- **V5 targeting (NFR2)** — this ships in the V5 major release, so breaking changes are permitted + where they yield a simpler, more elegant design. We are *not* constrained to bolt streaming onto + the existing interfaces without touching them. +- **Consistency (NFR5)** — the streaming API should read as a natural sibling of the existing + async path (naming, cancellation, `IQueryContext`, DI registration) so there is "one obvious way" + to do each thing. +- **Laziness (FR6 / NFR1)** — the framework must not buffer the sequence; items are produced on + demand. This has sharp consequences for span lifetime, handler/decorator lifetime, and exception + propagation, because an `async` iterator method **defers execution until enumeration**. +- **Existing machinery** — `PipelineBuilder` (internal) resolves the handler by type from + a registry, builds a decorator chain from attributes ordered by `Step`, and invokes via + reflection. Streaming must extend this machinery, not fork it. +- **Prior art** — MediatR keeps streaming *fully parallel* to request/response: a dedicated + `IStreamRequest` marker, a dedicated `CreateStream` method returning + `IAsyncEnumerable` (no `Task`), and a dedicated `IStreamPipelineBehavior` whose `next` is a `StreamHandlerDelegate`. (See requirements — verbatim + signatures.) + +### The reflection/laziness interaction (why this ADR is not just "add another overload") + +The current pipeline calls `MethodInfo.Invoke(handler, args)` and unwraps +`TargetInvocationException` via `ExceptionDispatchInfo`. For an `async IAsyncEnumerable` iterator +method, `Invoke` returns the enumerable **immediately without running the body** — the body runs +during `MoveNextAsync`. Therefore: + +- Handler exceptions surface during `await foreach`, *not* from `Invoke`, so the + `TargetInvocationException` unwrap that guards `Execute`/`ExecuteAsync` is neither triggered nor + needed for enumeration faults — they propagate naturally with their original stack trace. +- The processor cannot use its usual `try { return Invoke(); } finally { EndSpan(); } ` + + `using (pipelineBuilder)` shape: those would end the span and release the handler/decorators + *before the first item is produced*. Span and pipeline lifetime must instead be bound to the + **enumeration** lifetime. + +## Decision + +Adopt a **fully-parallel streaming path** modelled on MediatR's shape but expressed in Darker's +idiom (Execute-style naming, `IQueryContext`, attribute-driven decorators, per-query +factory/lifetime). Concretely: + +### 1. A dedicated stream query marker — `IStreamQuery` + +```csharp +// TResult is the ITEM type, not the enumerable. +public interface IStreamQuery : IQuery { } +``` + +- **`TResult` is the item type** (a `IStreamQuery` yields `Order` items), *not* + `IAsyncEnumerable`. Reusing `IQuery>` instead was rejected: it + routes through the single-result pipeline as `Task>`, defeating laziness + and reading as "a task of an enumerable" rather than "a query that yields a stream." +- **Derives from the generic `IQuery`** (not the non-generic `IQuery`). This is the choice + that actually compiles against the existing tracing code: `Query` — the id/tracing base + class — is declared `Query : IQuery` (`Observability/Query.cs`), and + `IAmADarkerTracer.CreateQuerySpan(IQuery query, …)` plus its + `query is Query` id extraction (`Observability/DarkerTracer.cs`) both require an + `IQuery`. A stream query can therefore reuse `Query` for a stable `Id` and be + passed to `CreateQuerySpan` unchanged. Here `TResult` is the item type, so the span is typed on + the item — acceptable and consistent with treating `TResult` as the item type throughout. +- **Stream-vs-single dispatch is by method + registry, not by the marker.** Because + `IStreamQuery` *is* an `IQuery`, the marker alone no longer separates the two + worlds. Separation comes from (a) the caller choosing `ExecuteStream` vs `ExecuteAsync`, and (b) + `ExecuteStream` resolving from `IStreamQueryHandlerRegistry` while `ExecuteAsync` resolves from + `IQueryHandlerRegistryAsync` (§5). **Consequence to document:** a stream query passed to + `ExecuteAsync` compiles (it is an `IQuery`) but fails cleanly at handler lookup with a + `ConfigurationException` ("no async handler registered"), because stream handlers live only in the + stream registry — a clear, early error rather than silent misbehaviour. + +### 2. A dedicated stream handler — `IStreamQueryHandler` + +```csharp +public interface IStreamQueryHandler : IQueryHandler + where TQuery : IStreamQuery +{ + IAsyncEnumerable ExecuteAsync(TQuery query, CancellationToken cancellationToken = default); +} + +public abstract class StreamQueryHandler : IStreamQueryHandler + where TQuery : IStreamQuery +{ + public IQueryContext Context { get; set; } + public abstract IAsyncEnumerable ExecuteAsync(TQuery query, CancellationToken cancellationToken = default); +} +``` + +- Reuses the existing non-generic `IQueryHandler` base (for `Context`) so the existing + **factory and lifetime machinery apply unchanged**. +- **No `Fallback` method.** Unlike single-result handlers, "fall back to a single value" does not + map onto "a partially-emitted stream." Dropping fallback from the stream contract is the simpler, + more honest V5 choice; a handler that needs a fallback stream composes it internally. +- Implementers put `[EnumeratorCancellation]` on their iterator method's token parameter (it is an + implementation concern, not part of the interface) so `await foreach (... .WithCancellation(ct))` + flows correctly. + +### 3. A dedicated stream decorator — `IStreamQueryHandlerDecorator` + +```csharp +public interface IStreamQueryHandlerDecorator : IQueryHandlerDecorator + where TQuery : IStreamQuery +{ + IAsyncEnumerable Execute( + TQuery query, + Func> next, + CancellationToken cancellationToken); +} + +[AttributeUsage(AttributeTargets.Method)] +public abstract class StreamQueryHandlerAttribute : Attribute // mirrors QueryHandlerAttributeAsync +{ + public int Step { get; } + protected StreamQueryHandlerAttribute(int step) => Step = step; + public abstract object[] GetAttributeParams(); + public abstract Type GetDecoratorType(); +} +``` + +- Reuses `IQueryHandlerDecorator` (for `Context` / `InitializeFromAttributeParams`), so the + existing decorator factory and lifetime apply unchanged. +- `next` is `Func>` — Darker's explicit + query+token style (vs MediatR's parameterless closure), consistent with the existing async + decorator's `next`. +- **No `fallback` continuation** (consistent with §2). +- A **new attribute base** (`StreamQueryHandlerAttribute`) is required so the stream pipeline + builder selects stream decorators and rejects sync/async decorator attributes (and vice versa), + mirroring the existing `ValidateNoMismatchedAttributes` guard. + +**Decorator semantics (resolves FR7):** + +> **On "fallback" — one crisp distinction used consistently below:** the **handler-method** fallback +> model (a `Fallback`/`FallbackPolicy` on the handler) is **removed** for streams; a **Polly +> fallback *strategy*** inside a resilience pipeline **is supported**, but only at stream +> *establishment* (§3a). "Fallback not supported" always means the former; "fallback available" +> always means the latter. + +| Decorator kind | V5 behaviour | +|---|---| +| **Logging / Telemetry** | **Supported.** Wraps the stream lifecycle: log/record on start, `yield return` each item, record item count + duration on completion, record exceptions raised during enumeration. Naturally expressible as an `async` iterator over `next`. | +| **Resilience pipeline (Polly v8) — retry / timeout / circuit-breaker / fallback** | **Supported at stream establishment** via a new `UseResiliencePipelineStreamHandler` (stream signature). The pipeline wraps enumerator creation **and the first `MoveNextAsync`**; items are yielded only *after* the pipeline succeeds, so a failure before the first item retries a **fresh** enumerable with **no duplicate emission**. Faults after the first item propagate un-retried. See §3a. | +| **Retry (legacy `RetryableQuery`)** | The request/response-only `RetryableQuery` attribute is **not** applied to streams (its Polly v7 policy model doesn't map). Stream retry is delivered through the resilience pipeline decorator above. | +| **Fallback (`FallbackPolicy` handler-method model)** | **Not supported** as a stream handler method (no `Fallback` on stream handlers, no `fallback` continuation). Fallback-to-an-alternate-stream *is* available through a Polly **fallback strategy** inside the resilience pipeline, applied at establishment (§3a). | +| **Caching** | Out of scope (OOS6). | + +### 3a. Resilience for streams — `UseResiliencePipelineStreamHandler` + +Darker's existing `UseResiliencePipelineHandlerAsync` executes the handler through a named Polly v8 +`ResiliencePipeline` resolved from `IQueryContext.ResiliencePipeline`. Naively wrapping a streaming +handler the same way is a no-op: the pipeline would wrap the *call that returns the enumerable*, +which — because iterator bodies defer — completes before any data is pulled, so it protects nothing. + +The fix (per Varnon, *Extending Polly retry policies to cover IAsyncEnumerables*) is to pull the +**first `MoveNextAsync` inside the pipeline boundary** and yield strictly afterwards. A new stream +decorator of the §3 signature, reusing the existing pipeline-resolution logic: + +```csharp +public sealed class UseResiliencePipelineStreamHandler + : IStreamQueryHandlerDecorator where TQuery : IStreamQuery +{ + public IQueryContext Context { get; set; } + // InitializeFromAttributeParams resolves _policy exactly as the async decorator. + + public async IAsyncEnumerable Execute( + TQuery query, + Func> next, + [EnumeratorCancellation] CancellationToken cancellationToken) + { + // Untyped pipeline ONLY — see note below on why a type-scoped ResiliencePipeline + // cannot wrap stream establishment. + var pipeline = Context.ResiliencePipeline.GetPipeline(_policy); + + // Establish the stream + first pull inside the pipeline; both are retriable. + // Callback disposes the FAILED attempt so retries don't leak enumerators. + // The pipeline's TState/TResult is the tuple, NOT the item type TResult. + async ValueTask<(IAsyncEnumerator, bool)> Establish(CancellationToken ct) + { + var e = next(query, ct).GetAsyncEnumerator(ct); + try { return (e, await e.MoveNextAsync()); } + catch { await e.DisposeAsync(); throw; } + } + + // Honour an ambient ResilienceContext, branching context-vs-token like the async decorator. + var resilienceContext = Context.ResilienceContext; + var (enumerator, moved) = resilienceContext != null + ? await pipeline.ExecuteAsync(ctx => Establish(ctx.CancellationToken), resilienceContext).ConfigureAwait(false) + : await pipeline.ExecuteAsync(ct => Establish(ct), cancellationToken).ConfigureAwait(false); + + await using (enumerator.ConfigureAwait(false)) + { + if (!moved) yield break; + do { yield return enumerator.Current; } // yielded only AFTER the pipeline succeeded + while (await enumerator.MoveNextAsync().ConfigureAwait(false)); + } + } +} +``` + +with `UseResiliencePipelineStreamAttribute(int step, string policy) : StreamQueryHandlerAttribute` +returning this decorator type. The ambient-`ResilienceContext` branch mirrors the context-vs-token +call-shapes in `UseResiliencePipelineHandlerAsync`. + +> **No `useTypePipeline` for streams (deliberate — differs from the async decorator).** The async +> decorator's `useTypePipeline` resolves a *result-type-scoped* `ResiliencePipeline`, which +> can only execute callbacks returning `ValueTask`. Stream establishment executes a callback +> returning `ValueTask<(IAsyncEnumerator, bool)>` — **not** a `TResult` — so a +> `ResiliencePipeline` is type-incompatible with it and the typed branch cannot compile. +> The stream decorator therefore supports only the **untyped** `GetPipeline(_policy)` (whose +> `ExecuteAsync` is generic over the callback's return type, here the tuple). If a +> per-`(key, item-type)` pipeline is ever wanted for streams, it would have to be keyed on the tuple +> type, not the item type; that is out of scope here. + +**Correctness property.** No item leaves the decorator until the pipeline has already succeeded, so +a failure during establishment/first-item retries a **fresh** enumerable and **cannot** re-emit an +already-yielded item. Once the first item is yielded, the pipeline has exited; subsequent +`MoveNextAsync` faults propagate. This is a well-defined **"resilience covers stream +establishment"** semantic. + +**Strategy applicability** (the pipeline wraps first-item acquisition only): + +- **Retry / circuit-breaker** — apply to establishment + first item. Well-defined. +- **Timeout** — bounds *getting the stream going*, **not** total enumeration time. Documented. +- **Fallback** — a fallback strategy may substitute an alternate `(enumerator, moved)`, i.e. fall + back to an alternate stream at establishment. +- **Hedging** — **unsupported** for streams (would race multiple enumerables → duplicate items and + loser-disposal complexity). Documented; not wired up. + +**Caveats baked into the design:** + +- The in-pipeline callback **disposes the enumerator on throw** so retried attempts don't leak + (the reference article omits this). +- The handler body up to the first `yield` **re-runs on each retry** — same as any retry; the + handler must tolerate repeated pre-first-item side effects. +- **Untyped pipeline only** (no `useTypePipeline`) — a `ResiliencePipeline` cannot wrap the + tuple-returning establishment callback (see the note above). The attribute has no + `useTypePipeline` parameter. +- **Cancellation is not special-cased.** `Establish`'s `catch` rethrows *any* exception — including + an `OperationCanceledException` from the first `MoveNextAsync` — back into the pipeline, so a + retry/circuit-breaker strategy will treat a *cancelled* first pull as a fault **unless the + caller's `ShouldHandle` excludes `OperationCanceledException`** (Polly's defaults exclude it when + the cancelling token matches the execution token). Darker authors no predicates itself; this is a + documented caller concern. +- **Fallback disposal is clean.** If the primary `Establish` created an enumerator and then threw, + that enumerator is disposed by `Establish`'s own `catch` *before* any Polly fallback strategy + fires. A fallback-substituted `(enumerator, moved)` is therefore the only enumerator the outer + `await using` owns — no double-dispose, no primary leak. + +### 4. Processor entry point — `IQueryProcessor.ExecuteStream` (async iterator, no `Task`) + +```csharp +IAsyncEnumerable ExecuteStream( + IStreamQuery query, + IQueryContext? queryContext = null, + CancellationToken cancellationToken = default); +``` + +- Added to the **existing `IQueryProcessor`**, not a separate `IStreamQueryProcessor`: dispatching a + query through a pipeline to its handler is the processor's single responsibility (coordinator + role); streaming is the same responsibility with a different result shape. One entry point = + "one obvious way." (V5 makes this an interface change, which NFR2 permits.) +- **No `Async` suffix and no `Task` wrapper**: the method is itself an `async IAsyncEnumerable` + iterator that returns the sequence directly; there is nothing to `await` on the call itself (the + `await` is in the caller's `await foreach`). This mirrors MediatR's `CreateStream` and the BCL + convention for `IAsyncEnumerable`-returning members. *(Naming — `ExecuteStream` vs + `ExecuteStreamAsync` — is the one point flagged for confirmation at review.)* + +The implementation ties span **and** pipeline lifetime to enumeration: + +```csharp +public async IAsyncEnumerable ExecuteStream( + IStreamQuery query, + IQueryContext? queryContext = null, + [EnumeratorCancellation] CancellationToken cancellationToken = default) +{ + // PipelineBuilder gains a new ctor slot for the stream registry (see §5); the handler & + // decorator factories are the reused async ones. + var pipelineBuilder = new PipelineBuilder(_streamHandlerRegistry, _handlerFactoryAsync, _decoratorFactoryAsync); + try + { + queryContext ??= _queryContextFactory.Create(); + InitQueryContext(queryContext); + var span = _tracer?.CreateQuerySpan(query, queryContext.Span, queryContext, _instrumentationOptions); + queryContext.Span = span; queryContext.Tracer = _tracer; + + var entryPoint = pipelineBuilder.BuildStream(query, queryContext, _instrumentationOptions); + try + { + await foreach (var item in entryPoint(query, cancellationToken).WithCancellation(cancellationToken).ConfigureAwait(false)) + yield return item; // lazy: item surfaces before the handler completes + } + finally { _tracer?.EndSpan(span); } + } + finally { pipelineBuilder.Dispose(); } // handler + decorators released when enumeration ends +} +``` + +- **Laziness (FR6/NFR1):** because `ExecuteStream` and every decorator are `async` iterators, no + body runs until the caller pulls; framework overhead is O(1) in result-set size. +- **Lifetime (A2):** the `finally` runs when the enumerator is disposed — i.e. when the caller's + `await foreach` completes *or* `break`s early — so handler/decorators/span are released exactly + once, at true end-of-stream, including early termination. +- **Exceptions:** faults raised during enumeration propagate through `MoveNextAsync` with their + original stack trace; no `TargetInvocationException` unwrap is required for the stream body + (argument-binding errors from `MethodInfo.Invoke`, which occur before enumeration, are still + unwrapped as today). +- **Deferred configuration errors (behavioural difference — documented, deliberate):** because the + whole method is one `async` iterator, `BuildStream` — and therefore handler resolution and + attribute-mismatch validation — runs on the **first `MoveNextAsync`**, not on the `ExecuteStream` + call. So a missing/mismatched handler surfaces its `ConfigurationException` from the caller's first + `await foreach` iteration, whereas `Execute`/`ExecuteAsync` throw eagerly at call time + (`QueryProcessor.cs` builds before returning). We accept this: it is the price of the + leak-on-abandon safety above (resolving the handler eagerly in a non-iterator wrapper would either + leak the handler/`QueryLifetimeScope` when the caller never enumerates, or require splitting + `PipelineBuilder` into a non-allocating "resolve type + validate" phase and a separate + "create instances" phase). The error still surfaces immediately on first use and is unambiguous, so + the simpler single-iterator shape wins. *(Alternative — eager type-only resolution — noted under + Alternatives Considered.)* +- **Multiple / concurrent enumeration:** the returned `IAsyncEnumerable` is **cold and not + cached**. Each `await foreach` (each `GetAsyncEnumerator` call) starts a fresh iterator state + machine, hence a fresh `PipelineBuilder`, handler, and decorators — so re-enumerating + **re-executes and re-traces** the query. This differs from `ExecuteAsync`, whose materialised + `TResult` can be re-read freely. Callers who need to iterate twice should buffer (e.g. + `ToListAsync`) themselves. This is intended; it is not a single-execution/replayable stream. + - **Context ownership caveat (concurrency safety).** Safe concurrent/repeat enumeration requires + the processor to **own the context**: when `queryContext == null` each enumeration gets a fresh + `_queryContextFactory.Create()` and its own span, so the pipelines are truly independent. When + the caller **passes** a non-null `queryContext`, that single instance is shared and the iterator + mutates `queryContext.Span` / `.Tracer` on it; two overlapping enumerations would then race on + those single-valued properties (one `EndSpan` could stop a span the other is still writing, and + decorators reading `Context.Span` per FR9 could observe a torn value). **Decision:** a + caller-supplied `queryContext` is therefore scoped to a **single enumeration**; concurrent or + repeated enumeration is supported only on the processor-created-context path. This is documented + on `ExecuteStream`; a future option is to snapshot/clone a supplied context per enumeration. + +### 5. Handler resolution — new `IStreamQueryHandlerRegistry`, **reused** factories + +```csharp +public interface IStreamQueryHandlerRegistry +{ + Type Get(Type queryType); + void Register() + where TQuery : IStreamQuery + where THandler : IStreamQueryHandler; + void Register(Type queryType, Type resultType, Type handlerType); +} +``` + +- A **new registry** is warranted because its typed `Register` constrains to `IStreamQueryHandler` + (a distinct role — "knowing which stream handler serves a stream query"). +- The **handler and decorator factories are reused** (`IQueryHandlerFactoryAsync`, + `IQueryHandlerDecoratorFactoryAsync`): their `Create(Type, IAmALifetime) : IQueryHandler` / + generic `Create` already resolve any `IQueryHandler` / `IQueryHandlerDecorator` by type, and + `IStreamQueryHandler : IQueryHandler`, `IStreamQueryHandlerDecorator : IQueryHandlerDecorator`. + We add new types only where the role genuinely differs, reusing where the responsibility is + identical. +- **Registration mirrors async handlers *for the user*, but is net-new plumbing *internally*** — it + is NOT a reuse of the existing scan. The current assembly scan hard-matches a single closed + interface (`i.GetGenericTypeDefinition() == typeof(IQueryHandlerAsync<,>)` in + `QueryHandlerRegistryAsync.RegisterFromAssemblies`), so it will **never** pick up an + `IStreamQueryHandler<,>`. Delivering FR4 therefore requires the following explicit wiring, all of + which this ADR treats as in scope: + 1. A **new stream `RegisterFromAssemblies`** that matches `typeof(IStreamQueryHandler<,>)`, plus + the concrete `StreamQueryHandlerRegistry` implementing `IStreamQueryHandlerRegistry`. + 2. `IHandlerConfiguration` / `HandlerConfiguration` gain a **`StreamHandlerRegistry`** member + (optional; null when streaming is unused). + 3. **`PipelineBuilder` gains a new constructor parameter** for the stream registry (its + current ctor has slots only for the sync + async registries/factories); `BuildStream` resolves + the handler type from it. + 4. **`QueryProcessor`** reads `StreamHandlerRegistry` off the configuration and threads it (with + the reused async factories) into the `PipelineBuilder` it constructs in `ExecuteStream` (§4). + 5. The DI `AddDarker(...).AddHandlers...` builder invokes the new stream scan alongside the async + one, so from the *user's* perspective registration looks identical — but none of the above is + shared code with the async path. + +### 6. Pipeline construction — `PipelineBuilder.BuildStream` + +Add a third build method alongside `Build` / `BuildAsync`: + +```csharp +public Func, CancellationToken, IAsyncEnumerable> BuildStream( + IStreamQuery query, IQueryContext queryContext, InstrumentationOptions options); +``` + +The **shared, factored** steps with `BuildAsync` are: resolve the handler from the (stream) +registry, validate attributes, and order decorators by `Step` descending. Two things **differ** and +must not be glossed: + +1. **Generic close.** `BuildAsync` closes decorators as + `GetDecoratorType().MakeGenericType(typeof(IQuery), typeof(TResult))` + (`PipelineBuilder.cs:243,:279`). Stream decorators are constrained + `where TQuery : IStreamQuery`, so `BuildStream` must close them (and the handler) over + **`typeof(IStreamQuery)`**, not `IQuery` — closing with `IQuery` would + violate the constraint. +2. **Sink + delegate type.** The delegates return `IAsyncEnumerable` (not `Task`); + the sink invokes the handler's `ExecuteAsync` returning the enumerable, with **no** + `TargetInvocationException` unwrap around enumeration (§4 — the iterator defers, so `Invoke` + returns the enumerable without running the body). + +Attribute-mismatch validation reuses the existing `ValidateNoMismatchedAttributes(MemberInfo, Type, +string)` guard (`PipelineBuilder.cs:184`), which is driven by the attribute **base type** (not the +method name), rejecting `QueryHandlerAttribute` / `QueryHandlerAttributeAsync` on a stream handler's +`ExecuteAsync` (and `StreamQueryHandlerAttribute` on sync/async handlers), with a clear +`ConfigurationException`. + +**Method resolution must be by signature, not by bare name.** The existing async resolver is +`handlerType.GetMethod("ExecuteAsync")` with no argument types (`PipelineBuilder.cs:179`). A stream +handler's method is *also* named `ExecuteAsync` (§2), so `BuildStream` must resolve the stream method +by its **signature** — return type `IAsyncEnumerable`, parameters `(TQuery, +CancellationToken)` — rather than a bare-name lookup, to avoid binding to an async `Task` +`ExecuteAsync` and to avoid `AmbiguousMatchException` on any type that exposed both. + +### 7. Target frameworks — all current targets + +Support streaming on **`netstandard2.0;net8.0;net9.0`** (Darker's existing targets). `IAsyncEnumerable` +is native on net8.0+; on `netstandard2.0` add a conditional +`Microsoft.Bcl.AsyncInterfaces` package reference (small, Microsoft-owned, managed via CPM in +`Directory.Packages.props`). This avoids `#if`-guarding the streaming API and keeps one consistent +surface across targets — simpler for users than a net8.0-only streaming path. + +**NFR4 (AOT/trimming).** `BuildStream` uses the same `MakeGenericType` + `MethodInfo.Invoke` +reflection the existing `BuildAsync` already uses (`PipelineBuilder.cs:243,:144`); the project is +`IsAotCompatible` on net8.0+ (`Paramore.Darker.csproj`) with the existing `IL2026/IL3050` +suppressions. Streaming introduces no reflection *beyond* the current pipeline, so the existing AOT +posture carries over unchanged — satisfying NFR4's wording ("must not introduce trim/AOT-hostile +reflection beyond what the existing pipeline already uses"). + +### Architecture Overview + +``` +Caller + │ await foreach (var item in processor.ExecuteStream(query, ct)) + ▼ +IQueryProcessor.ExecuteStream (async iterator: owns span + pipeline lifetime) + │ creates IQueryContext + span, builds pipeline, yields items, releases on enumerator dispose + ▼ +PipelineBuilder.BuildStream (resolves handler from IStreamQueryHandlerRegistry) + │ chains IStreamQueryHandlerDecorator (attrs ordered by Step) innermost→outermost + ▼ Func, CancellationToken, IAsyncEnumerable> +[ logging/telemetry decorator ] → … → [ handler.ExecuteAsync ] (all async iterators — lazy) +``` + +### Key Components + +| Component | Role (RDD) | New / Reused | +|---|---|---| +| `IStreamQuery` | information holder — "a query that yields a stream" | **New** | +| `IStreamQueryHandler` / `StreamQueryHandler<,>` | service provider — produces the stream | **New** | +| `IStreamQueryHandlerDecorator` + `StreamQueryHandlerAttribute` | interfacer — wraps the stream | **New** | +| `UseResiliencePipelineStreamHandler` + `UseResiliencePipelineStreamAttribute` | interfacer — applies a Polly v8 pipeline at stream establishment | **New** (reuses pipeline-resolution logic from the async decorator) | +| `IStreamQueryHandlerRegistry` | information holder — query type → stream handler type | **New** | +| `IQueryProcessor.ExecuteStream` | coordinator — dispatch + lifetime | Extends existing role | +| `PipelineBuilder.BuildStream` | structurer — assemble the chain | Extends existing type | +| `IQueryHandlerFactoryAsync`, `IQueryHandlerDecoratorFactoryAsync`, `IAmALifetime`, `IQueryContext` | service providers | **Reused unchanged** | + +### Technology Choices + +- `System.Collections.Generic.IAsyncEnumerable` + C# 8 `async` iterators (`await foreach` / + `yield return`) — the idiomatic, lazy, cancellable streaming primitive. +- `Microsoft.Bcl.AsyncInterfaces` — only on `netstandard2.0`, to provide `IAsyncEnumerable`. +- `[EnumeratorCancellation]` — to unify the method token with `WithCancellation(ct)`. + +### Implementation Approach + +Delivered TDD-first (`/test-first` per task). Rough sequence, structural-before-behavioural +(Tidy First): (1) contracts `IStreamQuery` / `IStreamQueryHandler` / base class; (2) +`IStreamQueryHandlerRegistry` + concrete registry; (3) `PipelineBuilder.BuildStream` sink (no +decorators) + `IQueryProcessor.ExecuteStream`; (4) decorator contract + attribute + chain build + +mismatch validation; (5) logging/telemetry stream decorators; (6) `UseResiliencePipelineStreamHandler` ++ attribute (reusing pipeline resolution); (7) DI scanning + `HandlerConfiguration`; +(8) `netstandard2.0` package + multi-target build. Each behaviour (happy path, laziness, +cancellation mid-stream, exception mid-stream, early-`break` lifetime release, decorator ordering, +resilience: retry-before-first-item with **no** duplicate emission, failed-attempt disposal) gets a +failing test approved before implementation. + +## Consequences + +### Positive + +- Large/unbounded/real-time result sets stream with O(1) framework overhead; first item is + observable before the handler finishes. +- The streaming path reads as a natural sibling of the async path; factories, lifetime, and context + are reused, so there is little new surface to learn. +- Span and handler/decorator lifetime are correctly bound to enumeration (including early + termination) — a subtle correctness win over a naive overload. +- Enumeration faults propagate with original stack traces without reflection unwrap ceremony. +- **Polly v8 resilience (retry / timeout / circuit-breaker / fallback) is available for streams** + via `UseResiliencePipelineStreamHandler`, reusing the existing pipeline-resolution logic, with a + well-defined "covers establishment + first item, no duplicate emission" semantic — no capability + cliff versus the request/response path for the common case. + +### Negative + +- Net-new public types (query/handler/decorator/attribute/registry/resilience decorator) enlarge + the API surface. +- Stream resilience covers **establishment + first item only**: mid-stream faults after the first + item are not retried, and `Timeout` bounds start-up rather than total enumeration. `Hedging` is + unsupported for streams. These are documented semantics, not bugs, but differ from the + request/response pipeline. +- The legacy `RetryableQuery` / `FallbackPolicy` (handler-method) attributes do **not** apply to + streams; stream resilience goes exclusively through the resilience-pipeline decorator. +- `PipelineBuilder` grows a third build path, adding some duplication with `BuildAsync`. +- `netstandard2.0` gains a `Microsoft.Bcl.AsyncInterfaces` dependency. +- Adding `ExecuteStream` to `IQueryProcessor` is a breaking interface change for custom + implementers (acceptable under V5 / NFR2). + +### Risks and Mitigations + +- **Risk: accidental buffering** in a decorator (e.g. `ToListAsync`) silently defeats streaming. + *Mitigation:* a laziness test asserting the first item is observed before the handler produces + the last; document the "don't buffer" rule on the decorator contract. +- **Risk: span/handler leak** if enumeration is abandoned without disposal. *Mitigation:* the + `finally` in the processor iterator runs on enumerator `DisposeAsync`, which `await foreach` + always calls (including on `break`/exception); covered by an early-termination lifetime test. +- **Risk: users apply the legacy `RetryableQuery` / `FallbackPolicy` attributes to a stream and + expect them to work.** *Mitigation:* the attribute mismatch validator throws a clear + `ConfigurationException` (those are `QueryHandlerAttributeAsync`, not `StreamQueryHandlerAttribute`); + the doc points users to `UseResiliencePipelineStream` instead. +- **Risk: enumerator leak on resilience retry.** Each failed attempt inside the pipeline created an + enumerator and called `MoveNextAsync`. *Mitigation:* the in-pipeline callback disposes the + enumerator on throw before rethrowing (the reference article omits this); covered by a test that + asserts N failed attempts ⇒ N disposals. +- **Risk: users assume stream resilience covers the whole stream** (mid-stream retry, total-time + timeout). *Mitigation:* documented as "establishment + first item only"; a retry test asserts a + fault *after* the first item is **not** retried and items are **not** re-emitted. +- **Risk: `BuildAsync`/`BuildStream` divergence** over time. *Mitigation:* keep the shared + resolve/validate/order steps factored; the two differ only in return type and sink. + +## Alternatives Considered + +- **Reuse `IQuery>` with the existing async pipeline.** Rejected: routes + as `Task>`, invites buffering, muddies laziness/lifetime, and reads + poorly. A dedicated marker is clearer and lets the pipeline dispatch on type. +- **Separate `IStreamQueryProcessor`.** Rejected: fragments the processor's single dispatch + responsibility across two roles for no user benefit; MediatR keeps `CreateStream` on `ISender`. +- **Reuse `IQueryHandlerDecoratorAsync` for stream decorators.** Rejected: its `next`/`fallback` + are `Func<…, Task>`; a stream decorator must operate over `IAsyncEnumerable`. + A distinct contract + attribute is required and enables the mismatch guard. +- **`ExecuteStreamAsync` returning `Task>`.** Rejected: the extra `Task` + is pure ceremony (the method returns the enumerable synchronously) and misrepresents the model. +- **No resilience for streams in V5** (the earlier draft position). Superseded: Varnon's + first-`MoveNextAsync`-inside-the-pipeline pattern gives a safe, no-duplicate-emission semantic at + modest cost, reusing the existing pipeline-resolution logic, so §3a adopts it rather than + deferring. +- **Wrap *each* `MoveNextAsync` in the pipeline (per-item resilience).** Rejected: re-calling + `MoveNextAsync` on an enumerator that already threw is undefined/terminal — you cannot resume; the + only way to "retry item N" is to rebuild the whole stream and skip N, re-running side effects and + requiring positional idempotency. Not viable generically. Wrapping only the first item is the + safe choice. +- **Reuse the async `UseResiliencePipelineHandlerAsync` directly.** Rejected: wrapping the call that + returns the enumerable protects nothing (iterator bodies defer); a stream-specific decorator that + pulls the first item inside the pipeline is required. +- **Eager handler resolution / config validation before the iterator defers** (a non-iterator + `ExecuteStream` that resolves + validates, then returns an inner iterator). Rejected for the base + design: resolving the handler eagerly either leaks the handler + `QueryLifetimeScope` when the + caller never enumerates (breaking the leak-on-abandon property), or forces splitting + `PipelineBuilder` into a non-allocating "resolve type + validate attributes" phase and a separate + "create instances + run" phase. The single-iterator shape is simpler and the config error still + surfaces unambiguously on first enumeration (§4). A future refinement could add type-only eager + validation if the deferred error proves surprising in practice. +- **net8.0+ only for streaming.** Rejected in favour of all-targets via + `Microsoft.Bcl.AsyncInterfaces`: avoids `#if` fragmentation and a split user story. + +## References + +- Requirements: [specs/012-streaming_results/requirements.md](../../specs/012-streaming_results/requirements.md) + (includes verbatim MediatR streaming signatures as prior art) +- Linked issue: [#299](https://github.com/BrighterCommand/Darker/issues/299) +- Related ADRs: `0017-query-tracing-and-database-spans.md` (span lifecycle this ADR must respect), + `0016-pipeline-attribute-memoization.md` (attribute resolution the stream builder reuses) +- Current code: `src/Paramore.Darker/PipelineBuilder.cs`, `src/Paramore.Darker/QueryProcessor.cs`, + `src/Paramore.Darker/IQueryHandlerDecoratorAsync.cs`, + `src/Paramore.Darker/IQueryHandlerRegistryAsync.cs`, + `src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineHandlerAsync.cs` (the async resilience + decorator whose pipeline-resolution logic §3a reuses) +- External: MediatR streaming — +- External: A. Varnon, *Extending Polly retry policies to cover IAsyncEnumerables* — + + (the first-`MoveNextAsync`-inside-the-policy pattern adopted, with added failed-attempt disposal) diff --git a/specs/.current-spec b/specs/.current-spec index c79d9855..4488f283 100644 --- a/specs/.current-spec +++ b/specs/.current-spec @@ -1 +1 @@ -011-telemetry \ No newline at end of file +012-streaming_results \ No newline at end of file diff --git a/specs/012-streaming_results/.adr-list b/specs/012-streaming_results/.adr-list new file mode 100644 index 00000000..78ba67e5 --- /dev/null +++ b/specs/012-streaming_results/.adr-list @@ -0,0 +1 @@ +0019-streaming-query-pipeline.md diff --git a/specs/012-streaming_results/.design-approved b/specs/012-streaming_results/.design-approved new file mode 100644 index 00000000..e69de29b diff --git a/specs/012-streaming_results/.issue-number b/specs/012-streaming_results/.issue-number new file mode 100644 index 00000000..f491e22f --- /dev/null +++ b/specs/012-streaming_results/.issue-number @@ -0,0 +1 @@ +299 \ No newline at end of file diff --git a/specs/012-streaming_results/.requirements-approved b/specs/012-streaming_results/.requirements-approved new file mode 100644 index 00000000..e69de29b diff --git a/specs/012-streaming_results/README.md b/specs/012-streaming_results/README.md new file mode 100644 index 00000000..7a3ebe7d --- /dev/null +++ b/specs/012-streaming_results/README.md @@ -0,0 +1,29 @@ +# Streaming Results + +**Spec ID:** 012-streaming_results +**Created:** 2026-07-09 +**Status:** Design ✅ approved — ADR 0019 `Accepted` (3 adversarial rounds); ready for task breakdown + +## Overview + +_To be defined during requirements._ + +Add support for streaming query results (e.g. `IAsyncEnumerable`) through Darker's +query pipeline, so handlers can yield results incrementally rather than materialising a full +result set. Requirements, design, and scope to be established in the workflow below. + +## Status Checklist + +- [x] Requirements (`/spec:requirements`) — ✅ approved +- [x] Design / ADR (`/spec:design`) — ✅ ADR 0019 `Accepted` +- [x] Adversarial Review — 3 rounds (6 + 4 + 3 findings) all resolved in ADR 0019 (see `review-design.md`) +- [ ] Task Breakdown (`/spec:tasks` or `/spec:ralph-tasks`) +- [ ] Implementation (`/spec:implement` or `/spec:ralph-implement`) + +## Documents + +| Phase | File | Status | +|-------|------|--------| +| Requirements | `requirements.md` | ⬜ Not started | +| Design | `docs/adr/0019-streaming-query-pipeline.md` | ✅ Accepted | +| Tasks | `tasks.md` | ⬜ Not started | diff --git a/specs/012-streaming_results/requirements.md b/specs/012-streaming_results/requirements.md new file mode 100644 index 00000000..6e26d6f2 --- /dev/null +++ b/specs/012-streaming_results/requirements.md @@ -0,0 +1,241 @@ +# Requirements + +> **Note**: This document captures user requirements and needs. Technical design decisions and implementation details should be documented in an Architecture Decision Record (ADR) in `docs/adr/`. + +**Linked Issue**: [#299](https://github.com/BrighterCommand/Darker/issues/299) — Add streaming query support with `IAsyncEnumerable` + +## Problem Statement + +Today Darker only supports request/response queries: a query returns a single, fully-materialised +`TResult` via `IQueryProcessor.Execute` / `ExecuteAsync`. For large result sets, paged data, or +real-time feeds this forces the whole result into memory before the caller sees the first item. + +> **As an** application developer using Darker, +> **I would like** to execute a query that yields its results incrementally as an +> `IAsyncEnumerable`, +> **so that** I can process, forward, or render items as they arrive — enabling large/unbounded +> result sets, server-sent events, and gRPC server-streaming — without buffering the entire result +> set in memory. + +## Proposed Solution + +Add a first-class streaming path to Darker that mirrors the existing async request/response path: + +- A way to declare a **streaming query** whose result is a stream of `TResult` items. +- A **streaming handler** contract that yields items via + `IAsyncEnumerable ExecuteAsync(TQuery query, CancellationToken ct)`. +- A **processor entry point** to execute a streaming query and consume it with `await foreach`. +- Streaming queries flow through the **same attribute-driven decorator pipeline** as ordinary + queries, with clearly defined semantics for each decorator kind (see below). + +From the caller's perspective: + +```csharp +await foreach (var item in queryProcessor.ExecuteStreamAsync(new MyStreamQuery(), ct)) +{ + // handle each item as it is produced +} +``` + +The precise API shape (`IStreamQuery` vs `IQuery>`, a new +`ExecuteStreamAsync` on `IQueryProcessor` vs a separate `IStreamQueryProcessor`, and whether this +lives in core or a separate package) is an **open design decision deferred to the ADR** — this +document records the required behaviour, not the mechanism. + +## Requirements + +### Functional Requirements + +- **FR1 — Streaming query contract**: Provide a way to declare a query whose result is a stream of + `TResult` items, distinct from an ordinary `IQuery`. +- **FR2 — Streaming handler contract**: Provide a handler contract exposing + `IAsyncEnumerable ExecuteAsync(TQuery query, CancellationToken cancellationToken)`. +- **FR3 — Processor entry point**: Provide a processor method that returns + `IAsyncEnumerable` for a streaming query so callers can `await foreach` over it. +- **FR4 — Handler registration & resolution**: Streaming queries resolve to their handler via the + existing registry/factory mechanism, consistent with how sync/async handlers are registered + (including DI registration via `AddDarker(...).AddHandlers...`). +- **FR5 — Cancellation**: The `CancellationToken` passed to the processor flows to the handler and + observing cancellation stops enumeration promptly; `[EnumeratorCancellation]` semantics are + honoured for `await foreach (... .WithCancellation(ct))`. +- **FR6 — Lazy / incremental execution**: The handler body must not run to completion before the + first item is observed; items are produced on demand as the caller enumerates (no eager + buffering of the full result set by the framework). +- **FR7 — Decorator pipeline semantics** for streaming queries must be explicitly defined per + decorator category: + - **Logging / Telemetry**: wrap the whole stream lifecycle (start, completion, item count and/or + per-item as designed), and record exceptions raised during enumeration. + - **Retry**: define semantics explicitly — retrying mid-stream can re-emit already-yielded items, + so behaviour must be specified (e.g. retry only before first item, or documented as + unsupported / no-op for streams). + - **Fallback**: define how fallback applies when enumeration faults (e.g. fall back to an + alternate stream, or documented as unsupported). + - **Caching** (if/when present): documented behaviour for streams (likely not applicable). +- **FR8 — Coexistence**: Streaming and request/response querying coexist in one model. Streaming is + preferably additive, but per NFR2 the request/response surface (`Execute` / `ExecuteAsync`, query + and handler contracts) **may be reshaped** if doing so produces a cleaner unified design for V5. + The end state must still support ordinary single-result queries. +- **FR9 — Query context**: A streaming query has access to `IQueryContext` (bag, policies, tracing + span) consistent with the async path, for the lifetime of the stream. + +### Non-functional Requirements + +- **NFR1 — Memory**: Framework overhead per stream is O(1) with respect to result-set size; the + framework does not materialise the full sequence. +- **NFR2 — Simplicity over compatibility (V5)**: This feature targets the **V5** major release, so + **breaking changes to existing public APIs are permitted where they yield a simpler, more elegant + design**. Prefer the cleanest streaming model even if it means changing `IQueryProcessor`, + `IQuery`, handler base classes, or decorator interfaces. Breaking changes are not a goal + in themselves — introduce them only when they materially improve clarity/elegance, and record + each one (with migration notes) in the ADR and release notes. +- **NFR3 — Target frameworks**: Must build across Darker's current targets. `IAsyncEnumerable` + is native on `net8.0`/`net9.0`; on `netstandard2.0` it requires `Microsoft.Bcl.AsyncInterfaces`. + Whether `netstandard2.0` is supported for streaming, or streaming requires `net8.0+`, is a design + decision for the ADR (issue #299 suggests .NET 8+). +- **NFR4 — AOT / trimming**: Preserve existing AOT-compatibility posture (`IsAotCompatible` on + net8.0+); the streaming path must not introduce trim/AOT-hostile reflection beyond what the + existing pipeline already uses. +- **NFR5 — Consistency**: The streaming API should feel like a natural sibling of the existing + async API (naming, cancellation, context, DI registration) to minimise the learning curve. + +### Constraints and Assumptions + +- **C1**: Darker is in-process, query-side only (CQRS read side). Streaming here means in-process + `IAsyncEnumerable`, not a network/transport streaming protocol — transport concerns (SSE, gRPC) + are the caller's responsibility. +- **C2**: The existing pipeline builds decorators from attributes on the handler's execute method + and invokes via reflection (`PipelineBuilder`). Streaming must integrate with, or deliberately + parallel, this mechanism. +- **C3**: `netstandard2.0` support for `IAsyncEnumerable` requires an added dependency + (`Microsoft.Bcl.AsyncInterfaces`); adding it is subject to the dependency-management guidelines. +- **A1**: Assumes consumers target a runtime where `IAsyncEnumerable` is available (net8.0+, or + netstandard2.0 with the BCL async-interfaces package). +- **A2**: Assumes the same handler/decorator lifetime model (created per-query, released after the + pipeline completes) — for streams, "completion" means the stream is fully enumerated or disposed. + +### Out of Scope + +- **OOS1**: Synchronous streaming (`IEnumerable` / `yield return` on a sync handler). Only + `IAsyncEnumerable` is in scope. +- **OOS2**: Transport-level streaming implementations (server-sent events endpoints, gRPC service + wiring). Samples may demonstrate consumption but the transport is not part of Darker. +- **OOS3**: Back-pressure/flow-control primitives beyond what `IAsyncEnumerable` + `await foreach` + naturally provide. +- **OOS4**: Bidirectional or client-streaming patterns; only server-to-caller result streaming. +- **OOS5**: Gratuitous changes to request/response *behaviour/semantics*. Note: per NFR2/FR8, V5 + *structural* reshaping of the request/response API (signatures, contracts) is permitted where it + yields a cleaner unified model — what's out of scope is changing what existing queries *do* + without a design reason. +- **OOS6**: Caching of streamed results. + +## Acceptance Criteria + +**Definition of done:** + +- A streaming query + handler can be defined, registered, and executed, and the caller can + `await foreach` the results. +- Items are produced lazily (verifiable: a handler that logs/records per-item production shows the + first item observed before the handler has produced the last). +- Cancellation via the token stops enumeration promptly and propagates + `OperationCanceledException` as expected. +- The decorator pipeline runs for a streaming query with the semantics defined in FR7, and each + decorator category's behaviour is covered by a test. +- Exceptions thrown during enumeration surface to the caller (unwrapped, preserving stack trace, + consistent with the existing `TargetInvocationException` handling). +- The solution builds and its test suite is green on all supported target frameworks (NFR3). Where + V5 breaking changes reshape the request/response surface (NFR2), existing tests are updated to + the new API rather than treated as a compatibility contract; any behavioural change is deliberate + and documented. + +**Testing approach:** + +- TDD per the mandatory `/test-first` workflow: each behaviour is specified by a failing test that + is approved before implementation. +- Use real/Simple/InMemory test doubles (registries, `SimpleHandlerFactory`, + `InMemoryDecoratorRegistry`, `InMemoryQueryContextFactory`) per project conventions; Moq only as + a last resort. +- Cover: happy-path enumeration, lazy production, cancellation mid-stream, exception mid-stream, + and each decorator category from FR7. + +**Success metrics:** + +- Feature parity intent with MediatR's `IStreamRequest` / `IStreamRequestHandler` model, + adapted to Darker's decorator pipeline. +- Zero breaking changes to existing public API. + +## Additional Context + +### Prior art — MediatR streaming + +Reviewed and pulled the **verbatim +signatures from the MediatR source** (jbogard/MediatR, `master`). MediatR keeps the streaming path +*fully parallel* to request/response — a separate marker, a separate execute method, and a separate +behavior pipeline: + +```csharp +// src/MediatR.Contracts/IStreamRequest.cs +public interface IStreamRequest { } + +// src/MediatR/IStreamRequestHandler.cs +public interface IStreamRequestHandler + where TRequest : IStreamRequest +{ + IAsyncEnumerable Handle(TRequest request, CancellationToken cancellationToken); +} + +// src/MediatR/ISender.cs — sibling of Send(...), NOT an overload of it +IAsyncEnumerable CreateStream( + IStreamRequest request, CancellationToken cancellationToken = default); +IAsyncEnumerable CreateStream( + object request, CancellationToken cancellationToken = default); + +// src/MediatR/IStreamPipelineBehavior.cs — distinct from IPipelineBehavior +public delegate IAsyncEnumerable StreamHandlerDelegate(); + +public interface IStreamPipelineBehavior + where TRequest : notnull +{ + IAsyncEnumerable Handle( + TRequest request, StreamHandlerDelegate next, CancellationToken cancellationToken); +} +``` + +How this lands on our deferred design questions: + +- **Q1 (marker)**: MediatR uses a dedicated `IStreamRequest` marker, *not* an + overload of `IRequest`. Prior art for a dedicated `IStreamQuery`. +- **Q2 (execute method)**: `CreateStream` is a **sibling** of `Send` on `ISender`, returning + `IAsyncEnumerable` directly (no `Task` wrapper). Prior art for `ExecuteStreamAsync` + as its own method rather than reusing `ExecuteAsync`. +- **Q5 + FR7 (decorators)**: `IStreamPipelineBehavior` is a **separate + contract** from `IPipelineBehavior`; `next` is a parameterless `StreamHandlerDelegate` + returning `IAsyncEnumerable` (contrast with Darker's `IQueryHandlerDecoratorAsync` + whose `next` is `Func>`). Strong prior art that Darker + needs a stream-specific decorator contract, not a reuse of the async one. +- **FR5 (cancellation)**: the handler takes a `CancellationToken` directly; MediatR's docs stress + `[EnumeratorCancellation]` for correct `IAsyncEnumerable` cancellation. +- **FR6/NFR1 (lazy)**: MediatR warns handlers not to buffer all items — incremental, not eager. + +> Naming note: Darker uses `ExecuteAsync`/`Execute` where MediatR uses `Send`, so the Darker +> equivalents would read as `ExecuteStreamAsync` (processor) and a `Handle`-style stream method on +> the handler (or `ExecuteAsync` returning `IAsyncEnumerable`) — final naming is an ADR +> decision. + +Implication for Darker: MediatR's model is *fully parallel* (separate marker, separate execute +method, separate behavior pipeline) rather than overloading the request/response path. The ADR +should weigh adopting that same fully-parallel shape versus a lighter integration into the existing +`PipelineBuilder`. +- Codebase grounding (current async model to mirror): + - `IQueryProcessor` — `src/Paramore.Darker/IQueryProcessor.cs` (`ExecuteAsync`) + - Async handler — `src/Paramore.Darker/IQueryHandlerAsync.cs`, + `src/Paramore.Darker/QueryHandlerAsync.cs` + - Async decorator — `src/Paramore.Darker/IQueryHandlerDecoratorAsync.cs` + - Pipeline construction — `src/Paramore.Darker/PipelineBuilder.cs` (`BuildAsync` Func chain) + - Target frameworks — `src/Paramore.Darker/Paramore.Darker.csproj` + (`netstandard2.0;net8.0;net9.0`) +- Key **open design questions for the ADR** (deferred, not decided here): + 1. `IStreamQuery` marker vs reusing `IQuery>`. + 2. `ExecuteStreamAsync` on `IQueryProcessor` vs a separate `IStreamQueryProcessor`. + 3. Core library vs a separate `Paramore.Darker.Streaming` package. + 4. `netstandard2.0` support (via `Microsoft.Bcl.AsyncInterfaces`) vs net8.0+ only. + 5. Retry/fallback semantics for a faulting stream (supported, restricted, or no-op). diff --git a/specs/012-streaming_results/review-design.md b/specs/012-streaming_results/review-design.md new file mode 100644 index 00000000..c908df99 --- /dev/null +++ b/specs/012-streaming_results/review-design.md @@ -0,0 +1,215 @@ +# Review: design — 012-streaming_results (ADR 0019) + +**Date**: 2026-07-09 +**Threshold**: 60 +**Verdict**: NEEDS WORK + +## Findings + +### 1. `IStreamQuery : IQuery` cannot use the `Query` base class, and breaks the `CreateQuerySpan` call (Score: 90) + +§1 asserts two things that the code contradicts. It declares `IStreamQuery : IQuery` (the **non-generic** marker) and simultaneously claims this "shares the `Query` base class (stable `Id`, tracing)" while "remaining distinct from `IQuery` at the type level." Both cannot hold. + +- The actual base class is `public abstract class Query : IQuery` — it derives from the **generic** `IQuery`. A stream query deriving from `Query` to get the stable `Id` would therefore also be an `IQuery`, contradicting §1's "distinct from `IQuery`." +- §4's processor calls `_tracer?.CreateQuerySpan(query, ...)`, but the signature is `Activity? CreateQuerySpan(IQuery query, ...)`. An `IStreamQuery` implementing only non-generic `IQuery` does not satisfy the `IQuery` parameter → does not compile. +- `CreateQuerySpan` also does `query is Query q ? q.Id : null` (DarkerTracer.cs:70), which only yields the id if the stream query is a `Query` (i.e. an `IQuery`). + +**Evidence**: ADR §1 "Derives from the **non-generic `IQuery`** marker so it shares the `Query` base class ... while remaining distinct from `IQuery`". Code: `src/Paramore.Darker/Observability/Query.cs:16` `public abstract class Query : IQuery`; `src/Paramore.Darker/Observability/DarkerTracer.cs:47` `CreateQuerySpan(IQuery query, ...)` and line 70 `query is Query q`. ADR §4 passes an `IStreamQuery` to that method. + +**Recommendation**: Resolve the type model explicitly. Either (a) make `IStreamQuery : IQuery` (accepting stream queries are also `IQuery`, and move the stream-vs-single dispatch to the handler registry rather than the marker), or (b) introduce a stream-specific span-creation overload / a shared non-generic id-bearing base and state stream queries do NOT reuse `Query`. As written, §1 + §4 do not compile against the current tracer. + +--- + +### 2. §4 span creation runs inside the deferred iterator, so configuration/resolution errors surface on first enumeration, not on the `ExecuteStream` call (Score: 62) + +§4 places `CreateQuerySpan`, `InitQueryContext`, and `BuildStream` inside the `async IAsyncEnumerable` iterator body. An async iterator defers **all** body execution until the first `MoveNextAsync`, so the span/context/handler-resolution happen only when the caller starts enumerating. The ADR correctly leans on this for the "never-enumerated ⇒ no leak" property (which is sound), but does not acknowledge the flip side: `BuildStream` resolves the handler and can throw `ConfigurationException` (no handler registered, mismatched attributes) — with this shape that throws from the **first `MoveNextAsync`**, not from the `ExecuteStream` call, unlike `Execute`/`ExecuteAsync` which throw eagerly at build time. + +**Evidence**: ADR §4 builds the pipeline and span inside the iterator body. `src/Paramore.Darker/QueryProcessor.cs:71`,`:110` build eagerly today; `ConfigurationException` from `PipelineBuilder` (e.g. PipelineBuilder.cs:196,:127) surfaces synchronously. The ADR discusses leak-on-abandon but never the deferred-exception semantics. + +**Recommendation**: Document that stream configuration/resolution errors surface on first enumeration; decide whether that is acceptable or whether to eagerly validate (split a non-iterator outer method that resolves the handler, then an inner iterator that enumerates). + +--- + +### 3. Multiple / concurrent enumeration of the returned `IAsyncEnumerable` is not addressed (Score: 60) + +The ADR is silent on a stream query "executed once but enumerated twice." With the §4 shape, each `await foreach` spins up a fresh async-iterator state machine → fresh `PipelineBuilder`, handler, and span, so re-enumeration re-executes (and re-traces) the whole query. Defensible, but a sharp semantic difference from `ExecuteAsync` (materialised, re-readable `TResult`) and from any "the query ran once" expectation. Concurrent enumeration runs two independent pipelines/handlers/spans; per-enumeration span duplication has tracing implications. + +**Evidence**: ADR §4 is a plain `async IAsyncEnumerable` iterator; nothing memoises or guards re-enumeration. No mention in Decision/Consequences/Risks. + +**Recommendation**: Add an explicit paragraph: re-enumeration re-executes and re-traces; the returned enumerable is not cached/replayable; state the concurrent-enumeration behaviour. If single-execution is desired, design for it. + +--- + +### 4. §6 glosses the generic-closing difference between `BuildStream` and `BuildAsync` (Score: 58) + +§6 says `BuildStream` "follows the identical shape as `BuildAsync`." But existing decorator resolution closes the decorator generic with `IQuery` as `TQuery`: `MakeGenericType(typeof(IQuery), typeof(TResult))` (PipelineBuilder.cs:243,:279). Stream decorators are `IStreamQueryHandlerDecorator where TQuery : IStreamQuery` — closing them with `IQuery` violates the constraint. `BuildStream` must close with `typeof(IStreamQuery)`, and the sink/`next` types differ. Not "identical shape"; the one part of the reuse that actually differs is under-specified. + +**Evidence**: `src/Paramore.Darker/PipelineBuilder.cs:243`,`:279`; ADR §6 "It follows the identical shape as `BuildAsync`". + +**Recommendation**: State that `BuildStream` closes stream decorators/handlers over `IStreamQuery` (not `IQuery`); the shared factored steps are resolve/order-by-Step only — close and sink differ. + +--- + +### 5. §3a resilience code omits the `ResilienceContext` branch it claims to honour "exactly" (Score: 50) + +§3a states "`ResilienceContext` is honoured exactly as in the async decorator," but the shown `Execute` body only calls `pipeline.ExecuteAsync(async ct => {...}, cancellationToken)` — it never branches on `Context.ResilienceContext != null`, unlike the real async decorator's four call-shapes (typed/untyped × context/no-context). As written the sample ignores any ambient `ResilienceContext`. + +**Evidence**: `src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineHandlerAsync.cs:82-109` branches on `resilienceContext != null`; ADR §3a passes only `cancellationToken`. + +**Recommendation**: Add the `ResilienceContext` branch to the snippet or soften to "context-threading follows the async decorator's four-way branch." + +--- + +### 6. Fallback described as both "supported" and "not supported" without one crisp statement (Score: 48) + +FR7 says handler-method Fallback is "**Not supported**," §3a says a Polly fallback strategy "may substitute an alternate `(enumerator, moved)`," and Consequences (Positive) lists "retry / timeout / circuit-breaker / **fallback** ... available for streams." Individually reconcilable, but adjacent sections read as contradictory; the load-bearing distinction (handler-method fallback vs Polly fallback strategy) is only implied. + +**Evidence**: ADR FR7 table row vs Consequences (Positive). + +**Recommendation**: Add one up-front sentence: "Handler-method fallback: removed. Polly fallback strategy: supported at establishment only," and use it consistently. + +--- + +## Notes on claims that VERIFIED correctly (not findings) + +- `IQueryHandlerDecoratorAsync` `next`/`fallback` are `Func>` (IQueryHandlerDecoratorAsync.cs:10-13) — §3 mirroring accurate. +- `IQueryHandlerRegistryAsync` `Get`/`Register` exactly as §5 claims. +- Factory shapes: `IQueryHandlerFactoryAsync.Create(Type, IAmALifetime) : IQueryHandler` and generic `IQueryHandlerDecoratorFactoryAsync.Create` — §5 "factories reused unchanged" holds given `IStreamQueryHandler : IQueryHandler` and `IStreamQueryHandlerDecorator : IQueryHandlerDecorator`. +- Resilience pipeline resolution API (`GetPipeline`, `GetPipeline`, `TryGetPipeline`) verified (UseResiliencePipelineHandlerAsync.cs:62-108) — §3a reuse grounded. +- `PipelineBuilder` is `internal sealed`, generic, `IDisposable`, uses reflection `Invoke` + `TargetInvocationException`/`ExceptionDispatchInfo` unwrap; `ValidateNoMismatchedAttributes` is a generalizable guard (PipelineBuilder.cs:15,81-88,184-189) — §6 accurate on these. +- Target frameworks `netstandard2.0;net8.0;net9.0` confirmed (Paramore.Darker.csproj:3); `Microsoft.Bcl.AsyncInterfaces` not yet in Directory.Packages.props, consistent with §7. +- §4 deferred-execution / leak-on-abandon reasoning is **correct** (not a defect): the whole method is one async iterator, so `BuildStream`/`Dispose` never run if never enumerated. +- All five open questions are resolved with rationale (Q1 §1, Q2 §4, Q3 §7/Decision, Q4 §7, Q5 FR7+§3a). + +## Summary + +| Score Range | Count | +|-------------|-------| +| 90-100 (Critical) | 1 | +| 70-89 (High) | 0 | +| 50-69 (Medium) | 4 | +| 0-49 (Low) | 1 | + +**Total findings**: 6 +**Findings at or above threshold (60)**: 3 (Findings 1, 2, 3) + +**Verdict**: NEEDS WORK — driven primarily by Finding 1 (the `IStreamQuery : IQuery` type model contradicts both the `Query` base class and the `CreateQuerySpan` signature the ADR calls in §4), plus the undocumented deferred-error and multiple-enumeration semantics. + +--- + +## Resolution log (round 1 → ADR revised 2026-07-09) + +All six findings addressed in `docs/adr/0019-streaming-query-pipeline.md`: + +1. **[Critical] Type model** — `IStreamQuery` changed to derive from **`IQuery`** (user decision). §1 rewritten: now compiles against `Query` and `CreateQuerySpan`; stream-vs-single dispatch moved to method + registry; documented that a stream query passed to `ExecuteAsync` fails cleanly with `ConfigurationException`. +2. **[Med] Deferred config errors** — §4 now documents that handler-resolution/attribute errors surface on first enumeration (not the call), accepted as the price of leak-on-abandon safety; eager type-only resolution added to Alternatives. +3. **[Med] Multiple/concurrent enumeration** — §4 now states the returned enumerable is cold/not cached; re-enumeration re-executes and re-traces; concurrent enumeration runs independent pipelines. +4. **[Med] Generic-close gloss** — §6 now states `BuildStream` closes over `IStreamQuery` (not `IQuery`) and calls out the sink/delegate difference; only resolve/validate/order are shared. +5. **[Med] `ResilienceContext` branch** — §3a code now shows the context-vs-token branch; prose corrected. +6. **[Low] Fallback wording** — a crisp up-front distinction (handler-method fallback removed vs Polly fallback strategy supported at establishment) added before the FR7 table. + +**Status after revision**: round-2 review run (below). + +--- + +# Review: design (round 2) — 012-streaming_results (ADR 0019) + +**Date**: 2026-07-09 +**Threshold**: 60 +**Verdict**: NEEDS WORK (round 2) → all findings resolved (see resolution log) + +## Round-1 fix verification +- #1 Type model → CONFIRMED FIXED (compiles against `Query` / `CreateQuerySpan`; no variance conflict; `ExecuteStream`/`ExecuteAsync` are distinct names, no overload ambiguity). +- #2 Deferred config errors → CONFIRMED FIXED (`QueryLifetimeScope` real; leak-on-abandon sound). +- #3 Multiple enumeration → CONFIRMED FIXED for the created-context path; exposed a residual race for caller-supplied context → **round-2 Finding 2**. +- #4 Generic close → CONFIRMED FIXED and reflection-valid. +- #5 ResilienceContext branch → FIX INTRODUCED NEW ISSUE → **round-2 Finding 1** (typed-pipeline branch doesn't compile). +- #6 Fallback wording → CONFIRMED FIXED (consistent everywhere). + +## Round-2 findings + +### 1. §3a `useTypePipeline` branch does not compile (Score: 78) +A `ResiliencePipeline` (from `GetPipeline`) can only execute `ValueTask` callbacks; stream establishment executes a `(IAsyncEnumerator, bool)`-returning callback, so the typed pipeline is type-incompatible, and the `?:` between `ResiliencePipeline` and `ResiliencePipeline` does not type-unify. +**Evidence**: ADR §3a; `UseResiliencePipelineHandlerAsync.cs:84-109` keeps typed/untyped in disjoint `ValueTask` branches. +**Recommendation**: Drop the typed branch for streams; untyped `GetPipeline(_policy)` with `ExecuteAsync<(enumerator,bool)>` only. + +### 2. Caller-supplied `queryContext` is shared mutable state across re-enumerations (Score: 64) +§4 claims concurrent enumeration runs "independent pipelines," but a non-null supplied `queryContext` is mutated (`.Span`/`.Tracer`) on one shared instance → race between overlapping enumerations. +**Evidence**: ADR §4; single-valued mutable `IQueryContext.Span`/`.Tracer`. +**Recommendation**: Scope a supplied context to single enumeration; document; concurrent-safe only on the processor-created-context path. + +### 3. `BuildStream` method resolution under-specified — `ExecuteAsync` name collision (Score: 55, below threshold) +`GetMethod("ExecuteAsync")` (no arg types) risks binding to the async overload / `AmbiguousMatchException`. +**Recommendation**: Resolve by signature (`IAsyncEnumerable` return, `(TQuery, CancellationToken)`). + +### 4. NFR4 (AOT) unaddressed (Score: 40, below threshold) +Design is reflection-based but introduces nothing beyond the existing (already `IsAotCompatible`) pipeline; just needed an explicit sentence. + +## Requirements coverage +All FR1–FR9 covered (FR9 caveated for supplied-context concurrency by Finding 2). NFR1–NFR3, NFR5 covered; NFR4 was the only gap (Finding 4). + +## Summary +| Score Range | Count | +|---|---| +| 90-100 | 0 | +| 70-89 | 1 | +| 50-69 | 2 | +| 0-49 | 1 | +**Total**: 4 — **At/above threshold (60)**: 2 (Findings 1, 2) + +## Resolution log (round 2 → ADR revised 2026-07-09) +All four round-2 findings addressed in the ADR: +1. **[High] Typed-pipeline compile error** — §3a rewritten to use the **untyped** `GetPipeline(_policy)` only; added a note explaining a `ResiliencePipeline` cannot wrap the tuple-returning establishment callback; removed `useTypePipeline` from the stream attribute and its caveat. +2. **[Med] Supplied-context race** — §4 now documents that a caller-supplied `queryContext` is single-enumeration-scoped; concurrent/repeat enumeration is safe only on the processor-created-context path. +3. **[Med] Method resolution** — §6 now requires signature-based resolution of the stream `ExecuteAsync` (return `IAsyncEnumerable`, params `(TQuery, CancellationToken)`), avoiding name collision / `AmbiguousMatchException`. +4. **[Low] NFR4** — §7 now maps NFR4: same reflection as `BuildAsync`, existing `IsAotCompatible` posture carries over. + +**Status after round-2 revision**: round-3 review run (below). + +--- + +# Review: design (round 3) — 012-streaming_results (ADR 0019) + +**Date**: 2026-07-09 +**Threshold**: 60 +**Verdict**: NEEDS WORK (round 3) → resolved (see resolution log) + +## Round-2 fix verification +- #1 Typed-pipeline removal → **CONFIRMED FIXED** (reviewer probed Polly.Core 8.7.0 by reflection: non-generic `ResiliencePipeline.ExecuteAsync` accepts the tuple as `TResult` for both the token and `ResilienceContext` overloads; ternary type-unifies). +- #2 Supplied-context single-enumeration → **CONFIRMED FIXED** (document-only decision, as accepted). +- #3 Signature-based resolution → **CONFIRMED FIXED**. +- #4 NFR4/AOT → **CONFIRMED FIXED** (`IsAotCompatible` set; existing `MakeGenericType`/`Invoke` confirmed). +- Also verified clean: `out TResult` variance, empty-stream disposal, no successful-establish leak window, type model. + +## Round-3 findings + +### 1. §5 "DI scan reads the same as async handlers" hid net-new plumbing (Score: 64) +The existing scan hard-matches `typeof(IQueryHandlerAsync<,>)` (`QueryHandlerRegistryAsync.cs:56`) and cannot pick up `IStreamQueryHandler<,>`; and §4's `new PipelineBuilder(/* … */)` hand-waved a stream-registry ctor slot that `PipelineBuilder`'s ctor (`:35-49`) does not have. `QueryProcessor` (`:44-50`) must also read + thread a new `StreamHandlerRegistry`. +**Recommendation**: State the net-new wiring explicitly (new scan predicate, new registry type, new `PipelineBuilder` ctor slot, `QueryProcessor` threading, DI builder invoking the stream scan); "reads the same" applies to the *user*, not the internals. + +### 2. §3a cancellation treated as retryable establishment failure (Score: 40, below threshold) +`Establish`'s undifferentiated `catch … throw` rethrows `OperationCanceledException` into the pipeline; a retry strategy without a cancellation-excluding `ShouldHandle` would retry a cancelled first pull. +**Recommendation**: Document the caveat. + +### 3. §3a fallback/disposal interaction not walked through (Score: 35, below threshold) +Fallback-substitutes-alternate-stream is mechanically sound in Polly v8; the ADR didn't state that the primary enumerator is disposed by `Establish`'s catch before fallback fires (so no leak). +**Recommendation**: One clarifying sentence. + +## Requirements coverage +All FR/NFR covered; FR4 was the item stressed by Finding 1 (asserted but under-specified) — now grounded. + +## Summary +| Score Range | Count | +|---|---| +| 90-100 | 0 | +| 70-89 | 0 | +| 50-69 | 1 | +| 0-49 | 2 | +**Total**: 3 — **At/above threshold (60)**: 1 (Finding 1) + +## Resolution log (round 3 → ADR revised 2026-07-09) +1. **[Med] §5 DI/registry plumbing** — §5 rewritten: registration mirrors async *for the user* but is net-new internally; lists the 5 explicit wiring changes (new stream `RegisterFromAssemblies` matching `typeof(IStreamQueryHandler<,>)`, `StreamHandlerRegistry` on `IHandlerConfiguration`, new `PipelineBuilder` ctor slot, `QueryProcessor` threading, DI builder invoking the stream scan). §4 ctor comment corrected to pass `_streamHandlerRegistry`. +2. **[Low] Cancellation caveat** — added to §3a caveats. +3. **[Low] Fallback disposal** — added to §3a caveats. + +**Status after round-3 revision**: no known findings ≥ threshold remain. Three adversarial rounds complete (6 + 4 + 3 findings, all resolved). Ready for `/spec:approve design 0019`. From 0d7ee815c1e683df98dfbb60db324c03c360ebef Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 19:52:48 +0100 Subject: [PATCH 03/37] chore: Ignore .DS_Store and .junie/ Co-Authored-By: Claude Opus 4.8 (1M context) --- .gitignore | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.gitignore b/.gitignore index ef7b1721..567f195e 100644 --- a/.gitignore +++ b/.gitignore @@ -264,3 +264,9 @@ paket-files/ # Benchmark.Net BenchmarkDotNet.Artifacts/ + +# macOS +.DS_Store + +# JetBrains Junie +.junie/ From ab306f84ed144c406a14a80ab31f1ec8ceeb9c20 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 20:03:00 +0100 Subject: [PATCH 04/37] docs: Add task breakdown for streaming query results (spec 012) Add tasks.md with 31 TDD-first tasks across 9 phases, sequenced structural-before-behavioural per ADR 0019, with a dependency graph and risk-mitigation traceability table. Update spec README checklist. Co-Authored-By: Claude Opus 4.8 (1M context) --- specs/012-streaming_results/README.md | 4 +- specs/012-streaming_results/tasks.md | 477 ++++++++++++++++++++++++++ 2 files changed, 479 insertions(+), 2 deletions(-) create mode 100644 specs/012-streaming_results/tasks.md diff --git a/specs/012-streaming_results/README.md b/specs/012-streaming_results/README.md index 7a3ebe7d..04a498e6 100644 --- a/specs/012-streaming_results/README.md +++ b/specs/012-streaming_results/README.md @@ -17,7 +17,7 @@ result set. Requirements, design, and scope to be established in the workflow be - [x] Requirements (`/spec:requirements`) — ✅ approved - [x] Design / ADR (`/spec:design`) — ✅ ADR 0019 `Accepted` - [x] Adversarial Review — 3 rounds (6 + 4 + 3 findings) all resolved in ADR 0019 (see `review-design.md`) -- [ ] Task Breakdown (`/spec:tasks` or `/spec:ralph-tasks`) +- [x] Task Breakdown (`/spec:tasks`) — ✅ `tasks.md` created (31 tasks across 9 phases) - [ ] Implementation (`/spec:implement` or `/spec:ralph-implement`) ## Documents @@ -26,4 +26,4 @@ result set. Requirements, design, and scope to be established in the workflow be |-------|------|--------| | Requirements | `requirements.md` | ⬜ Not started | | Design | `docs/adr/0019-streaming-query-pipeline.md` | ✅ Accepted | -| Tasks | `tasks.md` | ⬜ Not started | +| Tasks | `tasks.md` | ✅ Created | diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md new file mode 100644 index 00000000..6f4de1c5 --- /dev/null +++ b/specs/012-streaming_results/tasks.md @@ -0,0 +1,477 @@ +# Tasks — 012 Streaming Results + +**Spec**: `specs/012-streaming_results/` +**Design**: `docs/adr/0019-streaming-query-pipeline.md` (Accepted) +**Issue**: [#299](https://github.com/BrighterCommand/Darker/issues/299) + +> **TDD is mandatory.** Every task tagged **TEST + IMPLEMENT** MUST be started with the exact +> `/test-first` command shown. Write the test, then **STOP and wait for the user to approve the test +> in their IDE** before writing any implementation. Do **not** hand-write a test and continue. +> +> Tasks tagged **STRUCTURAL** are Tidy-First refactors/scaffolding with *no behavioural change* +> (new marker interfaces, ctor slots, package refs). They carry no test of their own; use +> `/tidy-first` and keep them in separate commits from behavioural work. They must still compile. + +## Conventions + +- **Core src**: `src/Paramore.Darker/` +- **Core tests**: `test/Paramore.Darker.Core.Tests/` (flat `When_*.cs` files; test doubles in + `TestDoubles/`, namespace `Paramore.Darker.Core.Tests.TestDoubles`) +- **DI src / tests**: `src/Paramore.Darker.Extensions.DependencyInjection/` / + `test/Paramore.Darker.Extensions.Tests/` +- Prefer real/Simple/InMemory doubles (`StreamQueryHandlerRegistry`, `SimpleHandlerFactory`, + `InMemoryDecoratorRegistry`, `InMemoryQueryContextFactory`); Moq only as a last resort. + +--- + +## Phase 0 — Structural foundations (Tidy First, no behaviour) + +- [ ] **STRUCTURAL: T001 — `IAsyncEnumerable` available on all targets** + - Add `Microsoft.Bcl.AsyncInterfaces` version to `Directory.Packages.props` (CPM). + - Add a **conditional** `` in `src/Paramore.Darker/Paramore.Darker.csproj` + guarded to `netstandard2.0` only (native on net8.0/net9.0). + - Verify `dotnet build Darker.Filter.slnf -c Release` still succeeds on all targets. + - ADR §7. No behaviour; separate commit. + +- [ ] **STRUCTURAL: T002 — Stream query + handler contracts** + - Add `src/Paramore.Darker/IStreamQuery.cs`: + `public interface IStreamQuery : IQuery { }` (TResult = **item** type). + - Add `src/Paramore.Darker/IStreamQueryHandler.cs`: + `IStreamQueryHandler : IQueryHandler where TQuery : IStreamQuery` + with `IAsyncEnumerable ExecuteAsync(TQuery query, CancellationToken cancellationToken = default)`. + - Add `src/Paramore.Darker/StreamQueryHandler.cs`: abstract base with `IQueryContext Context { get; set; }` + and abstract `ExecuteAsync` (no `Fallback` method — ADR §2). XML docs incl. licence header. + - ADR §2. Must compile; no behaviour. + +- [ ] **STRUCTURAL: T003 — Stream decorator contract + attribute base** + - Add `src/Paramore.Darker/IStreamQueryHandlerDecorator.cs`: + `IStreamQueryHandlerDecorator : IQueryHandlerDecorator where TQuery : IStreamQuery` + with `IAsyncEnumerable Execute(TQuery query, Func> next, CancellationToken cancellationToken)`. + - Add `src/Paramore.Darker/StreamQueryHandlerAttribute.cs`: abstract `Attribute` + (`AttributeTargets.Method`) mirroring `QueryHandlerAttributeAsync` — `int Step`, + `abstract object[] GetAttributeParams()`, `abstract Type GetDecoratorType()`. + - ADR §3. Must compile; no behaviour. + +--- + +## Phase 1 — Handler registry (behaviour) + +- [ ] **TEST + IMPLEMENT: T004 — Stream registry resolves a stream handler by query type** + - **USE COMMAND**: `/test-first when stream query registered should resolve its stream handler type from the stream registry` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_query_registered_should_resolve_stream_handler_type.cs` + - Test should verify: + - `StreamQueryHandlerRegistry.Register()` then `Get(typeof(TQuery))` + returns the handler type + - `Get` for an unregistered query type returns `null` + - Registering a duplicate query type throws `ConfigurationException` (mirror async registry) + - Registering with a result type that does not match the query throws `ConfigurationException` + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Add `src/Paramore.Darker/IStreamQueryHandlerRegistry.cs` (ADR §5 signature: `Get`, + generic `Register` constrained to `IStreamQueryHandler`, + `Register(Type, Type, Type)`) + - Add `src/Paramore.Darker/StreamQueryHandlerRegistry.cs` modelled on + `QueryHandlerRegistryAsync` (dictionary, duplicate + result-type guards) + - Requires a stream query + handler test double in `TestDoubles/` + +- [ ] **TEST + IMPLEMENT: T005 — Assembly scan registers only stream handlers** + - **USE COMMAND**: `/test-first when scanning assemblies for stream handlers should register IStreamQueryHandler implementations and ignore async handlers` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_scanning_assemblies_for_stream_handlers_should_register_only_stream_handlers.cs` + - Test should verify: + - `RegisterFromAssemblies` on the stream registry picks up a public `IStreamQueryHandler<,>` + implementation and maps query → handler + - It does **not** register `IQueryHandlerAsync<,>` / `IQueryHandler<,>` implementations + - Only public (`ExportedTypes`) concrete non-abstract classes are registered (ADR 0011 §9-10) + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Add `RegisterFromAssemblies(IEnumerable)` to `StreamQueryHandlerRegistry` + hard-matching `i.GetGenericTypeDefinition() == typeof(IStreamQueryHandler<,>)` (ADR §5.1) + +--- + +## Phase 2 — Pipeline build + processor entry point (MVP happy path) + +- [ ] **STRUCTURAL: T006 — Thread the stream registry through builder + configuration** + - `PipelineBuilder` gains a new ctor parameter for `IStreamQueryHandlerRegistry` + (ADR §5.3) — additive, existing ctors unaffected. + - `IHandlerConfiguration` / `HandlerConfiguration` gain an optional + `IStreamQueryHandlerRegistry StreamHandlerRegistry` member (null when streaming unused, ADR §5.2). + - `QueryProcessor` reads `StreamHandlerRegistry` off the configuration into a field (ADR §5.4). + - Must compile; no behaviour yet. + +- [ ] **TEST + IMPLEMENT: T007 — `BuildStream` resolves the handler and yields its items (no decorators)** + - **USE COMMAND**: `/test-first when building a stream pipeline with no decorators should invoke the handler and yield its items` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_building_stream_pipeline_without_decorators_should_yield_handler_items.cs` + - Test should verify: + - `PipelineBuilder.BuildStream(query, context, options)` returns a + `Func, CancellationToken, IAsyncEnumerable>` + - `await foreach` over the returned delegate yields exactly the items the handler produces, in order + - The stream method is resolved **by signature** (`IAsyncEnumerable` return, + `(TQuery, CancellationToken)` params), not bare name — a handler with both an async + `Task ExecuteAsync` and a stream `ExecuteAsync` binds to the stream one with no + `AmbiguousMatchException` + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Add `BuildStream` to `PipelineBuilder.cs` (ADR §6): factor the shared resolve/validate/order + steps with `BuildAsync`; close handler + decorators over **`typeof(IStreamQuery)`** + (not `IQuery`) to satisfy the `where TQuery : IStreamQuery` constraint + - Resolve the handler type from the stream registry; add signature-based stream-method resolution + - Sink invokes the handler's stream `ExecuteAsync` returning the enumerable — **no** + `TargetInvocationException` unwrap around enumeration (iterator defers, ADR §4/§6) + +- [ ] **TEST + IMPLEMENT: T008 — `ExecuteStream` runs a stream query end-to-end** + - **USE COMMAND**: `/test-first when executing a stream query through the processor should yield all handler items via await foreach` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_executing_stream_query_should_yield_all_items.cs` + - Test should verify: + - `queryProcessor.ExecuteStream(query, ct)` returns an `IAsyncEnumerable` + - `await foreach` yields all items the handler produces, in order + - Works with a processor built from `HandlerConfiguration` + `InMemoryQueryContextFactory` + and the reused async handler/decorator factories + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Add `IAsyncEnumerable ExecuteStream(IStreamQuery query, IQueryContext? queryContext = null, CancellationToken cancellationToken = default)` + to `IQueryProcessor` (ADR §4) — **breaking interface change, permitted under V5/NFR2** + - Implement it on `QueryProcessor` as an `async` iterator with `[EnumeratorCancellation]`, + binding span **and** `PipelineBuilder.Dispose()` to enumeration lifetime via `try/finally` + (ADR §4 code sketch) + - Update **`FakeQueryProcessor`** (`src/Paramore.Darker.Testing/FakeQueryProcessor.cs`) and any + other `IQueryProcessor` implementers to satisfy the new interface member + +--- + +## Phase 3 — Core streaming correctness properties (each a distinct behaviour) + +- [ ] **TEST + IMPLEMENT: T009 — Items are produced lazily, not buffered** + - **USE COMMAND**: `/test-first when consuming a stream query should observe the first item before the handler produces the last item` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_consuming_stream_query_should_produce_items_lazily.cs` + - Test should verify (NFR1/FR6, ADR risk "accidental buffering"): + - A handler that records each item's production (e.g. increments a counter / signals per `yield`) + has produced **fewer** than all items at the moment the consumer observes the first item + - The framework does not materialise the whole sequence before the first `MoveNextAsync` + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Confirm the `ExecuteStream` + `BuildStream` chain are `async` iterators end-to-end (no + `ToListAsync`/eager await); fix any eager materialisation surfaced by the test + +- [ ] **TEST + IMPLEMENT: T010 — Cancellation stops enumeration promptly** + - **USE COMMAND**: `/test-first when the cancellation token is cancelled mid-stream should stop enumeration and propagate OperationCanceledException` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_cancelled_mid_enumeration_should_stop_and_throw_OperationCanceledException.cs` + - Test should verify (FR5): + - Cancelling the token during `await foreach` stops further item production promptly + - `OperationCanceledException` (or `TaskCanceledException`) propagates to the caller + - `await foreach (... .WithCancellation(ct))` flows the token into the handler's + `[EnumeratorCancellation]` parameter + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Ensure `ExecuteStream` applies `[EnumeratorCancellation]` and passes `.WithCancellation(ct)` + to the inner enumeration (ADR §4); no special-casing beyond token flow + +- [ ] **TEST + IMPLEMENT: T011 — Exceptions mid-stream surface unwrapped** + - **USE COMMAND**: `/test-first when the handler throws during enumeration should surface the original exception to the caller with its stack trace` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_handler_throws_during_enumeration_should_surface_original_exception.cs` + - Test should verify: + - A handler that yields some items then throws surfaces that exact exception type/message from + `await foreach`, **not** wrapped in `TargetInvocationException` + - Items yielded before the fault were still observed + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Confirm no `TargetInvocationException` unwrap is applied around enumeration (the iterator + defers, so `Invoke` returns the enumerable without running the body — ADR §4) + +- [ ] **TEST + IMPLEMENT: T012 — Early `break` releases handler, decorators, and span** + - **USE COMMAND**: `/test-first when the caller breaks out of the stream early should release the handler decorators and end the span exactly once` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_consumer_breaks_early_should_release_pipeline_and_end_span_once.cs` + - Test should verify (A2, ADR risk "span/handler leak"): + - Breaking out of `await foreach` after the first item disposes the enumerator, so the + processor's `finally` releases handler + decorators via the recording factory **exactly once** + - The tracer's span is ended exactly once on early termination + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Rely on the `try/finally` in the `ExecuteStream` iterator running on enumerator `DisposeAsync` + (ADR §4); use `RecordingHandlerFactory` / `RecordingDecoratorFactory` doubles to assert release + +- [ ] **TEST + IMPLEMENT: T013 — Stream query sent to `ExecuteAsync` fails cleanly** + - **USE COMMAND**: `/test-first when a stream query is passed to ExecuteAsync should throw ConfigurationException for no async handler` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_query_passed_to_ExecuteAsync_should_throw_ConfigurationException.cs` + - Test should verify (ADR §1 consequence): + - A query implementing `IStreamQuery` compiles as an `IQuery` argument to + `ExecuteAsync`, but fails at handler lookup with a clear `ConfigurationException` ("no async + handler registered"), because stream handlers live only in the stream registry + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - No new code expected beyond existing async-registry miss behaviour; test documents/locks the + cross-path guarantee + +- [ ] **TEST + IMPLEMENT: T014 — Missing/mismatched stream handler surfaces on first `MoveNextAsync`** + - **USE COMMAND**: `/test-first when no stream handler is registered should throw ConfigurationException on the first enumeration step` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_no_stream_handler_registered_should_throw_ConfigurationException_on_first_move_next.cs` + - Test should verify (ADR §4 "deferred configuration errors"): + - Calling `ExecuteStream` for an unregistered stream query does **not** throw at call time + - The `ConfigurationException` surfaces from the caller's **first** `await foreach` iteration + (deliberate behavioural difference vs eager `Execute`/`ExecuteAsync`) + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Confirm handler resolution/validation runs inside the iterator (first `MoveNextAsync`), per the + single-iterator shape (ADR §4) + +--- + +## Phase 4 — Stream decorator pipeline (behaviour) + +- [ ] **TEST + IMPLEMENT: T015 — Stream decorators run ordered by `Step` descending** + - **USE COMMAND**: `/test-first when a stream handler has multiple stream decorators should execute them ordered by step descending around the handler` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_handler_has_multiple_decorators_should_order_by_step_descending.cs` + - Test should verify: + - Two+ `StreamQueryHandlerAttribute`-derived decorators wrap the handler innermost→outermost by + `Step` (higher `Step` executes first), asserted via a recording/step-event decorator double + - Each decorator can observe/transform items as they stream through `next` + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - `BuildStream` orders `StreamQueryHandlerAttribute`s by `Step` descending and chains + `IStreamQueryHandlerDecorator` instances via `next` (ADR §6), reusing the async decorator factory + - Add step-event stream decorator + attribute test doubles in `TestDoubles/` + +- [ ] **TEST + IMPLEMENT: T016 — Mismatched decorator attributes are rejected** + - **USE COMMAND**: `/test-first when a stream handler has a sync or async decorator attribute should throw ConfigurationException and vice versa` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_handler_has_mismatched_decorator_attribute_should_throw_ConfigurationException.cs` + - Test should verify (ADR §3/§6, risk "legacy attributes on a stream"): + - `QueryHandlerAttribute` / `QueryHandlerAttributeAsync` (e.g. `RetryableQuery`, `FallbackPolicy`) + on a stream handler's `ExecuteAsync` throws `ConfigurationException` + - A `StreamQueryHandlerAttribute` on a sync/async handler throws `ConfigurationException` + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Reuse `ValidateNoMismatchedAttributes(MemberInfo, Type, string)` driven by attribute base type + in `BuildStream` (and add the reciprocal guard in `Build`/`BuildAsync` for + `StreamQueryHandlerAttribute`) — ADR §6 + +- [ ] **TEST + IMPLEMENT: T017 — Re-enumeration re-executes the query** + - **USE COMMAND**: `/test-first when a stream is enumerated twice should re-execute the handler with a fresh pipeline each time` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_enumerated_twice_should_re_execute_with_fresh_pipeline.cs` + - Test should verify (ADR §4 "multiple/concurrent enumeration"): + - Two `await foreach` passes over the same returned `IAsyncEnumerable` each start a fresh + iterator → fresh `PipelineBuilder`/handler (recording factory shows two creations) and re-yield + the items (cold, not cached) + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Confirm the processor owns a fresh context per enumeration on the null-context path (ADR §4 + context-ownership caveat); no caching of the enumerable + +--- + +## Phase 5 — Logging / telemetry stream decorators (behaviour) + +- [ ] **TEST + IMPLEMENT: T018 — Stream logging decorator wraps the stream lifecycle** + - **USE COMMAND**: `/test-first when a stream query has the logging decorator should log start yield each item and log completion with item count and duration` + - Test location: `test/Paramore.Darker.Core.Tests` (Logging folder) + - Test file: `When_stream_query_logged_should_wrap_lifecycle_with_item_count.cs` + - Test should verify (FR7 Logging): + - Logs on stream start, yields each item through `next` unchanged, and logs completion with item + count + elapsed duration + - Laziness preserved (decorator does not buffer — items flow through as produced) + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Add a stream logging decorator (`src/Paramore.Darker/Logging/Handlers/`) as an `async` iterator + over `next`, modelled on `QueryLoggingDecoratorAsync`, plus a `StreamQueryHandlerAttribute`-derived + logging attribute and a builder-extension registration + +- [ ] **TEST + IMPLEMENT: T019 — Stream logging/telemetry records enumeration faults** + - **USE COMMAND**: `/test-first when a stream faults during enumeration should record the exception in the logging decorator and rethrow` + - Test location: `test/Paramore.Darker.Core.Tests` (Logging folder) + - Test file: `When_stream_faults_during_enumeration_should_record_exception_in_logging_decorator.cs` + - Test should verify (FR7 Logging): + - When the handler throws mid-stream, the logging decorator records the exception (log/span event) + and the exception still propagates to the caller + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Wrap the decorator's `await foreach`/`yield` in `try/catch` that records then rethrows + +- [ ] **TEST + IMPLEMENT: T020 — `BuildStream` writes a span event per pipeline step** + - **USE COMMAND**: `/test-first when building a stream pipeline with a span should write a query event per pipeline step` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_building_stream_pipeline_with_span_should_write_event_per_step.cs` + - Test should verify (FR7 Telemetry, mirror `When_building_async_pipeline_with_span_should_write_event_per_step`): + - With a span on the context, enumerating the stream writes one `DarkerTracer.WriteQueryEvent` per + decorator + the handler sink (correct `isAsync`/`isSink` tags) + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Emit `WriteQueryEvent` at each step in `BuildStream` consistent with `BuildAsync` (ADR §6) + +--- + +## Phase 6 — Resilience for streams (behaviour — highest risk) + +- [ ] **STRUCTURAL: T021 — Stream resilience attribute** + - Add `UseResiliencePipelineStreamAttribute(int step, string policy) : StreamQueryHandlerAttribute` + (`src/Paramore.Darker/Policies/Attributes/`) returning the stream resilience decorator type; + **no** `useTypePipeline` parameter (ADR §3a — untyped pipeline only). Must compile. + +- [ ] **TEST + IMPLEMENT: T022 — Resilience decorator yields items on the happy path** + - **USE COMMAND**: `/test-first when a stream uses the resilience pipeline decorator and establishment succeeds should yield all items` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_resilience_establishment_succeeds_should_yield_all_items.cs` + - Test should verify (ADR §3a): + - With a no-op/succeeding named pipeline resolved from `IQueryContext.ResiliencePipeline`, the + decorator establishes the stream, pulls the first item inside the pipeline, then yields all items + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Add `UseResiliencePipelineStreamHandler : IStreamQueryHandlerDecorator` + (`src/Paramore.Darker/Policies/Handlers/`) per the ADR §3a code: untyped + `GetPipeline(_policy)`, `Establish` callback returning `(IAsyncEnumerator, bool)`, + `[EnumeratorCancellation]`, yield only after the pipeline succeeds; reuse the async decorator's + `InitializeFromAttributeParams` pipeline-resolution logic + +- [ ] **TEST + IMPLEMENT: T023 — Retry before the first item does not duplicate emission** + - **USE COMMAND**: `/test-first when establishment fails before the first item should retry a fresh stream with no duplicate emission` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_establishment_fails_before_first_item_should_retry_without_duplicates.cs` + - Test should verify (ADR §3a correctness property — key risk): + - A retry pipeline + a handler that throws on the first attempt before yielding, then succeeds, + produces the full item sequence **exactly once** (no re-emission of already-yielded items) + - The handler body was re-run on retry (fresh enumerable) + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Rely on first-`MoveNextAsync`-inside-the-pipeline (ADR §3a); no item leaves the decorator until + the pipeline succeeds + +- [ ] **TEST + IMPLEMENT: T024 — Faults after the first item are not retried** + - **USE COMMAND**: `/test-first when a stream faults after the first item should propagate without retry and without re-emitting items` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_faults_after_first_item_should_not_retry.cs` + - Test should verify (ADR §3a, risk "users assume resilience covers whole stream"): + - A fault raised **after** the first item has been yielded propagates to the caller and is **not** + retried; previously yielded items are not re-emitted + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Confirm the pipeline has exited once the first item is yielded (subsequent `MoveNextAsync` + faults propagate) — ADR §3a + +- [ ] **TEST + IMPLEMENT: T025 — Each failed establishment attempt disposes its enumerator** + - **USE COMMAND**: `/test-first when establishment retries N times should dispose the enumerator from each failed attempt` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_establishment_retries_should_dispose_each_failed_enumerator.cs` + - Test should verify (ADR §3a caveat + risk "enumerator leak on retry"): + - With N failed establishment attempts, the decorator disposes N enumerators (the failed ones), + asserted via an enumerable double that counts `DisposeAsync` + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Ensure the in-pipeline `Establish` callback disposes the enumerator in its `catch` before + rethrowing (the reference article omits this) — ADR §3a + +- [ ] **TEST + IMPLEMENT: T026 — Fallback strategy substitutes an alternate stream at establishment** + - **USE COMMAND**: `/test-first when the resilience pipeline has a fallback strategy should substitute an alternate stream at establishment` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_stream_establishment_fails_with_fallback_strategy_should_yield_alternate_stream.cs` + - Test should verify (ADR §3a Strategy applicability — Fallback): + - A pipeline whose fallback strategy supplies an alternate `(enumerator, moved)` yields the + alternate stream's items when the primary establishment faults + - No double-dispose / primary leak (primary enumerator disposed by `Establish`'s `catch` before + fallback fires) + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Confirm the outer `await using` owns only the (possibly fallback-substituted) enumerator (ADR §3a + "Fallback disposal is clean") + +--- + +## Phase 7 — Dependency Injection & registration wiring (behaviour) + +- [ ] **TEST + IMPLEMENT: T027 — `AddHandlersFromAssemblies` registers stream handlers for DI** + - **USE COMMAND**: `/test-first when AddDarker scans assemblies should register stream handlers so ExecuteStream resolves them from the container` + - Test location: `test/Paramore.Darker.Extensions.Tests` + - Test file: `When_AddHandlersFromAssemblies_scans_assembly_should_register_stream_handlers.cs` + - Test should verify (FR4, ADR §5.5): + - After `services.AddDarker().AddHandlersFromAssemblies(asm)` and `BuildServiceProvider`, resolving + `IQueryProcessor` and calling `ExecuteStream` for a scanned stream query yields the handler's items + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Add a `ServiceCollectionStreamHandlerRegistry` (mirror `ServiceCollectionHandlerRegistryAsync`) + and invoke the stream scan from `AddHandlersFromAssemblies` (ADR §5.1/§5.5) + - Thread the stream registry into the `HandlerConfiguration` built in + `ServiceCollectionExtensions.BuildQueryProcessor` (ADR §5.4) + +- [ ] **TEST + IMPLEMENT: T028 — Explicit `AddStreamHandlers` registration** + - **USE COMMAND**: `/test-first when registering stream handlers explicitly via the builder should resolve and execute the stream query` + - Test location: `test/Paramore.Darker.Extensions.Tests` + - Test file: `When_registering_stream_handlers_explicitly_should_execute_stream_query.cs` + - Test should verify (FR4/NFR5): + - `AddDarker().AddStreamHandlers(r => r.Register())` registers the + handler and `ExecuteStream` yields its items + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Add `AddStreamHandlers(Action)` to `IDarkerHandlerBuilder` / + `ServiceCollectionDarkerHandlerBuilder` (mirror `AddAsyncHandlers`) + +- [ ] **TEST + IMPLEMENT: T029 — Non-DI `QueryProcessorBuilder` wires the stream registry** + - **USE COMMAND**: `/test-first when a processor is built via QueryProcessorBuilder with a stream registry should execute a stream query` + - Test location: `test/Paramore.Darker.Core.Tests` + - Test file: `When_QueryProcessorBuilder_configured_with_stream_registry_should_execute_stream_query.cs` + - Test should verify: + - A `QueryProcessor` built through the fluent `QueryProcessorBuilder` (with a stream registry + + reused async factories) executes a stream query via `ExecuteStream` + - **STOP HERE - WAIT FOR USER APPROVAL in IDE before implementing** + - Implementation should: + - Extend the builder's handler-configuration wiring (`Builder/`) to accept/pass the stream registry + into `HandlerConfiguration` (only if not already covered by T006/T008) + +--- + +## Phase 8 — Cross-target build, AOT, docs (verification) + +- [ ] **VERIFY: T030 — Green on all target frameworks + AOT** + - `dotnet build Darker.Filter.slnf -c Release` and `dotnet test Darker.Filter.slnf -c Release --no-build` + pass on `netstandard2.0;net8.0;net9.0` (NFR3). + - `Paramore.Darker.Tests.AOT` still builds/publishes (NFR4 — no new trim/AOT-hostile reflection + beyond existing `BuildAsync`). + - Not a `/test-first` task — runs the existing suite; fix regressions surfaced. + +- [ ] **DOCS: T031 — Document streaming usage and semantics** + - Update README / user docs with: `ExecuteStream` + `await foreach` usage; DI registration; and the + **documented semantics** — deferred config error on first `MoveNextAsync` (ADR §4); resilience + covers **establishment + first item only**, `Timeout` bounds start-up, `Hedging` unsupported (ADR §3a); + legacy `RetryableQuery`/`FallbackPolicy` do **not** apply to streams (use + `UseResiliencePipelineStream`); re-enumeration re-executes; caller-supplied context is single-enumeration. + - Update `specs/012-streaming_results/README.md` checklist (Tasks ✅ / Implementation). + +--- + +## Dependencies + +``` +T001 (pkg) ─┐ +T002 (contracts) ─┬─> T004,T005 (registry) ─┐ +T003 (decorator contract) ─────────────────┤ + ├─> T006 (wiring) ─> T007 (BuildStream) ─> T008 (ExecuteStream) + │ │ + T008 ─> T009..T014 (correctness: lazy, cancel, throw, lifetime, cross-path, deferred) │ + T007/T008 ─> T015..T017 (decorator chain, mismatch, re-enumeration) │ + T015 ─> T018,T019 (logging) T007 ─> T020 (telemetry events) │ + T003 ─> T021 ─> T022 ─> T023,T024,T025,T026 (resilience) │ + T006/T008 ─> T027,T028,T029 (DI + builder wiring) │ + everything ─> T030 (cross-target/AOT) ─> T031 (docs) +``` + +## Risk-mitigation coverage (traceability to ADR "Risks and Mitigations") + +| ADR risk | Covered by | +|---|---| +| Accidental buffering defeats streaming | **T009** (laziness) | +| Span/handler leak on abandoned enumeration | **T012** (early-break lifetime) | +| Legacy `RetryableQuery`/`FallbackPolicy` on a stream | **T016** (mismatch validation) + **T031** (docs) | +| Enumerator leak on resilience retry | **T025** (N failures ⇒ N disposals) | +| Users assume resilience covers whole stream | **T024** (post-first-item fault not retried) + **T031** | +| Duplicate emission on retry | **T023** (fresh stream, no duplicates) | +| `BuildAsync`/`BuildStream` divergence | keep resolve/validate/order factored (**T007**, **T015**) | +| Deferred config error surprising | **T014** (documents first-`MoveNextAsync` error) + **T031** | From 98334a09a0e2eb775899599bce11340294aeadb2 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 20:56:16 +0100 Subject: [PATCH 05/37] chore: Approve tasks phase for streaming query results (spec 012) Co-Authored-By: Claude Opus 4.8 (1M context) --- specs/012-streaming_results/.tasks-approved | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 specs/012-streaming_results/.tasks-approved diff --git a/specs/012-streaming_results/.tasks-approved b/specs/012-streaming_results/.tasks-approved new file mode 100644 index 00000000..e69de29b From c0e5c9355268942abb607c11a061c5e12ee9ca10 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 20:59:15 +0100 Subject: [PATCH 06/37] feat: Add structural foundations for streaming query pipeline (T001-T003) Add IAsyncEnumerable support via Microsoft.Bcl.AsyncInterfaces (netstandard2.0 only), stream query/handler/decorator contracts, and the StreamQueryHandlerAttribute base. Co-Authored-By: Claude Sonnet 4.6 --- Directory.Packages.props | 1 + src/Paramore.Darker/IStreamQuery.cs | 10 +++++++++ src/Paramore.Darker/IStreamQueryHandler.cs | 14 ++++++++++++ .../IStreamQueryHandlerDecorator.cs | 18 +++++++++++++++ src/Paramore.Darker/Paramore.Darker.csproj | 3 +++ src/Paramore.Darker/StreamQueryHandler.cs | 17 ++++++++++++++ .../StreamQueryHandlerAttribute.cs | 22 +++++++++++++++++++ 7 files changed, 85 insertions(+) create mode 100644 src/Paramore.Darker/IStreamQuery.cs create mode 100644 src/Paramore.Darker/IStreamQueryHandler.cs create mode 100644 src/Paramore.Darker/IStreamQueryHandlerDecorator.cs create mode 100644 src/Paramore.Darker/StreamQueryHandler.cs create mode 100644 src/Paramore.Darker/StreamQueryHandlerAttribute.cs diff --git a/Directory.Packages.props b/Directory.Packages.props index 6eb14c9c..dc7aa937 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -5,6 +5,7 @@ + diff --git a/src/Paramore.Darker/IStreamQuery.cs b/src/Paramore.Darker/IStreamQuery.cs new file mode 100644 index 00000000..2dd02cf1 --- /dev/null +++ b/src/Paramore.Darker/IStreamQuery.cs @@ -0,0 +1,10 @@ +namespace Paramore.Darker +{ + /// + /// Marker interface for queries that yield results incrementally as an async stream. + /// TResult is the item type, not the enumerable. + /// + public interface IStreamQuery : IQuery + { + } +} diff --git a/src/Paramore.Darker/IStreamQueryHandler.cs b/src/Paramore.Darker/IStreamQueryHandler.cs new file mode 100644 index 00000000..97104109 --- /dev/null +++ b/src/Paramore.Darker/IStreamQueryHandler.cs @@ -0,0 +1,14 @@ +using System.Collections.Generic; +using System.Threading; + +namespace Paramore.Darker +{ + /// + /// Handler that processes a stream query and yields results incrementally. + /// + public interface IStreamQueryHandler : IQueryHandler + where TQuery : IStreamQuery + { + IAsyncEnumerable ExecuteAsync(TQuery query, CancellationToken cancellationToken = default); + } +} diff --git a/src/Paramore.Darker/IStreamQueryHandlerDecorator.cs b/src/Paramore.Darker/IStreamQueryHandlerDecorator.cs new file mode 100644 index 00000000..3807c110 --- /dev/null +++ b/src/Paramore.Darker/IStreamQueryHandlerDecorator.cs @@ -0,0 +1,18 @@ +using System; +using System.Collections.Generic; +using System.Threading; + +namespace Paramore.Darker +{ + /// + /// Decorator that wraps a stream query handler, allowing cross-cutting concerns over the async stream. + /// + public interface IStreamQueryHandlerDecorator : IQueryHandlerDecorator + where TQuery : IStreamQuery + { + IAsyncEnumerable Execute( + TQuery query, + Func> next, + CancellationToken cancellationToken); + } +} diff --git a/src/Paramore.Darker/Paramore.Darker.csproj b/src/Paramore.Darker/Paramore.Darker.csproj index 6f8ce597..0759814e 100644 --- a/src/Paramore.Darker/Paramore.Darker.csproj +++ b/src/Paramore.Darker/Paramore.Darker.csproj @@ -6,6 +6,9 @@ Unsupported on netstandard2.0 (UnconditionalSuppressMessageAttribute is internal there), so net8/net9 only. --> true + + + diff --git a/src/Paramore.Darker/StreamQueryHandler.cs b/src/Paramore.Darker/StreamQueryHandler.cs new file mode 100644 index 00000000..1897dfd7 --- /dev/null +++ b/src/Paramore.Darker/StreamQueryHandler.cs @@ -0,0 +1,17 @@ +using System.Collections.Generic; +using System.Threading; + +namespace Paramore.Darker +{ + /// + /// Abstract base class for stream query handlers. + /// No Fallback method — a partially-emitted stream has no meaningful fallback value. + /// + public abstract class StreamQueryHandler : IStreamQueryHandler + where TQuery : IStreamQuery + { + public IQueryContext Context { get; set; } + + public abstract IAsyncEnumerable ExecuteAsync(TQuery query, CancellationToken cancellationToken = default); + } +} diff --git a/src/Paramore.Darker/StreamQueryHandlerAttribute.cs b/src/Paramore.Darker/StreamQueryHandlerAttribute.cs new file mode 100644 index 00000000..bf208b25 --- /dev/null +++ b/src/Paramore.Darker/StreamQueryHandlerAttribute.cs @@ -0,0 +1,22 @@ +using System; + +namespace Paramore.Darker +{ + /// + /// Base attribute for stream query handler decorators. Mirrors QueryHandlerAttributeAsync for the stream pipeline. + /// + [AttributeUsage(AttributeTargets.Method)] + public abstract class StreamQueryHandlerAttribute : Attribute + { + public int Step { get; } + + protected StreamQueryHandlerAttribute(int step) + { + Step = step; + } + + public abstract object[] GetAttributeParams(); + + public abstract Type GetDecoratorType(); + } +} From ece540067fe37dc597ea9c3bda175d27be917b41 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 21:23:54 +0100 Subject: [PATCH 07/37] feat: Add StreamQueryHandlerRegistry resolving stream handlers by query type (T004) Adds IStreamQueryHandlerRegistry and StreamQueryHandlerRegistry with generic/type-based registration, duplicate and result-type guards, and RegisterFromAssemblies scan. Co-Authored-By: Claude Sonnet 4.6 --- .../IStreamQueryHandlerRegistry.cs | 27 ++++++++ .../StreamQueryHandlerRegistry.cs | 68 +++++++++++++++++++ .../TestDoubles/StreamTestQuery.cs | 25 +++++++ ...ered_should_resolve_stream_handler_type.cs | 65 ++++++++++++++++++ 4 files changed, 185 insertions(+) create mode 100644 src/Paramore.Darker/IStreamQueryHandlerRegistry.cs create mode 100644 src/Paramore.Darker/StreamQueryHandlerRegistry.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/StreamTestQuery.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_stream_query_registered_should_resolve_stream_handler_type.cs diff --git a/src/Paramore.Darker/IStreamQueryHandlerRegistry.cs b/src/Paramore.Darker/IStreamQueryHandlerRegistry.cs new file mode 100644 index 00000000..b60cd1a1 --- /dev/null +++ b/src/Paramore.Darker/IStreamQueryHandlerRegistry.cs @@ -0,0 +1,27 @@ +using System; + +namespace Paramore.Darker +{ + /// + /// Registry that maps stream query types to their stream handler types. + /// + public interface IStreamQueryHandlerRegistry + { + /// + /// Returns the handler type registered for the given query type, or null if not registered. + /// + Type Get(Type queryType); + + /// + /// Registers a stream handler for a stream query using generic type parameters. + /// + void Register() + where TQuery : IStreamQuery + where THandler : IStreamQueryHandler; + + /// + /// Registers a stream handler for a stream query using runtime types. + /// + void Register(Type queryType, Type resultType, Type handlerType); + } +} diff --git a/src/Paramore.Darker/StreamQueryHandlerRegistry.cs b/src/Paramore.Darker/StreamQueryHandlerRegistry.cs new file mode 100644 index 00000000..2d2ba893 --- /dev/null +++ b/src/Paramore.Darker/StreamQueryHandlerRegistry.cs @@ -0,0 +1,68 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Reflection; +using Paramore.Darker.Exceptions; + +namespace Paramore.Darker +{ + /// + /// Maps stream query types to their stream handler types. + /// + public class StreamQueryHandlerRegistry : IStreamQueryHandlerRegistry + { + private readonly IDictionary _registry = new Dictionary(); + + /// + public virtual Type Get(Type queryType) => + _registry.TryGetValue(queryType, out var handlerType) ? handlerType : null; + + /// + public virtual void Register() + where TQuery : IStreamQuery + where THandler : IStreamQueryHandler + { + Register(typeof(TQuery), typeof(TResult), typeof(THandler)); + } + + /// + public virtual void Register(Type queryType, Type resultType, Type handlerType) + { + if (_registry.ContainsKey(queryType)) + throw new ConfigurationException($"Registry already contains an entry for {queryType.Name}"); + + if (!HasMatchingResultType(queryType, resultType)) + throw new ConfigurationException($"Result type not valid for query {queryType.Name}"); + + _registry.Add(queryType, handlerType); + } + + /// + /// Scans the given assemblies and registers all public, concrete IStreamQueryHandler implementations. + /// + public void RegisterFromAssemblies(IEnumerable assemblies) + { + // IMPORTANT: ExportedTypes is load-bearing — see ADR 0011 §9-10. + // It ring-fences the scan to public types only so that internal + // TestDoubles in test assemblies are not registered as handlers. + var subscribers = + from t in assemblies.SelectMany(a => a.ExportedTypes) + let ti = t.GetTypeInfo() + where ti.IsClass && !ti.IsAbstract && !ti.IsInterface + from i in ti.ImplementedInterfaces + where i.GetTypeInfo().IsGenericType && i.GetGenericTypeDefinition() == typeof(IStreamQueryHandler<,>) + select new + { + QueryType = i.GenericTypeArguments.ElementAt(0), + ResultType = i.GenericTypeArguments.ElementAt(1), + HandlerType = t + }; + + foreach (var subscriber in subscribers) + Register(subscriber.QueryType, subscriber.ResultType, subscriber.HandlerType); + } + + private static bool HasMatchingResultType(Type queryType, Type resultType) => + queryType.GetInterfaces().Any(i => i.GenericTypeArguments.Any(t => t == resultType)); + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/StreamTestQuery.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamTestQuery.cs new file mode 100644 index 00000000..38944098 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamTestQuery.cs @@ -0,0 +1,25 @@ +using System.Collections.Generic; +using System.Threading; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + internal class StreamTestQuery : IStreamQuery + { + } + + internal class StreamTestQueryHandler : IStreamQueryHandler + { + public IQueryContext Context { get; set; } + +#pragma warning disable CS1998 + public async IAsyncEnumerable ExecuteAsync(StreamTestQuery query, CancellationToken cancellationToken = default) +#pragma warning restore CS1998 + { + yield return "item"; + } + } + + internal class StreamTestQueryOfDifferentResult : IStreamQuery + { + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_stream_query_registered_should_resolve_stream_handler_type.cs b/test/Paramore.Darker.Core.Tests/When_stream_query_registered_should_resolve_stream_handler_type.cs new file mode 100644 index 00000000..c8551489 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_stream_query_registered_should_resolve_stream_handler_type.cs @@ -0,0 +1,65 @@ +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Exceptions; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_stream_query_registered_should_resolve_stream_handler_type + { + [Fact] + public void When_registering_a_stream_handler_should_return_handler_type_on_get() + { + // Arrange + var registry = new StreamQueryHandlerRegistry(); + registry.Register(); + + // Act + var handlerType = registry.Get(typeof(StreamTestQuery)); + + // Assert + handlerType.ShouldBe(typeof(StreamTestQueryHandler)); + } + + [Fact] + public void When_querying_unregistered_type_should_return_null() + { + // Arrange + var registry = new StreamQueryHandlerRegistry(); + registry.Register(); + + // Act + var handlerType = registry.Get(typeof(StreamTestQueryOfDifferentResult)); + + // Assert + handlerType.ShouldBeNull(); + } + + [Fact] + public void When_registering_duplicate_query_type_should_throw_ConfigurationException() + { + // Arrange + var registry = new StreamQueryHandlerRegistry(); + registry.Register(); + + // Act / Assert + var exception = Assert.Throws( + () => registry.Register()); + + exception.Message.ShouldBe($"Registry already contains an entry for {typeof(StreamTestQuery).Name}"); + } + + [Fact] + public void When_registering_with_mismatched_result_type_should_throw_ConfigurationException() + { + // Arrange + var registry = new StreamQueryHandlerRegistry(); + + // Act / Assert — StreamTestQuery yields string, but we claim int as the result type + var exception = Assert.Throws( + () => registry.Register(typeof(StreamTestQuery), typeof(int), typeof(StreamTestQueryHandler))); + + exception.Message.ShouldBe($"Result type not valid for query {typeof(StreamTestQuery).Name}"); + } + } +} From 642774dbc057b686b02e6c00cfbe70b8b8917b15 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 21:25:00 +0100 Subject: [PATCH 08/37] test: Verify RegisterFromAssemblies scans only IStreamQueryHandler implementations (T005) Assembly scan was already implemented in T004; these tests lock in the behavior: stream handlers are discovered, async/sync handlers are ignored. Co-Authored-By: Claude Sonnet 4.6 --- .../Exported/ExportedStreamQuery.cs | 6 +++ .../Exported/ExportedStreamQueryHandler.cs | 17 +++++++ ...rs_should_register_only_stream_handlers.cs | 49 +++++++++++++++++++ 3 files changed, 72 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/Exported/ExportedStreamQuery.cs create mode 100644 test/Paramore.Darker.Core.Tests/Exported/ExportedStreamQueryHandler.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_scanning_assemblies_for_stream_handlers_should_register_only_stream_handlers.cs diff --git a/test/Paramore.Darker.Core.Tests/Exported/ExportedStreamQuery.cs b/test/Paramore.Darker.Core.Tests/Exported/ExportedStreamQuery.cs new file mode 100644 index 00000000..495ba969 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Exported/ExportedStreamQuery.cs @@ -0,0 +1,6 @@ +namespace Paramore.Darker.Core.Tests.Exported +{ + public class ExportedStreamQuery : IStreamQuery + { + } +} diff --git a/test/Paramore.Darker.Core.Tests/Exported/ExportedStreamQueryHandler.cs b/test/Paramore.Darker.Core.Tests/Exported/ExportedStreamQueryHandler.cs new file mode 100644 index 00000000..a0ce95e6 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Exported/ExportedStreamQueryHandler.cs @@ -0,0 +1,17 @@ +using System.Collections.Generic; +using System.Threading; + +namespace Paramore.Darker.Core.Tests.Exported +{ + public class ExportedStreamQueryHandler : IStreamQueryHandler + { + public IQueryContext Context { get; set; } + +#pragma warning disable CS1998 + public async IAsyncEnumerable ExecuteAsync(ExportedStreamQuery query, CancellationToken cancellationToken = default) +#pragma warning restore CS1998 + { + yield return "exported-item"; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_scanning_assemblies_for_stream_handlers_should_register_only_stream_handlers.cs b/test/Paramore.Darker.Core.Tests/When_scanning_assemblies_for_stream_handlers_should_register_only_stream_handlers.cs new file mode 100644 index 00000000..1cdfe5f8 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_scanning_assemblies_for_stream_handlers_should_register_only_stream_handlers.cs @@ -0,0 +1,49 @@ +using Paramore.Darker.Core.Tests.Exported; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_scanning_assemblies_for_stream_handlers_should_register_only_stream_handlers + { + [Fact] + public void When_scanning_assembly_should_register_public_stream_handler_implementations() + { + // Arrange + var registry = new StreamQueryHandlerRegistry(); + + // Act — scan this test assembly which contains ExportedStreamQueryHandler + registry.RegisterFromAssemblies(new[] { typeof(ExportedStreamQueryHandler).Assembly }); + + // Assert — the exported stream handler is found + registry.Get(typeof(ExportedStreamQuery)).ShouldBe(typeof(ExportedStreamQueryHandler)); + } + + [Fact] + public void When_scanning_assembly_should_not_register_async_query_handlers() + { + // Arrange + var registry = new StreamQueryHandlerRegistry(); + + // Act + registry.RegisterFromAssemblies(new[] { typeof(TestQueryHandlerAsync).Assembly }); + + // Assert — IQueryHandlerAsync implementations are NOT in the stream registry + registry.Get(typeof(TestQueryA)).ShouldBeNull(); + } + + [Fact] + public void When_scanning_assembly_should_not_register_sync_query_handlers() + { + // Arrange + var registry = new StreamQueryHandlerRegistry(); + + // Act + registry.RegisterFromAssemblies(new[] { typeof(TestQueryHandler).Assembly }); + + // Assert — IQueryHandler implementations are NOT in the stream registry + // TestQueryHandler handles TestQueryA; it should not appear in the stream registry + registry.Get(typeof(TestQueryA)).ShouldBeNull(); + } + } +} From 0ae74f659b83e4f68f3e5cb335f3ce2b9bca17be Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 21:26:44 +0100 Subject: [PATCH 09/37] feat: Thread IStreamQueryHandlerRegistry through configuration, builder, and processor (T006) Additive structural wiring: HandlerConfiguration gains optional StreamHandlerRegistry member, PipelineBuilder gains a new ctor slot, QueryProcessor reads it from configuration. No behaviour change. Co-Authored-By: Claude Sonnet 4.6 --- src/Paramore.Darker/HandlerConfiguration.cs | 19 +++++++++++++++++++ src/Paramore.Darker/IHandlerConfiguration.cs | 3 +++ src/Paramore.Darker/PipelineBuilder.cs | 6 +++++- src/Paramore.Darker/QueryProcessor.cs | 4 ++++ 4 files changed, 31 insertions(+), 1 deletion(-) diff --git a/src/Paramore.Darker/HandlerConfiguration.cs b/src/Paramore.Darker/HandlerConfiguration.cs index 83f9dedf..795bc3bb 100644 --- a/src/Paramore.Darker/HandlerConfiguration.cs +++ b/src/Paramore.Darker/HandlerConfiguration.cs @@ -14,6 +14,9 @@ public sealed class HandlerConfiguration : IHandlerConfiguration public IQueryHandlerDecoratorRegistryAsync DecoratorRegistryAsync { get; } public IQueryHandlerDecoratorFactoryAsync DecoratorFactoryAsync { get; } + /// + public IStreamQueryHandlerRegistry StreamHandlerRegistry { get; } + public HandlerConfiguration( IQueryHandlerRegistry handlerRegistry, IQueryHandlerFactory handlerFactory, @@ -44,5 +47,21 @@ public HandlerConfiguration( DecoratorRegistryAsync = decoratorRegistryAsync; DecoratorFactoryAsync = decoratorFactoryAsync; } + + public HandlerConfiguration( + IQueryHandlerRegistry handlerRegistry, + IQueryHandlerFactory handlerFactory, + IQueryHandlerDecoratorRegistry decoratorRegistry, + IQueryHandlerDecoratorFactory decoratorFactory, + IQueryHandlerRegistryAsync handlerRegistryAsync, + IQueryHandlerFactoryAsync handlerFactoryAsync, + IQueryHandlerDecoratorRegistryAsync decoratorRegistryAsync, + IQueryHandlerDecoratorFactoryAsync decoratorFactoryAsync, + IStreamQueryHandlerRegistry streamHandlerRegistry) + : this(handlerRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + handlerRegistryAsync, handlerFactoryAsync, decoratorRegistryAsync, decoratorFactoryAsync) + { + StreamHandlerRegistry = streamHandlerRegistry; + } } } diff --git a/src/Paramore.Darker/IHandlerConfiguration.cs b/src/Paramore.Darker/IHandlerConfiguration.cs index 6fd791b3..7a6158ac 100644 --- a/src/Paramore.Darker/IHandlerConfiguration.cs +++ b/src/Paramore.Darker/IHandlerConfiguration.cs @@ -11,5 +11,8 @@ public interface IHandlerConfiguration IQueryHandlerFactoryAsync HandlerFactoryAsync { get; } IQueryHandlerDecoratorRegistryAsync DecoratorRegistryAsync { get; } IQueryHandlerDecoratorFactoryAsync DecoratorFactoryAsync { get; } + + /// Null when streaming is not used. + IStreamQueryHandlerRegistry StreamHandlerRegistry { get; } } } \ No newline at end of file diff --git a/src/Paramore.Darker/PipelineBuilder.cs b/src/Paramore.Darker/PipelineBuilder.cs index f7d3ec57..c3139641 100644 --- a/src/Paramore.Darker/PipelineBuilder.cs +++ b/src/Paramore.Darker/PipelineBuilder.cs @@ -27,6 +27,8 @@ internal sealed class PipelineBuilder : IDisposable private readonly IQueryHandlerFactoryAsync _handlerFactoryAsync; private readonly IQueryHandlerDecoratorFactoryAsync _decoratorFactoryAsync; + private readonly IStreamQueryHandlerRegistry _streamHandlerRegistry; + private IQueryHandler _handler; private IReadOnlyList, TResult>> _decorators; private IReadOnlyList, TResult>> _asyncDecorators; @@ -38,7 +40,8 @@ public PipelineBuilder( IQueryHandlerDecoratorFactory decoratorFactory, IQueryHandlerRegistryAsync handlerRegistryAsync = null, IQueryHandlerFactoryAsync handlerFactoryAsync = null, - IQueryHandlerDecoratorFactoryAsync decoratorFactoryAsync = null) + IQueryHandlerDecoratorFactoryAsync decoratorFactoryAsync = null, + IStreamQueryHandlerRegistry streamHandlerRegistry = null) { _handlerRegistry = handlerRegistry; _handlerFactory = handlerFactory; @@ -46,6 +49,7 @@ public PipelineBuilder( _handlerRegistryAsync = handlerRegistryAsync; _handlerFactoryAsync = handlerFactoryAsync; _decoratorFactoryAsync = decoratorFactoryAsync; + _streamHandlerRegistry = streamHandlerRegistry; } public Func, TResult> Build(IQuery query, IQueryContext queryContext, diff --git a/src/Paramore.Darker/QueryProcessor.cs b/src/Paramore.Darker/QueryProcessor.cs index f0022b06..df0d4755 100644 --- a/src/Paramore.Darker/QueryProcessor.cs +++ b/src/Paramore.Darker/QueryProcessor.cs @@ -24,6 +24,8 @@ public sealed class QueryProcessor : IQueryProcessor private readonly IQueryHandlerFactoryAsync _handlerFactoryAsync; private readonly IQueryHandlerDecoratorFactoryAsync _decoratorFactoryAsync; + private readonly IStreamQueryHandlerRegistry _streamHandlerRegistry; + private readonly IPolicyRegistry _policyRegistry; private readonly ResiliencePipelineProvider _resiliencePipelineProvider; @@ -49,6 +51,8 @@ public QueryProcessor( _handlerFactoryAsync = handlerConfiguration.HandlerFactoryAsync; _decoratorFactoryAsync = handlerConfiguration.DecoratorFactoryAsync; + _streamHandlerRegistry = handlerConfiguration.StreamHandlerRegistry; + _queryContextFactory = queryContextFactory ?? throw new ArgumentNullException(nameof(queryContextFactory)); _policyRegistry = policyRegistry; _resiliencePipelineProvider = resiliencePipelineProvider; From 6f0d37e14f0463f03530f3c33ae68e1e15ef10a5 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 22:32:08 +0100 Subject: [PATCH 10/37] feat: Implement PipelineBuilder.BuildStream and IQueryProcessor.ExecuteStream (T007) BuildStream resolves the stream handler by signature (not bare name) to avoid AmbiguousMatchException; ExecuteStream is an async iterator binding span and pipeline lifetime to enumeration. FakeQueryProcessor updated for new interface. Co-Authored-By: Claude Sonnet 4.6 --- .../FakeQueryProcessor.cs | 11 +++ src/Paramore.Darker/IQueryProcessor.cs | 7 ++ src/Paramore.Darker/PipelineBuilder.cs | 56 +++++++++++++++ src/Paramore.Darker/QueryProcessor.cs | 33 ++++++++- .../TestDoubles/DualExecuteStreamHandler.cs | 30 ++++++++ ...t_decorators_should_yield_handler_items.cs | 70 +++++++++++++++++++ 6 files changed, 206 insertions(+), 1 deletion(-) create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/DualExecuteStreamHandler.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_building_stream_pipeline_without_decorators_should_yield_handler_items.cs diff --git a/src/Paramore.Darker.Testing/FakeQueryProcessor.cs b/src/Paramore.Darker.Testing/FakeQueryProcessor.cs index 906e4928..25b0778a 100644 --- a/src/Paramore.Darker.Testing/FakeQueryProcessor.cs +++ b/src/Paramore.Darker.Testing/FakeQueryProcessor.cs @@ -1,6 +1,7 @@ using System; using System.Collections.Generic; using System.Linq; +using System.Runtime.CompilerServices; using System.Threading; using System.Threading.Tasks; @@ -54,6 +55,16 @@ public TResponse Execute(IQuery query, IQueryContext query return Task.FromResult(Execute(query, queryContext)); } +#pragma warning disable CS1998 + public async IAsyncEnumerable ExecuteStream( + IStreamQuery query, + IQueryContext queryContext = null, + [EnumeratorCancellation] CancellationToken cancellationToken = default) +#pragma warning restore CS1998 + { + yield break; + } + public void SetupResultFor(Predicate predicate, object result) { var queryType = typeof(TQuery); diff --git a/src/Paramore.Darker/IQueryProcessor.cs b/src/Paramore.Darker/IQueryProcessor.cs index a64b9a88..5a2a556a 100644 --- a/src/Paramore.Darker/IQueryProcessor.cs +++ b/src/Paramore.Darker/IQueryProcessor.cs @@ -1,3 +1,4 @@ +using System.Collections.Generic; using System.Threading; using System.Threading.Tasks; @@ -8,5 +9,11 @@ public interface IQueryProcessor TResult Execute(IQuery query, IQueryContext queryContext = null); Task ExecuteAsync(IQuery query, IQueryContext queryContext = null, CancellationToken cancellationToken = default(CancellationToken)); + + /// + /// Executes a stream query, yielding results lazily as an async sequence. + /// Span and handler lifetime are bound to enumeration — release on enumerator disposal. + /// + IAsyncEnumerable ExecuteStream(IStreamQuery query, IQueryContext queryContext = null, CancellationToken cancellationToken = default); } } \ No newline at end of file diff --git a/src/Paramore.Darker/PipelineBuilder.cs b/src/Paramore.Darker/PipelineBuilder.cs index c3139641..e49472da 100644 --- a/src/Paramore.Darker/PipelineBuilder.cs +++ b/src/Paramore.Darker/PipelineBuilder.cs @@ -9,6 +9,7 @@ using Paramore.Darker.Logging; using Paramore.Darker.Observability; using System.Runtime.ExceptionServices; +using System.Runtime.CompilerServices; namespace Paramore.Darker { @@ -264,6 +265,61 @@ private IReadOnlyList, TResult>> GetDecor return decorators; } + public Func, CancellationToken, IAsyncEnumerable> BuildStream( + IStreamQuery query, IQueryContext queryContext, + InstrumentationOptions instrumentationOptions = InstrumentationOptions.None) + { + _lifetime = new QueryLifetimeScope(); + + var queryType = query.GetType(); + _logger.LogInformation("Building stream pipeline for {QueryType}", queryType.Name); + + var (handlerType, handler) = ResolveStreamHandler(queryType); + _handler = handler; + _handler.Context = queryContext; + + // Resolve by signature to avoid AmbiguousMatchException when the handler + // also exposes a Task ExecuteAsync with different param types. + var executeMethodInfo = handlerType.GetMethod(ExecuteAsyncMethodName, new[] { queryType, typeof(CancellationToken) }); + if (executeMethodInfo == null) + throw new ConfigurationException($"Handler {handlerType.FullName} does not implement a stream ExecuteAsync(TQuery, CancellationToken). Ensure it implements IStreamQueryHandler."); + + ValidateNoMismatchedAttributes(executeMethodInfo, typeof(QueryHandlerAttribute), + "Stream handler has sync attribute(s) on ExecuteAsync. Use StreamQueryHandlerAttribute for stream handlers."); + ValidateNoMismatchedAttributes(executeMethodInfo, typeof(QueryHandlerAttributeAsync), + "Stream handler has async attribute(s) on ExecuteAsync. Use StreamQueryHandlerAttribute for stream handlers."); + + var span = queryContext.Span; + + // No TargetInvocationException unwrap: the iterator body runs on MoveNextAsync, not Invoke. + return (r, ct) => + { + DarkerTracer.WriteQueryEvent(span, handlerType.Name, isAsync: true, instrumentationOptions, isSink: true); + return (IAsyncEnumerable)executeMethodInfo.Invoke(_handler, new object[] { r, ct }); + }; + } + + private (Type handlerType, IQueryHandler handler) ResolveStreamHandler(Type queryType) + { + if (_streamHandlerRegistry == null) + throw new ConfigurationException("No stream handler registry configured. Use a HandlerConfiguration with StreamHandlerRegistry set."); + + var handlerType = _streamHandlerRegistry.Get(queryType); + if (handlerType == null) + throw new ConfigurationException($"No stream handler registered for query: {queryType.FullName}"); + + _logger.LogDebug("Found stream handler type for {QueryType}: {HandlerType}", queryType.Name, handlerType.Name); + + var handler = _handlerFactoryAsync != null + ? _handlerFactoryAsync.Create(handlerType, _lifetime) + : _handlerFactory?.Create(handlerType, _lifetime); + + if (handler == null) + throw new ConfigurationException($"Stream handler could not be created for type: {handlerType.FullName}"); + + return (handlerType, handler); + } + private IReadOnlyList, TResult>> GetDecoratorsAsync(MemberInfo executeMethod, IQueryContext queryContext) { var attributes = executeMethod.GetCustomAttributes(typeof(QueryHandlerAttributeAsync), true) diff --git a/src/Paramore.Darker/QueryProcessor.cs b/src/Paramore.Darker/QueryProcessor.cs index df0d4755..a60b9047 100644 --- a/src/Paramore.Darker/QueryProcessor.cs +++ b/src/Paramore.Darker/QueryProcessor.cs @@ -1,13 +1,15 @@ using System; +using System.Collections.Generic; using System.Diagnostics; using System.Reflection; +using System.Runtime.CompilerServices; +using System.Runtime.ExceptionServices; using System.Threading; using System.Threading.Tasks; using Microsoft.Extensions.Logging; using Paramore.Darker.Logging; using Paramore.Darker.Observability; using Polly.Registry; -using System.Runtime.ExceptionServices; namespace Paramore.Darker { @@ -137,6 +139,35 @@ public TResult Execute(IQuery query, IQueryContext queryContex } } + public async IAsyncEnumerable ExecuteStream( + IStreamQuery query, + IQueryContext queryContext = null, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + var pipelineBuilder = new PipelineBuilder( + _handlerRegistry, _handlerFactory, _decoratorFactory, + _handlerRegistryAsync, _handlerFactoryAsync, _decoratorFactoryAsync, + _streamHandlerRegistry); + try + { + queryContext ??= _queryContextFactory.Create(); + InitQueryContext(queryContext); + + var span = _tracer?.CreateQuerySpan(query, queryContext.Span, queryContext, _instrumentationOptions); + queryContext.Span = span; + queryContext.Tracer = _tracer; + + var entryPoint = pipelineBuilder.BuildStream(query, queryContext, _instrumentationOptions); + try + { + await foreach (var item in entryPoint(query, cancellationToken).WithCancellation(cancellationToken).ConfigureAwait(false)) + yield return item; + } + finally { _tracer?.EndSpan(span); } + } + finally { pipelineBuilder.Dispose(); } + } + private void InitQueryContext(IQueryContext queryContext) { if (queryContext.Policies == null) diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/DualExecuteStreamHandler.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/DualExecuteStreamHandler.cs new file mode 100644 index 00000000..0aa2599d --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/DualExecuteStreamHandler.cs @@ -0,0 +1,30 @@ +using System.Collections.Generic; +using System.Threading; +using System.Threading.Tasks; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A stream handler that ALSO exposes a ExecuteAsync overload + /// with a broader parameter type, causing GetMethod("ExecuteAsync") without type args to throw + /// AmbiguousMatchException. BuildStream must resolve by signature, not bare name. + /// + internal class DualExecuteStreamHandler : IStreamQueryHandler + { + public static readonly string[] Items = { "alpha", "beta", "gamma" }; + + public IQueryContext Context { get; set; } + + // Broad-parameter overload: makes bare GetMethod("ExecuteAsync") ambiguous. + public Task ExecuteAsync(IStreamQuery query, CancellationToken cancellationToken = default) + => Task.FromResult("wrong"); + +#pragma warning disable CS1998 + public async IAsyncEnumerable ExecuteAsync(StreamTestQuery query, CancellationToken cancellationToken = default) +#pragma warning restore CS1998 + { + foreach (var item in Items) + yield return item; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_building_stream_pipeline_without_decorators_should_yield_handler_items.cs b/test/Paramore.Darker.Core.Tests/When_building_stream_pipeline_without_decorators_should_yield_handler_items.cs new file mode 100644 index 00000000..d1996f81 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_building_stream_pipeline_without_decorators_should_yield_handler_items.cs @@ -0,0 +1,70 @@ +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_building_stream_pipeline_without_decorators_should_yield_handler_items + { + private static QueryProcessor BuildProcessor(StreamQueryHandlerRegistry streamRegistry) + { + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => new StreamTestQueryHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + null, null, null, null, + streamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public async Task When_executing_stream_query_should_yield_items_in_order() + { + // Arrange + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + var processor = BuildProcessor(streamRegistry); + + // Act + var results = new List(); + await foreach (var item in processor.ExecuteStream(new StreamTestQuery())) + results.Add(item); + + // Assert + results.ShouldBe(new[] { "item" }); + } + + [Fact] + public async Task When_handler_has_ambiguous_ExecuteAsync_should_resolve_stream_signature_without_AmbiguousMatchException() + { + // Arrange — DualExecuteStreamHandler exposes two ExecuteAsync overloads; + // BuildStream must resolve by return type + params, not bare method name. + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => new DualExecuteStreamHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + null, null, null, null, + streamRegistry); + var processor = new QueryProcessor(config, new InMemoryQueryContextFactory()); + + // Act + var results = new List(); + await foreach (var item in processor.ExecuteStream(new StreamTestQuery())) + results.Add(item); + + // Assert — yielded the stream items, not the Task overload result + results.ShouldBe(DualExecuteStreamHandler.Items); + } + } +} From 58bdbf5443f0f40c118765ef9fe5968152812639 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Thu, 9 Jul 2026 22:33:00 +0100 Subject: [PATCH 11/37] test: Verify ExecuteStream end-to-end wiring with async factories and provided context (T008) ExecuteStream was implemented in T007; these tests lock in the full wiring behavior including cancellation token flow and caller-supplied context preservation. Co-Authored-By: Claude Sonnet 4.6 --- .../TestDoubles/MultiItemStreamQuery.cs | 29 +++++++ ...ing_stream_query_should_yield_all_items.cs | 84 +++++++++++++++++++ 2 files changed, 113 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/MultiItemStreamQuery.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_executing_stream_query_should_yield_all_items.cs diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/MultiItemStreamQuery.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/MultiItemStreamQuery.cs new file mode 100644 index 00000000..f8e316d2 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/MultiItemStreamQuery.cs @@ -0,0 +1,29 @@ +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + internal class MultiItemStreamQuery : IStreamQuery + { + } + + internal class MultiItemStreamHandler : IStreamQueryHandler + { + public static readonly string[] Items = { "first", "second", "third" }; + + public IQueryContext Context { get; set; } + + public async IAsyncEnumerable ExecuteAsync( + MultiItemStreamQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + foreach (var item in Items) + { + cancellationToken.ThrowIfCancellationRequested(); + yield return item; + } + await System.Threading.Tasks.Task.CompletedTask; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_executing_stream_query_should_yield_all_items.cs b/test/Paramore.Darker.Core.Tests/When_executing_stream_query_should_yield_all_items.cs new file mode 100644 index 00000000..ace7fc97 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_executing_stream_query_should_yield_all_items.cs @@ -0,0 +1,84 @@ +using System.Collections.Generic; +using System.Threading; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_executing_stream_query_should_yield_all_items + { + private static QueryProcessor BuildProcessorWithAsyncFactories(StreamQueryHandlerRegistry streamRegistry) + { + // Wire up with the reused async factories (as per ADR §5 — stream handlers reuse async factory). + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => new MultiItemStreamHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public async Task When_executing_stream_query_should_yield_all_items_in_order() + { + // Arrange + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + var processor = BuildProcessorWithAsyncFactories(streamRegistry); + var query = new MultiItemStreamQuery(); + + // Act + var results = new List(); + await foreach (var item in processor.ExecuteStream(query)) + results.Add(item); + + // Assert + results.ShouldBe(MultiItemStreamHandler.Items); + } + + [Fact] + public async Task When_executing_stream_query_with_cancellation_token_should_yield_items() + { + // Arrange + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + var processor = BuildProcessorWithAsyncFactories(streamRegistry); + using var cts = new CancellationTokenSource(); + + // Act + var results = new List(); + await foreach (var item in processor.ExecuteStream(new MultiItemStreamQuery(), cancellationToken: cts.Token)) + results.Add(item); + + // Assert + results.ShouldBe(MultiItemStreamHandler.Items); + } + + [Fact] + public async Task When_executing_stream_query_with_provided_context_should_use_that_context() + { + // Arrange + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + var processor = BuildProcessorWithAsyncFactories(streamRegistry); + var queryContext = new InMemoryQueryContextFactory().Create(); + queryContext.Bag["test-key"] = "test-value"; + + // Act + var results = new List(); + await foreach (var item in processor.ExecuteStream(new MultiItemStreamQuery(), queryContext: queryContext)) + results.Add(item); + + // Assert — items yielded and the provided context was used (not replaced) + results.ShouldBe(MultiItemStreamHandler.Items); + queryContext.Bag["test-key"].ShouldBe("test-value"); + } + } +} From b7f55c0cf9aca8da8c6013813f6d7f24ebccae7e Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 13:46:28 +0100 Subject: [PATCH 12/37] test: Verify ExecuteStream items are produced lazily with no eager buffering (T009) Co-Authored-By: Claude Sonnet 4.6 --- .../TestDoubles/LazyTrackingStreamHandler.cs | 50 +++++++++++++ ...tream_query_should_produce_items_lazily.cs | 74 +++++++++++++++++++ 2 files changed, 124 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/LazyTrackingStreamHandler.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_consuming_stream_query_should_produce_items_lazily.cs diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/LazyTrackingStreamHandler.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/LazyTrackingStreamHandler.cs new file mode 100644 index 00000000..4a2f12df --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/LazyTrackingStreamHandler.cs @@ -0,0 +1,50 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + internal class LazyStreamQuery : IStreamQuery { } + + /// + /// Handler that increments a shared counter before each yield, allowing tests to + /// observe how many items have been produced at any given point during enumeration. + /// + internal class LazyTrackingStreamHandler : IStreamQueryHandler + { + public const int TotalItems = 5; + + private readonly int[] _producedCount; + + public LazyTrackingStreamHandler(int[] producedCount) => _producedCount = producedCount; + + public IQueryContext Context { get; set; } + + public async IAsyncEnumerable ExecuteAsync( + LazyStreamQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + for (int i = 1; i <= TotalItems; i++) + { + cancellationToken.ThrowIfCancellationRequested(); + _producedCount[0]++; + yield return i; + } + await Task.CompletedTask; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_consuming_stream_query_should_produce_items_lazily.cs b/test/Paramore.Darker.Core.Tests/When_consuming_stream_query_should_produce_items_lazily.cs new file mode 100644 index 00000000..df0a97df --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_consuming_stream_query_should_produce_items_lazily.cs @@ -0,0 +1,74 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_consuming_stream_query_should_produce_items_lazily + { + private static QueryProcessor BuildProcessor(int[] producedCount) + { + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => new LazyTrackingStreamHandler(producedCount)); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public async Task When_consuming_a_stream_query_should_observe_first_item_before_handler_produces_last() + { + // Arrange + var producedCount = new int[1]; + var processor = BuildProcessor(producedCount); + var query = new LazyStreamQuery(); + + // Act — use manual enumerator to capture the production count the instant the first item arrives + var enumerator = processor.ExecuteStream(query).GetAsyncEnumerator(); + try + { + await enumerator.MoveNextAsync(); + var firstItem = enumerator.Current; + var producedWhenFirstObserved = producedCount[0]; + + // Exhaust remaining items + while (await enumerator.MoveNextAsync()) { } + + // Assert — only 1 item produced when consumer observes the first item (no eager buffering) + producedWhenFirstObserved.ShouldBe(1, + "the framework must not buffer the stream; only the item just yielded should have been produced"); + producedWhenFirstObserved.ShouldBeLessThan(LazyTrackingStreamHandler.TotalItems); + firstItem.ShouldBe(1); + producedCount[0].ShouldBe(LazyTrackingStreamHandler.TotalItems); + } + finally + { + await enumerator.DisposeAsync(); + } + } + } +} From d50117f58565223e173828e096da710376e426c0 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 14:30:38 +0100 Subject: [PATCH 13/37] test: Verify mid-stream cancellation stops enumeration and propagates OperationCanceledException (T010) Co-Authored-By: Claude Sonnet 4.6 --- ...op_and_throw_OperationCanceledException.cs | 74 +++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/When_stream_cancelled_mid_enumeration_should_stop_and_throw_OperationCanceledException.cs diff --git a/test/Paramore.Darker.Core.Tests/When_stream_cancelled_mid_enumeration_should_stop_and_throw_OperationCanceledException.cs b/test/Paramore.Darker.Core.Tests/When_stream_cancelled_mid_enumeration_should_stop_and_throw_OperationCanceledException.cs new file mode 100644 index 00000000..93a6bde0 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_stream_cancelled_mid_enumeration_should_stop_and_throw_OperationCanceledException.cs @@ -0,0 +1,74 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Threading; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_stream_cancelled_mid_enumeration_should_stop_and_throw_OperationCanceledException + { + private static QueryProcessor BuildProcessor() + { + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => new MultiItemStreamHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public async Task When_stream_cancelled_mid_enumeration_should_stop_and_propagate_OperationCanceledException() + { + // Arrange — MultiItemStreamHandler yields 3 items and checks ThrowIfCancellationRequested before each + var processor = BuildProcessor(); + using var cts = new CancellationTokenSource(); + var receivedItems = new List(); + OperationCanceledException caughtException = null; + + // Act — WithCancellation flows the token into the handler via [EnumeratorCancellation] + try + { + await foreach (var item in processor.ExecuteStream(new MultiItemStreamQuery()) + .WithCancellation(cts.Token)) + { + receivedItems.Add(item); + cts.Cancel(); // signal cancellation after observing the first item + } + } + catch (OperationCanceledException ex) + { + caughtException = ex; + } + + // Assert + caughtException.ShouldNotBeNull("cancellation must propagate as OperationCanceledException"); + receivedItems.Count.ShouldBe(1, "enumeration must stop after the cancellation signal"); + receivedItems[0].ShouldBe(MultiItemStreamHandler.Items[0]); + } + } +} From 465b216f556ad500b792d23841c85628bdb9f8f8 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 14:37:40 +0100 Subject: [PATCH 14/37] test: Verify mid-stream exceptions surface unwrapped without TargetInvocationException (T011) Co-Authored-By: Claude Sonnet 4.6 --- .../TestDoubles/FaultingStreamHandler.cs | 45 +++++++++++ ...ation_should_surface_original_exception.cs | 76 +++++++++++++++++++ 2 files changed, 121 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/FaultingStreamHandler.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_stream_handler_throws_during_enumeration_should_surface_original_exception.cs diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/FaultingStreamHandler.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/FaultingStreamHandler.cs new file mode 100644 index 00000000..8a3ea7d2 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/FaultingStreamHandler.cs @@ -0,0 +1,45 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + internal class FaultingStreamQuery : IStreamQuery { } + + /// + /// Yields a fixed number of items then throws, allowing tests to verify that exceptions + /// mid-stream surface unwrapped (not as TargetInvocationException). + /// + internal class FaultingStreamHandler : IStreamQueryHandler + { + public const string ExceptionMessage = "Fault during stream enumeration"; + public const int ItemsBeforeFault = 2; + + public IQueryContext Context { get; set; } + +#pragma warning disable CS1998 + public async IAsyncEnumerable ExecuteAsync( + FaultingStreamQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) +#pragma warning restore CS1998 + { + yield return "item-1"; + yield return "item-2"; + throw new InvalidOperationException(ExceptionMessage); + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_stream_handler_throws_during_enumeration_should_surface_original_exception.cs b/test/Paramore.Darker.Core.Tests/When_stream_handler_throws_during_enumeration_should_surface_original_exception.cs new file mode 100644 index 00000000..aab7264e --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_stream_handler_throws_during_enumeration_should_surface_original_exception.cs @@ -0,0 +1,76 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Reflection; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_stream_handler_throws_during_enumeration_should_surface_original_exception + { + private static QueryProcessor BuildProcessor() + { + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => new FaultingStreamHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public async Task When_stream_handler_throws_during_enumeration_should_surface_original_exception_not_wrapped() + { + // Arrange — FaultingStreamHandler yields 2 items then throws InvalidOperationException + var processor = BuildProcessor(); + var receivedItems = new List(); + Exception caughtException = null; + + // Act + try + { + await foreach (var item in processor.ExecuteStream(new FaultingStreamQuery())) + receivedItems.Add(item); + } + catch (Exception ex) + { + caughtException = ex; + } + + // Assert — original exception type and message reach the caller unwrapped + caughtException.ShouldNotBeNull(); + caughtException.ShouldBeOfType( + "iterator body faults propagate directly from MoveNextAsync, not via TargetInvocationException"); + caughtException.Message.ShouldBe(FaultingStreamHandler.ExceptionMessage); + caughtException.ShouldNotBeOfType(); + + // Items produced before the fault were still observed + receivedItems.Count.ShouldBe(FaultingStreamHandler.ItemsBeforeFault); + receivedItems[0].ShouldBe("item-1"); + receivedItems[1].ShouldBe("item-2"); + } + } +} From 1a1148e2a318682438b219ba8057424b9cfafe65 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 15:01:41 +0100 Subject: [PATCH 15/37] test: Verify early stream break releases handler and ends span exactly once (T012) Co-Authored-By: Claude Sonnet 4.6 --- ...ould_release_pipeline_and_end_span_once.cs | 91 +++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/When_stream_consumer_breaks_early_should_release_pipeline_and_end_span_once.cs diff --git a/test/Paramore.Darker.Core.Tests/When_stream_consumer_breaks_early_should_release_pipeline_and_end_span_once.cs b/test/Paramore.Darker.Core.Tests/When_stream_consumer_breaks_early_should_release_pipeline_and_end_span_once.cs new file mode 100644 index 00000000..a6b96a5f --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_stream_consumer_breaks_early_should_release_pipeline_and_end_span_once.cs @@ -0,0 +1,91 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Diagnostics; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Observability; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + [Collection("DarkerActivitySource")] + public class When_stream_consumer_breaks_early_should_release_pipeline_and_end_span_once + { + private static ActivityListener CreateListener(List completed) + { + var listener = new ActivityListener + { + ShouldListenTo = s => s.Name == DarkerSemanticConventions.SourceName, + Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllData, + SampleUsingParentId = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllData, + ActivityStopped = a => completed.Add(a), + }; + ActivitySource.AddActivityListener(listener); + return listener; + } + + [Fact] + public async Task When_stream_consumer_breaks_early_should_release_handler_and_end_span_exactly_once() + { + // Arrange — recording factory tracks which handlers were released + var completed = new List(); + using var listener = CreateListener(completed); + using var tracer = new DarkerTracer(); + + IQueryHandler createdHandler = null; + var handlerFactory = new RecordingHandlerFactory(t => + { + createdHandler = new MultiItemStreamHandler(); + return createdHandler; + }); + + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + var config = new HandlerConfiguration( + new QueryHandlerRegistry(), handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + var processor = new QueryProcessor( + config, + new InMemoryQueryContextFactory(), + tracer: tracer, + instrumentationOptions: InstrumentationOptions.QueryInformation); + + // Act — break after the first item (early consumer exit) + var receivedItems = new List(); + await foreach (var item in processor.ExecuteStream(new MultiItemStreamQuery())) + { + receivedItems.Add(item); + break; + } + + // Assert — handler released exactly once via the try/finally in ExecuteStream + createdHandler.ShouldNotBeNull(); + handlerFactory.ReleaseCount(createdHandler).ShouldBe(1, + "the ExecuteStream finally block must release the handler exactly once on early break"); + + // Assert — span ended exactly once (inner finally calls EndSpan, outer finally disposes pipeline) + completed.Count.ShouldBe(1, "the span must be ended exactly once when the consumer breaks early"); + + receivedItems.Count.ShouldBe(1); + receivedItems[0].ShouldBe(MultiItemStreamHandler.Items[0]); + } + } +} From 7f7e4d0bcf365a1d264804792dc05c6e8cbb1f95 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 15:12:11 +0100 Subject: [PATCH 16/37] test: Verify stream query passed to ExecuteAsync throws ConfigurationException (T013) Co-Authored-By: Claude Sonnet 4.6 --- ...ync_should_throw_ConfigurationException.cs | 61 +++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/When_stream_query_passed_to_ExecuteAsync_should_throw_ConfigurationException.cs diff --git a/test/Paramore.Darker.Core.Tests/When_stream_query_passed_to_ExecuteAsync_should_throw_ConfigurationException.cs b/test/Paramore.Darker.Core.Tests/When_stream_query_passed_to_ExecuteAsync_should_throw_ConfigurationException.cs new file mode 100644 index 00000000..dfd52158 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_stream_query_passed_to_ExecuteAsync_should_throw_ConfigurationException.cs @@ -0,0 +1,61 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Exceptions; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_stream_query_passed_to_ExecuteAsync_should_throw_ConfigurationException + { + private static QueryProcessor BuildProcessor() + { + // MultiItemStreamQuery is registered only in the stream registry, + // not in the async registry — so ExecuteAsync cannot find a handler. + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => + throw new InvalidOperationException("should not be called")); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => + throw new InvalidOperationException("should not be called")); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public async Task When_stream_query_passed_to_ExecuteAsync_should_throw_ConfigurationException_for_no_async_handler() + { + // Arrange — IStreamQuery also satisfies IQuery, so this compiles + var processor = BuildProcessor(); + + // Act — passing a stream query to ExecuteAsync; stream handlers live only in the stream registry + var exception = await Should.ThrowAsync( + () => processor.ExecuteAsync(new MultiItemStreamQuery())); + + // Assert — clear message indicating no async handler (not a misleading error) + exception.Message.ShouldContain(nameof(MultiItemStreamQuery)); + } + } +} From 8f926aae11a567d36496045399314b85ee5e80ad Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 16:05:08 +0100 Subject: [PATCH 17/37] test: Verify ConfigurationException is deferred to first MoveNextAsync for unregistered stream query (T014) Co-Authored-By: Claude Sonnet 4.6 --- ...nfigurationException_on_first_move_next.cs | 95 +++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/When_no_stream_handler_registered_should_throw_ConfigurationException_on_first_move_next.cs diff --git a/test/Paramore.Darker.Core.Tests/When_no_stream_handler_registered_should_throw_ConfigurationException_on_first_move_next.cs b/test/Paramore.Darker.Core.Tests/When_no_stream_handler_registered_should_throw_ConfigurationException_on_first_move_next.cs new file mode 100644 index 00000000..07242c34 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_no_stream_handler_registered_should_throw_ConfigurationException_on_first_move_next.cs @@ -0,0 +1,95 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Exceptions; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_no_stream_handler_registered_should_throw_ConfigurationException_on_first_move_next + { + private static QueryProcessor BuildProcessorWithEmptyStreamRegistry() + { + // Empty stream registry — MultiItemStreamQuery is not registered + var emptyStreamRegistry = new StreamQueryHandlerRegistry(); + + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => + throw new InvalidOperationException("should not be called")); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => + throw new InvalidOperationException("should not be called")); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + emptyStreamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public void When_no_stream_handler_registered_should_not_throw_at_call_time() + { + // Arrange + var processor = BuildProcessorWithEmptyStreamRegistry(); + + // Act — ExecuteStream is a deferred iterator; calling it must not throw + IAsyncEnumerable stream = null; + var callSiteException = Record.Exception(() => + { + stream = processor.ExecuteStream(new MultiItemStreamQuery()); + }); + + // Assert — no exception thrown at call time (deferred evaluation) + callSiteException.ShouldBeNull( + "ExecuteStream is an async iterator; the body (including handler resolution) runs on MoveNextAsync, not here"); + stream.ShouldNotBeNull(); + } + + [Fact] + public async Task When_no_stream_handler_registered_should_throw_ConfigurationException_on_first_MoveNextAsync() + { + // Arrange — handler resolution runs inside the iterator, deferred until first MoveNextAsync + var processor = BuildProcessorWithEmptyStreamRegistry(); + var stream = processor.ExecuteStream(new MultiItemStreamQuery()); + + // Act — first MoveNextAsync triggers handler resolution, which throws + var enumerator = stream.GetAsyncEnumerator(); + Exception caughtException = null; + try + { + await enumerator.MoveNextAsync(); + } + catch (Exception ex) + { + caughtException = ex; + } + finally + { + await enumerator.DisposeAsync(); + } + + // Assert — ConfigurationException surfaces on the first MoveNextAsync, not at call time + caughtException.ShouldNotBeNull(); + caughtException.ShouldBeOfType( + "handler resolution inside the iterator must throw ConfigurationException for an unregistered query"); + caughtException.Message.ShouldContain(nameof(MultiItemStreamQuery)); + } + } +} From 61d819fd65fdb53e35c717bf55647e3664f962a2 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 16:57:52 +0100 Subject: [PATCH 18/37] feat: Add stream decorator pipeline support ordered by Step descending (T015) Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 30 +++---- src/Paramore.Darker/PipelineBuilder.cs | 80 ++++++++++++++++++- .../TestDoubles/StepOrderStreamHandler.cs | 47 +++++++++++ .../TestDoubles/StreamStepEventDecorator.cs | 68 ++++++++++++++++ ...orators_should_order_by_step_descending.cs | 64 +++++++++++++++ 5 files changed, 270 insertions(+), 19 deletions(-) create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/StepOrderStreamHandler.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/StreamStepEventDecorator.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_stream_handler_has_multiple_decorators_should_order_by_step_descending.cs diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index 6f4de1c5..256b9267 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -26,14 +26,14 @@ ## Phase 0 — Structural foundations (Tidy First, no behaviour) -- [ ] **STRUCTURAL: T001 — `IAsyncEnumerable` available on all targets** +- [x] **STRUCTURAL: T001 — `IAsyncEnumerable` available on all targets** - Add `Microsoft.Bcl.AsyncInterfaces` version to `Directory.Packages.props` (CPM). - Add a **conditional** `` in `src/Paramore.Darker/Paramore.Darker.csproj` guarded to `netstandard2.0` only (native on net8.0/net9.0). - Verify `dotnet build Darker.Filter.slnf -c Release` still succeeds on all targets. - ADR §7. No behaviour; separate commit. -- [ ] **STRUCTURAL: T002 — Stream query + handler contracts** +- [x] **STRUCTURAL: T002 — Stream query + handler contracts** - Add `src/Paramore.Darker/IStreamQuery.cs`: `public interface IStreamQuery : IQuery { }` (TResult = **item** type). - Add `src/Paramore.Darker/IStreamQueryHandler.cs`: @@ -43,7 +43,7 @@ and abstract `ExecuteAsync` (no `Fallback` method — ADR §2). XML docs incl. licence header. - ADR §2. Must compile; no behaviour. -- [ ] **STRUCTURAL: T003 — Stream decorator contract + attribute base** +- [x] **STRUCTURAL: T003 — Stream decorator contract + attribute base** - Add `src/Paramore.Darker/IStreamQueryHandlerDecorator.cs`: `IStreamQueryHandlerDecorator : IQueryHandlerDecorator where TQuery : IStreamQuery` with `IAsyncEnumerable Execute(TQuery query, Func> next, CancellationToken cancellationToken)`. @@ -56,7 +56,7 @@ ## Phase 1 — Handler registry (behaviour) -- [ ] **TEST + IMPLEMENT: T004 — Stream registry resolves a stream handler by query type** +- [x] **TEST + IMPLEMENT: T004 — Stream registry resolves a stream handler by query type** - **USE COMMAND**: `/test-first when stream query registered should resolve its stream handler type from the stream registry` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_query_registered_should_resolve_stream_handler_type.cs` @@ -75,7 +75,7 @@ `QueryHandlerRegistryAsync` (dictionary, duplicate + result-type guards) - Requires a stream query + handler test double in `TestDoubles/` -- [ ] **TEST + IMPLEMENT: T005 — Assembly scan registers only stream handlers** +- [x] **TEST + IMPLEMENT: T005 — Assembly scan registers only stream handlers** - **USE COMMAND**: `/test-first when scanning assemblies for stream handlers should register IStreamQueryHandler implementations and ignore async handlers` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_scanning_assemblies_for_stream_handlers_should_register_only_stream_handlers.cs` @@ -93,7 +93,7 @@ ## Phase 2 — Pipeline build + processor entry point (MVP happy path) -- [ ] **STRUCTURAL: T006 — Thread the stream registry through builder + configuration** +- [x] **STRUCTURAL: T006 — Thread the stream registry through builder + configuration** - `PipelineBuilder` gains a new ctor parameter for `IStreamQueryHandlerRegistry` (ADR §5.3) — additive, existing ctors unaffected. - `IHandlerConfiguration` / `HandlerConfiguration` gain an optional @@ -101,7 +101,7 @@ - `QueryProcessor` reads `StreamHandlerRegistry` off the configuration into a field (ADR §5.4). - Must compile; no behaviour yet. -- [ ] **TEST + IMPLEMENT: T007 — `BuildStream` resolves the handler and yields its items (no decorators)** +- [x] **TEST + IMPLEMENT: T007 — `BuildStream` resolves the handler and yields its items (no decorators)** - **USE COMMAND**: `/test-first when building a stream pipeline with no decorators should invoke the handler and yield its items` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_building_stream_pipeline_without_decorators_should_yield_handler_items.cs` @@ -122,7 +122,7 @@ - Sink invokes the handler's stream `ExecuteAsync` returning the enumerable — **no** `TargetInvocationException` unwrap around enumeration (iterator defers, ADR §4/§6) -- [ ] **TEST + IMPLEMENT: T008 — `ExecuteStream` runs a stream query end-to-end** +- [x] **TEST + IMPLEMENT: T008 — `ExecuteStream` runs a stream query end-to-end** - **USE COMMAND**: `/test-first when executing a stream query through the processor should yield all handler items via await foreach` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_executing_stream_query_should_yield_all_items.cs` @@ -145,7 +145,7 @@ ## Phase 3 — Core streaming correctness properties (each a distinct behaviour) -- [ ] **TEST + IMPLEMENT: T009 — Items are produced lazily, not buffered** +- [x] **TEST + IMPLEMENT: T009 — Items are produced lazily, not buffered** - **USE COMMAND**: `/test-first when consuming a stream query should observe the first item before the handler produces the last item` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_consuming_stream_query_should_produce_items_lazily.cs` @@ -158,7 +158,7 @@ - Confirm the `ExecuteStream` + `BuildStream` chain are `async` iterators end-to-end (no `ToListAsync`/eager await); fix any eager materialisation surfaced by the test -- [ ] **TEST + IMPLEMENT: T010 — Cancellation stops enumeration promptly** +- [x] **TEST + IMPLEMENT: T010 — Cancellation stops enumeration promptly** - **USE COMMAND**: `/test-first when the cancellation token is cancelled mid-stream should stop enumeration and propagate OperationCanceledException` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_cancelled_mid_enumeration_should_stop_and_throw_OperationCanceledException.cs` @@ -172,7 +172,7 @@ - Ensure `ExecuteStream` applies `[EnumeratorCancellation]` and passes `.WithCancellation(ct)` to the inner enumeration (ADR §4); no special-casing beyond token flow -- [ ] **TEST + IMPLEMENT: T011 — Exceptions mid-stream surface unwrapped** +- [x] **TEST + IMPLEMENT: T011 — Exceptions mid-stream surface unwrapped** - **USE COMMAND**: `/test-first when the handler throws during enumeration should surface the original exception to the caller with its stack trace` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_handler_throws_during_enumeration_should_surface_original_exception.cs` @@ -185,7 +185,7 @@ - Confirm no `TargetInvocationException` unwrap is applied around enumeration (the iterator defers, so `Invoke` returns the enumerable without running the body — ADR §4) -- [ ] **TEST + IMPLEMENT: T012 — Early `break` releases handler, decorators, and span** +- [x] **TEST + IMPLEMENT: T012 — Early `break` releases handler, decorators, and span** - **USE COMMAND**: `/test-first when the caller breaks out of the stream early should release the handler decorators and end the span exactly once` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_consumer_breaks_early_should_release_pipeline_and_end_span_once.cs` @@ -198,7 +198,7 @@ - Rely on the `try/finally` in the `ExecuteStream` iterator running on enumerator `DisposeAsync` (ADR §4); use `RecordingHandlerFactory` / `RecordingDecoratorFactory` doubles to assert release -- [ ] **TEST + IMPLEMENT: T013 — Stream query sent to `ExecuteAsync` fails cleanly** +- [x] **TEST + IMPLEMENT: T013 — Stream query sent to `ExecuteAsync` fails cleanly** - **USE COMMAND**: `/test-first when a stream query is passed to ExecuteAsync should throw ConfigurationException for no async handler` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_query_passed_to_ExecuteAsync_should_throw_ConfigurationException.cs` @@ -211,7 +211,7 @@ - No new code expected beyond existing async-registry miss behaviour; test documents/locks the cross-path guarantee -- [ ] **TEST + IMPLEMENT: T014 — Missing/mismatched stream handler surfaces on first `MoveNextAsync`** +- [x] **TEST + IMPLEMENT: T014 — Missing/mismatched stream handler surfaces on first `MoveNextAsync`** - **USE COMMAND**: `/test-first when no stream handler is registered should throw ConfigurationException on the first enumeration step` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_no_stream_handler_registered_should_throw_ConfigurationException_on_first_move_next.cs` @@ -228,7 +228,7 @@ ## Phase 4 — Stream decorator pipeline (behaviour) -- [ ] **TEST + IMPLEMENT: T015 — Stream decorators run ordered by `Step` descending** +- [x] **TEST + IMPLEMENT: T015 — Stream decorators run ordered by `Step` descending** - **USE COMMAND**: `/test-first when a stream handler has multiple stream decorators should execute them ordered by step descending around the handler` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_handler_has_multiple_decorators_should_order_by_step_descending.cs` diff --git a/src/Paramore.Darker/PipelineBuilder.cs b/src/Paramore.Darker/PipelineBuilder.cs index e49472da..1e3191fd 100644 --- a/src/Paramore.Darker/PipelineBuilder.cs +++ b/src/Paramore.Darker/PipelineBuilder.cs @@ -33,6 +33,7 @@ internal sealed class PipelineBuilder : IDisposable private IQueryHandler _handler; private IReadOnlyList, TResult>> _decorators; private IReadOnlyList, TResult>> _asyncDecorators; + private IReadOnlyList, TResult>> _streamDecorators; private IAmALifetime _lifetime; public PipelineBuilder( @@ -289,14 +290,74 @@ public Func, CancellationToken, IAsyncEnumerable> ValidateNoMismatchedAttributes(executeMethodInfo, typeof(QueryHandlerAttributeAsync), "Stream handler has async attribute(s) on ExecuteAsync. Use StreamQueryHandlerAttribute for stream handlers."); + _streamDecorators = GetStreamDecorators(executeMethodInfo, queryContext); + var span = queryContext.Span; - // No TargetInvocationException unwrap: the iterator body runs on MoveNextAsync, not Invoke. - return (r, ct) => + // Sink: no TargetInvocationException unwrap — iterator body runs on MoveNextAsync, not Invoke. + var pipeline = new List, CancellationToken, IAsyncEnumerable>> { - DarkerTracer.WriteQueryEvent(span, handlerType.Name, isAsync: true, instrumentationOptions, isSink: true); - return (IAsyncEnumerable)executeMethodInfo.Invoke(_handler, new object[] { r, ct }); + (r, ct) => + { + DarkerTracer.WriteQueryEvent(span, handlerType.Name, isAsync: true, instrumentationOptions, isSink: true); + return (IAsyncEnumerable)executeMethodInfo.Invoke(_handler, new object[] { r, ct }); + } }; + + foreach (var decorator in _streamDecorators) + { + _logger.LogDebug("Adding stream decorator to pipeline: {Decorator}", decorator.GetType().Name); + var next = pipeline.Last(); + var dec = decorator; + pipeline.Add((r, ct) => + { + DarkerTracer.WriteQueryEvent(span, dec.GetType().Name, isAsync: true, instrumentationOptions); + return dec.Execute(r, next, ct); + }); + } + + return pipeline.Last(); + } + + private IReadOnlyList, TResult>> GetStreamDecorators(MemberInfo executeMethod, IQueryContext queryContext) + { + var attributes = executeMethod.GetCustomAttributes(typeof(StreamQueryHandlerAttribute), true) + .Cast() + .OrderByDescending(attr => attr.Step) + .ToList(); + + _logger.LogDebug("Found {AttributesCount} stream query handler attributes", attributes.Count); + + var decorators = new List, TResult>>(); + + foreach (var attribute in attributes) + { + var decoratorType = attribute.GetDecoratorType().MakeGenericType(typeof(IStreamQuery), typeof(TResult)); + + _logger.LogDebug("Resolving stream decorator instance of type {DecoratorType}...", decoratorType.Name); + + IStreamQueryHandlerDecorator, TResult> decorator; + if (_decoratorFactoryAsync != null) + decorator = _decoratorFactoryAsync.Create, TResult>>(decoratorType, _lifetime); + else if (_decoratorFactory != null) + decorator = _decoratorFactory.Create, TResult>>(decoratorType, _lifetime); + else + throw new ConfigurationException($"No decorator factory configured. Cannot create stream decorator for type: {decoratorType.FullName}"); + + if (decorator == null) + throw new ConfigurationException($"Stream decorator could not be created for type: {decoratorType.FullName}. Ensure it is registered in the decorator registry."); + + decorator.Context = queryContext; + + _logger.LogDebug("Initialising stream decorator from attribute params..."); + decorator.InitializeFromAttributeParams(attribute.GetAttributeParams()); + + decorators.Add(decorator); + } + + _logger.LogDebug("Finished initialising {DecoratorsCount} stream decorators", decorators.Count); + + return decorators; } private (Type handlerType, IQueryHandler handler) ResolveStreamHandler(Type queryType) @@ -378,6 +439,17 @@ public void Dispose() } } + if (_streamDecorators != null && _streamDecorators.Any()) + { + foreach (var decorator in _streamDecorators) + { + if (_decoratorFactoryAsync != null) + _decoratorFactoryAsync.Release(decorator, _lifetime); + else + _decoratorFactory?.Release(decorator, _lifetime); + } + } + // Dispose the per-query lifetime last, after releasing the handler and decorators, so // any resources it owns (e.g. a child service scope) are torn down once per query. _lifetime?.Dispose(); diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/StepOrderStreamHandler.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/StepOrderStreamHandler.cs new file mode 100644 index 00000000..0cba3994 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/StepOrderStreamHandler.cs @@ -0,0 +1,47 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + internal class StepOrderStreamQuery : IStreamQuery { } + + /// + /// A stream handler decorated with two s to verify + /// that PipelineBuilder.BuildStream orders decorators by step descending (step 2 → step 1 → handler). + /// + internal class StepOrderStreamHandler : IStreamQueryHandler + { + public static readonly string[] Items = { "a", "b" }; + + public IQueryContext Context { get; set; } + + [StreamStepEvent(step: 2)] + [StreamStepEvent(step: 1)] + public async IAsyncEnumerable ExecuteAsync( + StepOrderStreamQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + foreach (var item in Items) + { + cancellationToken.ThrowIfCancellationRequested(); + yield return item; + } + await Task.CompletedTask; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/StreamStepEventDecorator.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamStepEventDecorator.cs new file mode 100644 index 00000000..1e8ea774 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamStepEventDecorator.cs @@ -0,0 +1,68 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// Attribute that wires into the + /// stream pipeline for use in step-ordering tests. + /// + [AttributeUsage(AttributeTargets.Method, AllowMultiple = true)] + internal sealed class StreamStepEventAttribute : StreamQueryHandlerAttribute + { + public StreamStepEventAttribute(int step) : base(step) { } + + public override object[] GetAttributeParams() => new object[] { Step }; + + public override Type GetDecoratorType() => typeof(StreamStepEventDecorator<,>); + } + + /// + /// A pass-through stream decorator that records its step number into a shared list when + /// it first executes, allowing tests to verify that the stream pipeline orders decorators + /// by descending. + /// + internal sealed class StreamStepEventDecorator : IStreamQueryHandlerDecorator + where TQuery : IStreamQuery + { + private readonly List _enteredSteps; + private int _step; + + public StreamStepEventDecorator(List enteredSteps) => _enteredSteps = enteredSteps; + + public IQueryContext Context { get; set; } + + public void InitializeFromAttributeParams(object[] attributeParams) + { + _step = (int)attributeParams[0]; + } + + public async IAsyncEnumerable Execute( + TQuery query, + Func> next, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + var first = true; + await foreach (var item in next(query, cancellationToken)) + { + if (first) { _enteredSteps.Add(_step); first = false; } + yield return item; + } + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_stream_handler_has_multiple_decorators_should_order_by_step_descending.cs b/test/Paramore.Darker.Core.Tests/When_stream_handler_has_multiple_decorators_should_order_by_step_descending.cs new file mode 100644 index 00000000..976c207b --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_stream_handler_has_multiple_decorators_should_order_by_step_descending.cs @@ -0,0 +1,64 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_stream_handler_has_multiple_decorators_should_order_by_step_descending + { + private static QueryProcessor BuildProcessor(List enteredSteps) + { + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => new StepOrderStreamHandler()); + // Decorator factory creates StreamStepEventDecorator with the shared recording list; + // the step number is set later via InitializeFromAttributeParams. + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => + new StreamStepEventDecorator, string>(enteredSteps)); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public async Task When_stream_handler_has_multiple_decorators_should_execute_them_ordered_by_step_descending() + { + // Arrange — StepOrderStreamHandler declares [StreamStepEvent(2)] and [StreamStepEvent(1)] + var enteredSteps = new List(); + var processor = BuildProcessor(enteredSteps); + + // Act + var results = new List(); + await foreach (var item in processor.ExecuteStream(new StepOrderStreamQuery())) + results.Add(item); + + // Assert — step 2 entered first (outermost/highest step wraps first), then step 1 + enteredSteps.Count.ShouldBe(2, "both decorators must execute"); + enteredSteps[0].ShouldBe(2, "higher Step executes first (outermost decorator)"); + enteredSteps[1].ShouldBe(1, "lower Step executes second (inner decorator)"); + results.ShouldBe(StepOrderStreamHandler.Items); + } + } +} From 15e498abb09f4f0466823450edd0d9393c8c19a0 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 17:34:49 +0100 Subject: [PATCH 19/37] feat: Reject mismatched decorator attributes between stream and sync/async pipelines (T016) Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 2 +- src/Paramore.Darker/PipelineBuilder.cs | 4 + .../AsyncHandlerWithStreamAttribute.cs | 29 ++++ .../StreamHandlerWithAsyncAttribute.cs | 39 +++++ .../StreamHandlerWithSyncAttribute.cs | 39 +++++ .../SyncHandlerWithStreamAttribute.cs | 26 +++ ...ute_should_throw_ConfigurationException.cs | 163 ++++++++++++++++++ 7 files changed, 301 insertions(+), 1 deletion(-) create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/AsyncHandlerWithStreamAttribute.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/StreamHandlerWithAsyncAttribute.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/StreamHandlerWithSyncAttribute.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/SyncHandlerWithStreamAttribute.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_stream_handler_has_mismatched_decorator_attribute_should_throw_ConfigurationException.cs diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index 256b9267..966be1be 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -242,7 +242,7 @@ `IStreamQueryHandlerDecorator` instances via `next` (ADR §6), reusing the async decorator factory - Add step-event stream decorator + attribute test doubles in `TestDoubles/` -- [ ] **TEST + IMPLEMENT: T016 — Mismatched decorator attributes are rejected** +- [x] **TEST + IMPLEMENT: T016 — Mismatched decorator attributes are rejected** - **USE COMMAND**: `/test-first when a stream handler has a sync or async decorator attribute should throw ConfigurationException and vice versa` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_handler_has_mismatched_decorator_attribute_should_throw_ConfigurationException.cs` diff --git a/src/Paramore.Darker/PipelineBuilder.cs b/src/Paramore.Darker/PipelineBuilder.cs index 1e3191fd..2cb69761 100644 --- a/src/Paramore.Darker/PipelineBuilder.cs +++ b/src/Paramore.Darker/PipelineBuilder.cs @@ -72,6 +72,8 @@ public Func, TResult> Build(IQuery query, IQueryContext var executeMethodInfo = GetExecuteMethodInfo(handlerType, queryType) as MethodInfo; ValidateNoMismatchedAttributes(executeMethodInfo, typeof(QueryHandlerAttributeAsync), "Sync handler has async attribute(s) on Execute. Use sync attributes (e.g. QueryHandlerAttribute) for sync handlers, or switch to an async handler with ExecuteAsync."); + ValidateNoMismatchedAttributes(executeMethodInfo, typeof(StreamQueryHandlerAttribute), + "Sync handler has stream attribute(s) on Execute. Use StreamQueryHandlerAttribute only on stream handlers implementing IStreamQueryHandler."); _decorators = GetDecorators(executeMethodInfo, queryContext); // Capture the span once; null when no tracer is configured (WriteQueryEvent is null-safe). @@ -134,6 +136,8 @@ public Func, CancellationToken, Task> BuildAsync(IQuery ValidateNoMismatchedAttributes(executeAsyncMethodInfo, typeof(QueryHandlerAttribute), "Async handler has sync attribute(s) on ExecuteAsync. Use async attributes (e.g. QueryHandlerAttributeAsync) for async handlers, or switch to a sync handler with Execute."); + ValidateNoMismatchedAttributes(executeAsyncMethodInfo, typeof(StreamQueryHandlerAttribute), + "Async handler has stream attribute(s) on ExecuteAsync. Use StreamQueryHandlerAttribute only on stream handlers implementing IStreamQueryHandler."); _asyncDecorators = GetDecoratorsAsync(executeAsyncMethodInfo, queryContext); diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/AsyncHandlerWithStreamAttribute.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/AsyncHandlerWithStreamAttribute.cs new file mode 100644 index 00000000..60192e0c --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/AsyncHandlerWithStreamAttribute.cs @@ -0,0 +1,29 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Threading; +using System.Threading.Tasks; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// An async handler that incorrectly uses a StreamQueryHandlerAttribute on ExecuteAsync. + /// This should cause a ConfigurationException at pipeline build time. + /// + internal class AsyncHandlerWithStreamAttribute : QueryHandlerAsync + { + [StreamStepEvent(1)] + public override Task ExecuteAsync(AsyncTestQuery query, CancellationToken cancellationToken = default) + => Task.FromResult(new AsyncTestQuery.Result { Value = query.Id }); + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/StreamHandlerWithAsyncAttribute.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamHandlerWithAsyncAttribute.cs new file mode 100644 index 00000000..6321ab58 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamHandlerWithAsyncAttribute.cs @@ -0,0 +1,39 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; +using Paramore.Darker.Policies.Attributes; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A stream handler that incorrectly uses an async QueryHandlerAttributeAsync on ExecuteAsync. + /// This should cause a ConfigurationException when BuildStream validates attribute types. + /// + internal class StreamHandlerWithAsyncAttribute : IStreamQueryHandler + { + public IQueryContext Context { get; set; } + + [FallbackPolicyAttributeAsync(1)] + public async IAsyncEnumerable ExecuteAsync( + StreamTestQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + yield return "item"; + await Task.CompletedTask; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/StreamHandlerWithSyncAttribute.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamHandlerWithSyncAttribute.cs new file mode 100644 index 00000000..6992938b --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamHandlerWithSyncAttribute.cs @@ -0,0 +1,39 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; +using Paramore.Darker.Policies.Attributes; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A stream handler that incorrectly uses a sync QueryHandlerAttribute on ExecuteAsync. + /// This should cause a ConfigurationException when BuildStream validates attribute types. + /// + internal class StreamHandlerWithSyncAttribute : IStreamQueryHandler + { + public IQueryContext Context { get; set; } + + [FallbackPolicy(1)] + public async IAsyncEnumerable ExecuteAsync( + StreamTestQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + yield return "item"; + await Task.CompletedTask; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/SyncHandlerWithStreamAttribute.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/SyncHandlerWithStreamAttribute.cs new file mode 100644 index 00000000..74999cb5 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/SyncHandlerWithStreamAttribute.cs @@ -0,0 +1,26 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A sync handler that incorrectly uses a StreamQueryHandlerAttribute on Execute. + /// This should cause a ConfigurationException at pipeline build time. + /// + internal class SyncHandlerWithStreamAttribute : QueryHandler + { + [StreamStepEvent(1)] + public override SyncTestQuery.Result Execute(SyncTestQuery query) + => new SyncTestQuery.Result { Value = query.Id }; + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_stream_handler_has_mismatched_decorator_attribute_should_throw_ConfigurationException.cs b/test/Paramore.Darker.Core.Tests/When_stream_handler_has_mismatched_decorator_attribute_should_throw_ConfigurationException.cs new file mode 100644 index 00000000..05bb38f2 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_stream_handler_has_mismatched_decorator_attribute_should_throw_ConfigurationException.cs @@ -0,0 +1,163 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Exceptions; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_stream_handler_has_mismatched_decorator_attribute_should_throw_ConfigurationException + { + private static QueryProcessor BuildProcessorForSync(QueryHandlerRegistry syncRegistry) + where THandler : class, IQueryHandler + { + var handlerFactory = new SimpleHandlerFactory(type => (IQueryHandler)Activator.CreateInstance(type)); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + private static QueryProcessor BuildProcessorForAsync(QueryHandlerRegistryAsync asyncRegistry) + where THandler : class, IQueryHandler + { + var handlerFactory = new SimpleHandlerFactory(type => (IQueryHandler)Activator.CreateInstance(type)); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + new QueryHandlerRegistry(), handlerFactory, decoratorRegistry, decoratorFactory, + asyncRegistry, handlerFactory, decoratorRegistry, decoratorFactory); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + private static QueryProcessor BuildProcessorForStream(StreamQueryHandlerRegistry streamRegistry) + where THandler : class, IQueryHandler + { + var handlerFactory = new SimpleHandlerFactory(type => (IQueryHandler)Activator.CreateInstance(type)); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + new QueryHandlerRegistry(), handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + return new QueryProcessor(config, new InMemoryQueryContextFactory()); + } + + [Fact] + public void When_sync_handler_has_stream_attribute_should_throw_configuration_exception() + { + // Arrange — sync handler with [StreamStepEvent] (StreamQueryHandlerAttribute) on Execute + var syncRegistry = new QueryHandlerRegistry(); + syncRegistry.Register(); + var processor = BuildProcessorForSync(syncRegistry); + + // Act + var exception = Should.Throw( + () => processor.Execute(new SyncTestQuery(Guid.NewGuid()))); + + // Assert + exception.Message.ShouldContain("stream", Case.Insensitive); + } + + [Fact] + public async Task When_async_handler_has_stream_attribute_should_throw_configuration_exception() + { + // Arrange — async handler with [StreamStepEvent] (StreamQueryHandlerAttribute) on ExecuteAsync + var asyncRegistry = new QueryHandlerRegistryAsync(); + asyncRegistry.Register(); + var processor = BuildProcessorForAsync(asyncRegistry); + + // Act + var exception = await Should.ThrowAsync( + () => processor.ExecuteAsync(new AsyncTestQuery(Guid.NewGuid()))); + + // Assert + exception.Message.ShouldContain("stream", Case.Insensitive); + } + + [Fact] + public async Task When_stream_handler_has_sync_attribute_should_throw_configuration_exception() + { + // Arrange — stream handler with [FallbackPolicy] (QueryHandlerAttribute) on ExecuteAsync + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + var processor = BuildProcessorForStream(streamRegistry); + + // Act — BuildStream runs inside the iterator; exception is deferred to first MoveNextAsync + var stream = processor.ExecuteStream(new StreamTestQuery()); + var enumerator = stream.GetAsyncEnumerator(); + Exception caughtException = null; + try + { + await enumerator.MoveNextAsync(); + } + catch (Exception ex) + { + caughtException = ex; + } + finally + { + await enumerator.DisposeAsync(); + } + + // Assert + caughtException.ShouldNotBeNull(); + caughtException.ShouldBeOfType(); + caughtException.Message.ShouldContain("sync", Case.Insensitive); + caughtException.Message.ShouldContain("stream", Case.Insensitive); + } + + [Fact] + public async Task When_stream_handler_has_async_attribute_should_throw_configuration_exception() + { + // Arrange — stream handler with [FallbackPolicyAttributeAsync] (QueryHandlerAttributeAsync) on ExecuteAsync + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + var processor = BuildProcessorForStream(streamRegistry); + + // Act — BuildStream runs inside the iterator; exception is deferred to first MoveNextAsync + var stream = processor.ExecuteStream(new StreamTestQuery()); + var enumerator = stream.GetAsyncEnumerator(); + Exception caughtException = null; + try + { + await enumerator.MoveNextAsync(); + } + catch (Exception ex) + { + caughtException = ex; + } + finally + { + await enumerator.DisposeAsync(); + } + + // Assert + caughtException.ShouldNotBeNull(); + caughtException.ShouldBeOfType(); + caughtException.Message.ShouldContain("async", Case.Insensitive); + caughtException.Message.ShouldContain("stream", Case.Insensitive); + } + } +} From 5be425118598b196e0783e4e269d1ee4de85aa43 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 17:43:11 +0100 Subject: [PATCH 20/37] test: Document that re-enumeration re-executes handler with fresh pipeline (T017) Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 2 +- ...e_should_re_execute_with_fresh_pipeline.cs | 70 +++++++++++++++++++ 2 files changed, 71 insertions(+), 1 deletion(-) create mode 100644 test/Paramore.Darker.Core.Tests/When_stream_enumerated_twice_should_re_execute_with_fresh_pipeline.cs diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index 966be1be..f5ebc151 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -256,7 +256,7 @@ in `BuildStream` (and add the reciprocal guard in `Build`/`BuildAsync` for `StreamQueryHandlerAttribute`) — ADR §6 -- [ ] **TEST + IMPLEMENT: T017 — Re-enumeration re-executes the query** +- [x] **TEST + IMPLEMENT: T017 — Re-enumeration re-executes the query** - **USE COMMAND**: `/test-first when a stream is enumerated twice should re-execute the handler with a fresh pipeline each time` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_enumerated_twice_should_re_execute_with_fresh_pipeline.cs` diff --git a/test/Paramore.Darker.Core.Tests/When_stream_enumerated_twice_should_re_execute_with_fresh_pipeline.cs b/test/Paramore.Darker.Core.Tests/When_stream_enumerated_twice_should_re_execute_with_fresh_pipeline.cs new file mode 100644 index 00000000..e57cca3e --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_stream_enumerated_twice_should_re_execute_with_fresh_pipeline.cs @@ -0,0 +1,70 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class When_stream_enumerated_twice_should_re_execute_with_fresh_pipeline + { + [Fact] + public async Task When_stream_is_enumerated_twice_should_yield_all_items_each_time() + { + // Arrange — count how many handler instances are created across both enumerations + int handlerCreationCount = 0; + var handlerFactory = new SimpleHandlerFactory(_ => + { + handlerCreationCount++; + return new MultiItemStreamHandler(); + }); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => null); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var config = new HandlerConfiguration( + new QueryHandlerRegistry(), handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + var processor = new QueryProcessor(config, new InMemoryQueryContextFactory()); + var query = new MultiItemStreamQuery(); + + // Act — obtain the IAsyncEnumerable once, then enumerate it twice + var stream = processor.ExecuteStream(query); + + var firstPass = new List(); + await foreach (var item in stream) + firstPass.Add(item); + + var secondPass = new List(); + await foreach (var item in stream) + secondPass.Add(item); + + // Assert — both passes yield the full item sequence (cold, not cached) + firstPass.ShouldBe(MultiItemStreamHandler.Items, ignoreOrder: false, + "first pass must yield all handler items in order"); + secondPass.ShouldBe(MultiItemStreamHandler.Items, ignoreOrder: false, + "second pass must re-execute the handler and yield all items again — the stream is cold, not cached"); + + // Assert — a fresh handler was created per enumeration (fresh PipelineBuilder per GetAsyncEnumerator) + handlerCreationCount.ShouldBe(2, + "each await foreach calls GetAsyncEnumerator, which starts a fresh async iterator body creating a new PipelineBuilder and handler"); + } + } +} From 82dbc5301f8a4f5224040bd1f08eda170c317edb Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 21:20:50 +0100 Subject: [PATCH 21/37] feat: Add stream logging decorator wrapping lifecycle with item count and duration (T018) Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 2 +- .../Attributes/StreamQueryLoggingAttribute.cs | 32 ++++ .../Handlers/StreamQueryLoggingDecorator.cs | 82 ++++++++++ ...d_should_wrap_lifecycle_with_item_count.cs | 150 ++++++++++++++++++ .../LoggedMultiItemStreamHandler.cs | 43 +++++ 5 files changed, 308 insertions(+), 1 deletion(-) create mode 100644 src/Paramore.Darker/Logging/Attributes/StreamQueryLoggingAttribute.cs create mode 100644 src/Paramore.Darker/Logging/Handlers/StreamQueryLoggingDecorator.cs create mode 100644 test/Paramore.Darker.Core.Tests/Logging/When_stream_query_logged_should_wrap_lifecycle_with_item_count.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/LoggedMultiItemStreamHandler.cs diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index f5ebc151..5118031c 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -273,7 +273,7 @@ ## Phase 5 — Logging / telemetry stream decorators (behaviour) -- [ ] **TEST + IMPLEMENT: T018 — Stream logging decorator wraps the stream lifecycle** +- [x] **TEST + IMPLEMENT: T018 — Stream logging decorator wraps the stream lifecycle** - **USE COMMAND**: `/test-first when a stream query has the logging decorator should log start yield each item and log completion with item count and duration` - Test location: `test/Paramore.Darker.Core.Tests` (Logging folder) - Test file: `When_stream_query_logged_should_wrap_lifecycle_with_item_count.cs` diff --git a/src/Paramore.Darker/Logging/Attributes/StreamQueryLoggingAttribute.cs b/src/Paramore.Darker/Logging/Attributes/StreamQueryLoggingAttribute.cs new file mode 100644 index 00000000..6f284555 --- /dev/null +++ b/src/Paramore.Darker/Logging/Attributes/StreamQueryLoggingAttribute.cs @@ -0,0 +1,32 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using Paramore.Darker.Logging.Handlers; + +namespace Paramore.Darker.Logging.Attributes +{ + /// + /// Wires into the stream pipeline. + /// Logs stream start (with serialised query body), item count, and elapsed duration on completion. + /// + [AttributeUsage(AttributeTargets.Method)] + public sealed class StreamQueryLoggingAttribute : StreamQueryHandlerAttribute + { + public StreamQueryLoggingAttribute(int step) : base(step) { } + + public override object[] GetAttributeParams() => new object[0]; + + public override Type GetDecoratorType() => typeof(StreamQueryLoggingDecorator<,>); + } +} diff --git a/src/Paramore.Darker/Logging/Handlers/StreamQueryLoggingDecorator.cs b/src/Paramore.Darker/Logging/Handlers/StreamQueryLoggingDecorator.cs new file mode 100644 index 00000000..4de33c73 --- /dev/null +++ b/src/Paramore.Darker/Logging/Handlers/StreamQueryLoggingDecorator.cs @@ -0,0 +1,82 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Diagnostics; +#if NET8_0_OR_GREATER +using System.Diagnostics.CodeAnalysis; +#endif +using System.Text.Json; +using System.Runtime.CompilerServices; +using System.Threading; +using Microsoft.Extensions.Logging; + +namespace Paramore.Darker.Logging.Handlers +{ + /// + /// A stream decorator that logs stream start (with serialised query body), yields each item + /// unchanged, and logs completion with item count and elapsed duration. + /// Exceptions during enumeration are handled by T019's decorator (this decorator does not swallow). + /// + public class StreamQueryLoggingDecorator : IStreamQueryHandlerDecorator + where TQuery : IStreamQuery + { + private static readonly ILogger Logger = ApplicationLogging.CreateLogger>(); + + public IQueryContext Context { get; set; } + + public void InitializeFromAttributeParams(object[] attributeParams) + { + // nothing to do + } + + public async IAsyncEnumerable Execute( + TQuery query, + Func> next, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + var sw = Stopwatch.StartNew(); + var queryName = query.GetType().Name; + var itemCount = 0; + + Logger.LogInformation("Executing stream query {QueryName}: {Query}", queryName, Serialize(query)); + + try + { + await foreach (var item in next(query, cancellationToken)) + { + itemCount++; + yield return item; + } + } + finally + { + Logger.LogInformation( + "Stream execution of query {QueryName} completed; {ItemCount} items in {Elapsed}ms", + queryName, itemCount, sw.Elapsed.TotalMilliseconds); + } + } + +#if NET8_0_OR_GREATER + [UnconditionalSuppressMessage( + "Trimming", "IL2026:RequiresUnreferencedCodeAttribute", + Justification = "Same as QueryLoggingDecoratorAsync — consumers supply their own JsonSerializerOptions.")] + [UnconditionalSuppressMessage( + "AOT", "IL3050:RequiresDynamicCodeAttribute", + Justification = "Same as QueryLoggingDecoratorAsync — source-gen TypeInfoResolver is the supported escape hatch.")] +#endif + private string Serialize(T value) => + JsonSerializer.Serialize(value, value.GetType(), QueryLoggingJsonOptions.Options); + } +} diff --git a/test/Paramore.Darker.Core.Tests/Logging/When_stream_query_logged_should_wrap_lifecycle_with_item_count.cs b/test/Paramore.Darker.Core.Tests/Logging/When_stream_query_logged_should_wrap_lifecycle_with_item_count.cs new file mode 100644 index 00000000..9136a2ae --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Logging/When_stream_query_logged_should_wrap_lifecycle_with_item_count.cs @@ -0,0 +1,150 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Linq; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Logging; +using Paramore.Darker.Logging.Handlers; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests.Logging +{ + [Collection("QueryLoggingJsonOptions")] + public class When_stream_query_logged_should_wrap_lifecycle_with_item_count + { + private readonly LoggerCaptureFixture _logs; + + public When_stream_query_logged_should_wrap_lifecycle_with_item_count(LoggerCaptureFixture logs) + { + _logs = logs; + } + + [Fact] + public async Task When_stream_query_executes_with_logging_decorator_should_log_start_and_completion_with_item_count() + { + // Arrange — throwaway options so the serialize-lock never touches the shared default (C5) + var original = QueryLoggingJsonOptions.Options; + _logs.Clear(); + try + { + QueryLoggingJsonOptions.Options = new JsonSerializerOptions + { + ReferenceHandler = ReferenceHandler.IgnoreCycles, + WriteIndented = false + }; + + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var handlerFactory = new SimpleHandlerFactory(_ => new LoggedMultiItemStreamHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory( + _ => new StreamQueryLoggingDecorator, string>()); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + new QueryHandlerRegistry(), handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + var processor = new QueryProcessor(config, new InMemoryQueryContextFactory()); + + // Act — enumerate the full stream + var items = new List(); + await foreach (var item in processor.ExecuteStream(new MultiItemStreamQuery())) + items.Add(item); + + // Assert — all items yielded unchanged (decorator is pass-through) + items.ShouldBe(MultiItemStreamHandler.Items, ignoreOrder: false); + + // Assert — start log emitted before any items + var start = _logs.CapturedLogs.FirstOrDefault( + e => e.MessageTemplate == "Executing stream query {QueryName}: {Query}"); + start.ShouldNotBeNull("logging decorator must emit a start log with the query name and body"); + Argument(start, "QueryName").ShouldBe(nameof(MultiItemStreamQuery)); + + // Assert — completion log emitted with item count and elapsed + var completion = _logs.CapturedLogs.FirstOrDefault( + e => e.MessageTemplate == "Stream execution of query {QueryName} completed; {ItemCount} items in {Elapsed}ms"); + completion.ShouldNotBeNull("logging decorator must emit a completion log with item count and elapsed duration"); + Argument(completion, "QueryName").ShouldBe(nameof(MultiItemStreamQuery)); + ((int)Argument(completion, "ItemCount")).ShouldBe(MultiItemStreamHandler.Items.Length, + "item count in the completion log must equal the number of items yielded"); + } + finally + { + QueryLoggingJsonOptions.Options = original; + } + } + + [Fact] + public async Task When_stream_query_with_logging_decorator_items_are_not_buffered() + { + // Arrange — throwaway options so the serialize-lock never touches the shared default (C5) + var original = QueryLoggingJsonOptions.Options; + _logs.Clear(); + try + { + QueryLoggingJsonOptions.Options = new JsonSerializerOptions + { + ReferenceHandler = ReferenceHandler.IgnoreCycles, + WriteIndented = false + }; + + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var handlerFactory = new SimpleHandlerFactory(_ => new LoggedMultiItemStreamHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory( + _ => new StreamQueryLoggingDecorator, string>()); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + new QueryHandlerRegistry(), handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + var processor = new QueryProcessor(config, new InMemoryQueryContextFactory()); + + // Act — break after the first item; the stream is cold (lazy) so completion log includes only 1 + var items = new List(); + await foreach (var item in processor.ExecuteStream(new MultiItemStreamQuery())) + { + items.Add(item); + break; + } + + // Assert — only one item received (early break worked) + items.Count.ShouldBe(1); + + // Assert — completion log reflects the actual items observed, not the total buffered count + var completion = _logs.CapturedLogs.FirstOrDefault( + e => e.MessageTemplate == "Stream execution of query {QueryName} completed; {ItemCount} items in {Elapsed}ms"); + completion.ShouldNotBeNull("completion log must be emitted even on early break (finally block)"); + ((int)Argument(completion, "ItemCount")).ShouldBe(1, + "item count must be 1 because the consumer broke after the first item — the decorator does not buffer"); + } + finally + { + QueryLoggingJsonOptions.Options = original; + } + } + + private static object Argument(CapturedLogEntry entry, string key) + => entry.StructuredArguments.Single(kvp => kvp.Key == key).Value; + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/LoggedMultiItemStreamHandler.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/LoggedMultiItemStreamHandler.cs new file mode 100644 index 00000000..14064f9e --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/LoggedMultiItemStreamHandler.cs @@ -0,0 +1,43 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; +using Paramore.Darker.Logging.Attributes; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A stream handler decorated with for use in + /// logging lifecycle tests. + /// + internal sealed class LoggedMultiItemStreamHandler : IStreamQueryHandler + { + public IQueryContext Context { get; set; } + + [StreamQueryLogging(1)] + public async IAsyncEnumerable ExecuteAsync( + MultiItemStreamQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + foreach (var item in MultiItemStreamHandler.Items) + { + cancellationToken.ThrowIfCancellationRequested(); + yield return item; + } + await Task.CompletedTask; + } + } +} From 263d51a30ccac322db438ec73bf9815b994a9154 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Sun, 12 Jul 2026 22:49:45 +0100 Subject: [PATCH 22/37] feat: Record enumeration faults in stream logging decorator at Error level (T019) Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 2 +- .../Handlers/StreamQueryLoggingDecorator.cs | 33 +++++- ...d_record_exception_in_logging_decorator.cs | 107 ++++++++++++++++++ .../LoggedFaultingStreamHandler.cs | 42 +++++++ 4 files changed, 178 insertions(+), 6 deletions(-) create mode 100644 test/Paramore.Darker.Core.Tests/Logging/When_stream_faults_during_enumeration_should_record_exception_in_logging_decorator.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/LoggedFaultingStreamHandler.cs diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index 5118031c..f91917be 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -287,7 +287,7 @@ over `next`, modelled on `QueryLoggingDecoratorAsync`, plus a `StreamQueryHandlerAttribute`-derived logging attribute and a builder-extension registration -- [ ] **TEST + IMPLEMENT: T019 — Stream logging/telemetry records enumeration faults** +- [x] **TEST + IMPLEMENT: T019 — Stream logging/telemetry records enumeration faults** - **USE COMMAND**: `/test-first when a stream faults during enumeration should record the exception in the logging decorator and rethrow` - Test location: `test/Paramore.Darker.Core.Tests` (Logging folder) - Test file: `When_stream_faults_during_enumeration_should_record_exception_in_logging_decorator.cs` diff --git a/src/Paramore.Darker/Logging/Handlers/StreamQueryLoggingDecorator.cs b/src/Paramore.Darker/Logging/Handlers/StreamQueryLoggingDecorator.cs index 4de33c73..2a15a3fc 100644 --- a/src/Paramore.Darker/Logging/Handlers/StreamQueryLoggingDecorator.cs +++ b/src/Paramore.Darker/Logging/Handlers/StreamQueryLoggingDecorator.cs @@ -17,8 +17,9 @@ #if NET8_0_OR_GREATER using System.Diagnostics.CodeAnalysis; #endif -using System.Text.Json; using System.Runtime.CompilerServices; +using System.Runtime.ExceptionServices; +using System.Text.Json; using System.Threading; using Microsoft.Extensions.Logging; @@ -26,8 +27,8 @@ namespace Paramore.Darker.Logging.Handlers { /// /// A stream decorator that logs stream start (with serialised query body), yields each item - /// unchanged, and logs completion with item count and elapsed duration. - /// Exceptions during enumeration are handled by T019's decorator (this decorator does not swallow). + /// unchanged, records enumeration faults at Error level, and logs completion with item count + /// and elapsed duration. /// public class StreamQueryLoggingDecorator : IStreamQueryHandlerDecorator where TQuery : IStreamQuery @@ -52,20 +53,42 @@ public async IAsyncEnumerable Execute( Logger.LogInformation("Executing stream query {QueryName}: {Query}", queryName, Serialize(query)); + // Manual iteration so we can catch MoveNextAsync faults without a yield inside try/catch + // (C# does not allow yield return in a try block that has a catch clause). + var enumerator = next(query, cancellationToken).GetAsyncEnumerator(cancellationToken); + ExceptionDispatchInfo fault = null; try { - await foreach (var item in next(query, cancellationToken)) + while (true) { + bool moved; + try + { + moved = await enumerator.MoveNextAsync(); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + Logger.LogError(ex, + "Stream execution of query {QueryName} faulted after {ItemCount} items", + queryName, itemCount); + fault = ExceptionDispatchInfo.Capture(ex); + break; + } + + if (!moved) break; itemCount++; - yield return item; + yield return enumerator.Current; } } finally { + await enumerator.DisposeAsync(); Logger.LogInformation( "Stream execution of query {QueryName} completed; {ItemCount} items in {Elapsed}ms", queryName, itemCount, sw.Elapsed.TotalMilliseconds); } + + fault?.Throw(); } #if NET8_0_OR_GREATER diff --git a/test/Paramore.Darker.Core.Tests/Logging/When_stream_faults_during_enumeration_should_record_exception_in_logging_decorator.cs b/test/Paramore.Darker.Core.Tests/Logging/When_stream_faults_during_enumeration_should_record_exception_in_logging_decorator.cs new file mode 100644 index 00000000..3c668396 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Logging/When_stream_faults_during_enumeration_should_record_exception_in_logging_decorator.cs @@ -0,0 +1,107 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; +using Microsoft.Extensions.Logging; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Logging; +using Paramore.Darker.Logging.Handlers; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests.Logging +{ + [Collection("QueryLoggingJsonOptions")] + public class When_stream_faults_during_enumeration_should_record_exception_in_logging_decorator + { + private readonly LoggerCaptureFixture _logs; + + public When_stream_faults_during_enumeration_should_record_exception_in_logging_decorator(LoggerCaptureFixture logs) + { + _logs = logs; + } + + [Fact] + public async Task When_stream_handler_throws_mid_enumeration_should_log_error_and_propagate_exception() + { + // Arrange — throwaway options so the serialize-lock never touches the shared default (C5) + var original = QueryLoggingJsonOptions.Options; + _logs.Clear(); + try + { + QueryLoggingJsonOptions.Options = new JsonSerializerOptions + { + ReferenceHandler = ReferenceHandler.IgnoreCycles, + WriteIndented = false + }; + + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var handlerFactory = new SimpleHandlerFactory(_ => new LoggedFaultingStreamHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory( + _ => new StreamQueryLoggingDecorator, string>()); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + new QueryHandlerRegistry(), handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + var processor = new QueryProcessor(config, new InMemoryQueryContextFactory()); + + // Act — enumerate; expect fault after the items yielded before the throw + var items = new List(); + Exception caughtException = null; + try + { + await foreach (var item in processor.ExecuteStream(new FaultingStreamQuery())) + items.Add(item); + } + catch (Exception ex) + { + caughtException = ex; + } + + // Assert — items yielded before the fault were received by the caller + items.Count.ShouldBe(FaultingStreamHandler.ItemsBeforeFault, + "items produced before the fault must reach the caller"); + + // Assert — original exception propagated (not swallowed by the decorator) + caughtException.ShouldNotBeNull("the logging decorator must not swallow exceptions"); + caughtException.ShouldBeOfType(); + caughtException.Message.ShouldBe(FaultingStreamHandler.ExceptionMessage); + + // Assert — error-level log was recorded with the exception + var errorLog = _logs.CapturedLogs.FirstOrDefault(e => e.LogLevel == LogLevel.Error); + errorLog.ShouldNotBeNull("logging decorator must emit an Error log when the stream faults"); + errorLog.Exception.ShouldNotBeNull(); + errorLog.Exception.Message.ShouldBe(FaultingStreamHandler.ExceptionMessage, + "the logged exception must be the original fault from the handler"); + Argument(errorLog, "QueryName").ShouldBe(nameof(FaultingStreamQuery)); + } + finally + { + QueryLoggingJsonOptions.Options = original; + } + } + + private static object Argument(CapturedLogEntry entry, string key) + => entry.StructuredArguments.Single(kvp => kvp.Key == key).Value; + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/LoggedFaultingStreamHandler.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/LoggedFaultingStreamHandler.cs new file mode 100644 index 00000000..ff82694e --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/LoggedFaultingStreamHandler.cs @@ -0,0 +1,42 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using Paramore.Darker.Logging.Attributes; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A stream handler with that yields two items then + /// throws, allowing tests to verify that the logging decorator records enumeration faults. + /// + internal sealed class LoggedFaultingStreamHandler : IStreamQueryHandler + { + public IQueryContext Context { get; set; } + +#pragma warning disable CS1998 + [StreamQueryLogging(1)] + public async IAsyncEnumerable ExecuteAsync( + FaultingStreamQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) +#pragma warning restore CS1998 + { + yield return "item-1"; + yield return "item-2"; + throw new InvalidOperationException(FaultingStreamHandler.ExceptionMessage); + } + } +} From 4187232d915f789e70a875e29e28c29a8e9be13b Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 05:47:11 +0100 Subject: [PATCH 23/37] test: Document that BuildStream writes a span event per pipeline step (T020) Co-Authored-By: Claude Sonnet 4.6 --- .../TestDoubles/StreamSpanEventHandler.cs | 39 +++++ ...e_with_span_should_write_event_per_step.cs | 142 ++++++++++++++++++ 2 files changed, 181 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/StreamSpanEventHandler.cs create mode 100644 test/Paramore.Darker.Core.Tests/When_building_stream_pipeline_with_span_should_write_event_per_step.cs diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/StreamSpanEventHandler.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamSpanEventHandler.cs new file mode 100644 index 00000000..87bb8dec --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/StreamSpanEventHandler.cs @@ -0,0 +1,39 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A minimal stream handler decorated with one used to + /// verify that PipelineBuilder.BuildStream writes a step event for each pipeline step + /// (decorator + sink) when a span is present on the context. + /// + internal sealed class StreamSpanEventHandler : IStreamQueryHandler + { + public IQueryContext Context { get; set; } + + [StreamStepEvent(step: 1)] + public async IAsyncEnumerable ExecuteAsync( + StreamTestQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + yield return "item"; + await Task.CompletedTask; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/When_building_stream_pipeline_with_span_should_write_event_per_step.cs b/test/Paramore.Darker.Core.Tests/When_building_stream_pipeline_with_span_should_write_event_per_step.cs new file mode 100644 index 00000000..cd8508e7 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_building_stream_pipeline_with_span_should_write_event_per_step.cs @@ -0,0 +1,142 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Diagnostics; +using System.Linq; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Observability; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + /// + /// Verifies that PipelineBuilder.BuildStream weaves one WriteQueryEvent per step + /// (decorator + sink handler) into the stream pipeline when a span is present on the context, + /// and that no events are added when the context carries no span (zero-overhead pass-through). + /// + [Collection("DarkerActivitySource")] + public class PipelineBuilderStreamStepEventTests + { + private static ActivityListener CreateListener(List completed) + { + var listener = new ActivityListener + { + ShouldListenTo = s => s.Name == DarkerSemanticConventions.SourceName, + Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllData, + SampleUsingParentId = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllData, + ActivityStopped = a => completed.Add(a), + }; + ActivitySource.AddActivityListener(listener); + return listener; + } + + private static QueryProcessor CreateProcessorWithTracer(IAmADarkerTracer tracer, List enteredSteps) + { + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var syncRegistry = new QueryHandlerRegistry(); + var handlerFactory = new SimpleHandlerFactory(_ => new StreamSpanEventHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => + new StreamStepEventDecorator, string>(enteredSteps)); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + + return new QueryProcessor( + config, + new InMemoryQueryContextFactory(), + tracer: tracer, + instrumentationOptions: InstrumentationOptions.QueryInformation); + } + + [Fact] + public async Task When_building_stream_pipeline_with_span_should_write_event_per_step() + { + // Arrange + var completed = new List(); + using var listener = CreateListener(completed); + using var tracer = new DarkerTracer(); + var enteredSteps = new List(); + var processor = CreateProcessorWithTracer(tracer, enteredSteps); + var query = new StreamTestQuery(); + + // Act — must enumerate to trigger span events + var results = new List(); + await foreach (var item in processor.ExecuteStream(query)) + results.Add(item); + + // Assert + results.Count.ShouldBe(1); + completed.Count.ShouldBe(1); + var span = completed[0]; + + var events = span.Events.ToList(); + events.Count.ShouldBe(2); + + // First event: decorator (outermost in pipeline — executes before calling next) + var decoratorEvent = events[0]; + var decoratorTypeName = typeof(StreamStepEventDecorator<,>) + .MakeGenericType(typeof(IStreamQuery), typeof(string)) + .Name; + decoratorEvent.Name.ShouldBe(decoratorTypeName); + var decoratorTags = decoratorEvent.Tags.ToDictionary(kvp => kvp.Key, kvp => kvp.Value); + decoratorTags[DarkerSemanticConventions.HandlerType].ShouldBe("async"); + decoratorTags[DarkerSemanticConventions.IsSink].ShouldBe(false); + + // Second event: handler (innermost = sink — executes after decorator calls next) + var handlerEvent = events[1]; + handlerEvent.Name.ShouldBe(typeof(StreamSpanEventHandler).Name); + var handlerTags = handlerEvent.Tags.ToDictionary(kvp => kvp.Key, kvp => kvp.Value); + handlerTags[DarkerSemanticConventions.HandlerType].ShouldBe("async"); + handlerTags[DarkerSemanticConventions.IsSink].ShouldBe(true); + } + + [Fact] + public async Task When_building_stream_pipeline_without_span_should_not_add_events_and_run_cleanly() + { + // Arrange — no tracer so queryContext.Span is null; WriteQueryEvent must be a no-op + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var syncRegistry = new QueryHandlerRegistry(); + var enteredSteps = new List(); + var handlerFactory = new SimpleHandlerFactory(_ => new StreamSpanEventHandler()); + var decoratorFactory = new SimpleHandlerDecoratorFactory(_ => + new StreamStepEventDecorator, string>(enteredSteps)); + var decoratorRegistry = new InMemoryDecoratorRegistry(); + var config = new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + new QueryHandlerRegistryAsync(), handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry); + var processor = new QueryProcessor(config, new InMemoryQueryContextFactory()); + + var query = new StreamTestQuery(); + + // Act — pipeline must execute correctly with no span present + var results = new List(); + await foreach (var item in processor.ExecuteStream(query)) + results.Add(item); + + // Assert — result returned normally; no listener active so no events to inspect + results.Count.ShouldBe(1); + results[0].ShouldBe("item"); + } + } +} From a30d4e4e34a8d5595199fe3209b3e8fe0e648b70 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 05:51:24 +0100 Subject: [PATCH 24/37] refactor: Add UseResiliencePipelineStreamAttribute and placeholder handler stub (T021) Co-Authored-By: Claude Sonnet 4.6 --- .../UseResiliencePipelineStreamAttribute.cs | 67 ++++++++++++++++++ .../UseResiliencePipelineStreamHandler.cs | 70 +++++++++++++++++++ 2 files changed, 137 insertions(+) create mode 100644 src/Paramore.Darker/Policies/Attributes/UseResiliencePipelineStreamAttribute.cs create mode 100644 src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineStreamHandler.cs diff --git a/src/Paramore.Darker/Policies/Attributes/UseResiliencePipelineStreamAttribute.cs b/src/Paramore.Darker/Policies/Attributes/UseResiliencePipelineStreamAttribute.cs new file mode 100644 index 00000000..340c014e --- /dev/null +++ b/src/Paramore.Darker/Policies/Attributes/UseResiliencePipelineStreamAttribute.cs @@ -0,0 +1,67 @@ +#region Licence +/* The MIT License (MIT) +Copyright © 2025 Ian Cooper + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. */ +#endregion + +using System; +using Paramore.Darker.Policies.Handlers; + +namespace Paramore.Darker.Policies.Attributes +{ + /// + /// Applies a Polly V8 to a streaming query handler. + /// The named pipeline covers only stream establishment (calling next and pulling the first + /// item); faults after the first item has been yielded are not retried. Untyped pipelines only — + /// there is no useTypePipeline overload for streams. + /// + /// + /// Use this attribute on a stream handler's ExecuteAsync method. The resilience pipeline + /// is resolved at execution time from . + /// + [AttributeUsage(AttributeTargets.Method)] + public sealed class UseResiliencePipelineStreamAttribute : StreamQueryHandlerAttribute + { + private readonly string _policy; + + /// + /// Initializes a new instance of the class. + /// + /// The ordinal position of this decorator in the pipeline; higher executes first. + /// The registry key of the untyped resilience pipeline to apply. + public UseResiliencePipelineStreamAttribute(int step, string policy) + : base(step) + { + _policy = policy; + } + + /// + /// Returns the parameters passed to the decorator: the pipeline key. + /// + /// An array containing the policy key. + public override object[] GetAttributeParams() => new object[] { _policy }; + + /// + /// Returns the open generic type of the stream decorator that applies the resilience pipeline. + /// + /// The open generic type. + public override Type GetDecoratorType() => typeof(UseResiliencePipelineStreamHandler<,>); + } +} diff --git a/src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineStreamHandler.cs b/src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineStreamHandler.cs new file mode 100644 index 00000000..55664a73 --- /dev/null +++ b/src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineStreamHandler.cs @@ -0,0 +1,70 @@ +#region Licence +/* The MIT License (MIT) +Copyright © 2025 Ian Cooper + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. */ +#endregion + +using System; +using System.Collections.Generic; +using System.Threading; + +namespace Paramore.Darker.Policies.Handlers +{ + /// + /// Stream decorator that executes the establishment of a stream query through a named Polly V8 + /// resolved from the query context provider. + /// Resilience covers only stream establishment (the first MoveNextAsync); faults after + /// the first item has been yielded propagate directly to the caller without retry. + /// + /// The stream query type. + /// The item type produced by the stream. + public class UseResiliencePipelineStreamHandler : IStreamQueryHandlerDecorator + where TQuery : IStreamQuery + { + /// + /// The ambient query context, supplying the resilience pipeline provider and context. + /// + public IQueryContext Context { get; set; } + + /// + /// Initializes the decorator from the attribute parameters (pipeline key). + /// + /// The parameters supplied by the attribute. + public void InitializeFromAttributeParams(object[] attributeParams) + { + throw new NotImplementedException("UseResiliencePipelineStreamHandler is not yet implemented. It will be completed in a subsequent task."); + } + + /// + /// Executes the stream query through the resolved resilience pipeline. + /// + /// The stream query to execute. + /// The next step in the stream pipeline. + /// A token to cancel the operation. + /// An async enumerable of result items. + public IAsyncEnumerable Execute( + TQuery query, + Func> next, + CancellationToken cancellationToken = default) + { + throw new NotImplementedException("UseResiliencePipelineStreamHandler is not yet implemented. It will be completed in a subsequent task."); + } + } +} From ce653fd3cb21da3c12ec71cb446801fc8f8aec64 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 06:15:03 +0100 Subject: [PATCH 25/37] =?UTF-8?q?feat:=20Implement=20UseResiliencePipeline?= =?UTF-8?q?StreamHandler=20happy=20path=20=E2=80=94=20establishment=20succ?= =?UTF-8?q?eeds=20yields=20all=20items=20(T022)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Sonnet 4.6 --- .../UseResiliencePipelineStreamHandler.cs | 73 ++++++++++++++++--- ...ishment_succeeds_should_yield_all_items.cs | 61 ++++++++++++++++ 2 files changed, 123 insertions(+), 11 deletions(-) create mode 100644 test/Paramore.Darker.Core.Tests/Decorators/When_stream_resilience_establishment_succeeds_should_yield_all_items.cs diff --git a/src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineStreamHandler.cs b/src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineStreamHandler.cs index 55664a73..3d94a047 100644 --- a/src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineStreamHandler.cs +++ b/src/Paramore.Darker/Policies/Handlers/UseResiliencePipelineStreamHandler.cs @@ -23,48 +23,99 @@ THE SOFTWARE. */ using System; using System.Collections.Generic; +using System.Runtime.CompilerServices; using System.Threading; +using System.Threading.Tasks; +using Paramore.Darker.Exceptions; +using Polly; namespace Paramore.Darker.Policies.Handlers { /// /// Stream decorator that executes the establishment of a stream query through a named Polly V8 - /// resolved from the query context provider. - /// Resilience covers only stream establishment (the first MoveNextAsync); faults after - /// the first item has been yielded propagate directly to the caller without retry. + /// resolved from the query context provider. /// + /// + /// Resilience covers only stream establishment: the call to next and the first + /// MoveNextAsync run inside the pipeline boundary. Once the first item has been yielded the + /// pipeline has already exited; subsequent MoveNextAsync faults propagate directly to the + /// caller without retry. This gives a well-defined "no duplicate emission" guarantee — a retry + /// before the first item always starts a fresh . + /// + /// Only the untyped is supported (no useTypePipeline + /// overload): a ResiliencePipeline<TResult> cannot wrap the + /// (IAsyncEnumerator<TResult>, bool) tuple returned by the establishment callback. + /// + /// /// The stream query type. /// The item type produced by the stream. public class UseResiliencePipelineStreamHandler : IStreamQueryHandlerDecorator where TQuery : IStreamQuery { + private string _policy; + /// - /// The ambient query context, supplying the resilience pipeline provider and context. + /// The ambient query context, supplying the resilience pipeline provider and resilience context. /// public IQueryContext Context { get; set; } /// - /// Initializes the decorator from the attribute parameters (pipeline key). + /// Initializes the decorator from the attribute parameters. /// - /// The parameters supplied by the attribute. + /// + /// A single-element array whose first element is the resilience pipeline key (a ). + /// + /// + /// Thrown when no resilience pipeline provider is configured on the query context, or when the + /// named pipeline does not exist in the registry. + /// public void InitializeFromAttributeParams(object[] attributeParams) { - throw new NotImplementedException("UseResiliencePipelineStreamHandler is not yet implemented. It will be completed in a subsequent task."); + _policy = (string)attributeParams[0]; + + var provider = Context.ResiliencePipeline ?? throw new ConfigurationException( + "No resilience pipeline provider is configured. Set a resilience pipeline registry on the query context or pass one to the QueryProcessor constructor."); + + if (!provider.TryGetPipeline(_policy, out _)) + throw new ConfigurationException($"Resilience pipeline does not exist in the registry: {_policy}"); } /// - /// Executes the stream query through the resolved resilience pipeline. + /// Executes the stream query through the resolved resilience pipeline, yielding items only + /// after establishment succeeds. /// /// The stream query to execute. /// The next step in the stream pipeline. /// A token to cancel the operation. /// An async enumerable of result items. - public IAsyncEnumerable Execute( + public async IAsyncEnumerable Execute( TQuery query, Func> next, - CancellationToken cancellationToken = default) + [EnumeratorCancellation] CancellationToken cancellationToken = default) { - throw new NotImplementedException("UseResiliencePipelineStreamHandler is not yet implemented. It will be completed in a subsequent task."); + var pipeline = Context.ResiliencePipeline.GetPipeline(_policy); + + // Establish the stream + first pull inside the pipeline boundary. + // On failure the enumerator is disposed before rethrowing so retried attempts don't leak. + async ValueTask<(IAsyncEnumerator enumerator, bool moved)> Establish(CancellationToken ct) + { + var e = next(query, ct).GetAsyncEnumerator(ct); + try { return (e, await e.MoveNextAsync().ConfigureAwait(false)); } + catch { await e.DisposeAsync().ConfigureAwait(false); throw; } + } + + // Mirror the context-vs-token branching from UseResiliencePipelineHandlerAsync. + var resilienceContext = Context.ResilienceContext; + var (enumerator, moved) = resilienceContext != null + ? await pipeline.ExecuteAsync(ctx => Establish(ctx.CancellationToken), resilienceContext).ConfigureAwait(false) + : await pipeline.ExecuteAsync(ct => Establish(ct), cancellationToken).ConfigureAwait(false); + + await using (enumerator.ConfigureAwait(false)) + { + if (!moved) yield break; + do { yield return enumerator.Current; } + while (await enumerator.MoveNextAsync().ConfigureAwait(false)); + } } } } diff --git a/test/Paramore.Darker.Core.Tests/Decorators/When_stream_resilience_establishment_succeeds_should_yield_all_items.cs b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_resilience_establishment_succeeds_should_yield_all_items.cs new file mode 100644 index 00000000..551ccd38 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_resilience_establishment_succeeds_should_yield_all_items.cs @@ -0,0 +1,61 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Policies.Handlers; +using Polly; +using Polly.Registry; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests.Decorators +{ + public class StreamResilienceHappyPathTests + { + [Fact] + public async Task When_stream_uses_resilience_pipeline_and_establishment_succeeds_should_yield_all_items() + { + // Arrange — a no-op pipeline (long timeout, never fires) that lets everything succeed + const string pipelineName = "NoOp"; + var registry = new ResiliencePipelineRegistry(); + registry.TryAddBuilder(pipelineName, (builder, _) => + builder.AddTimeout(TimeSpan.FromMinutes(1))); + + var context = new QueryContext { ResiliencePipeline = registry }; + var decorator = new UseResiliencePipelineStreamHandler + { + Context = context + }; + decorator.InitializeFromAttributeParams(new object[] { pipelineName }); + + var query = new MultiItemStreamQuery(); + var handler = new MultiItemStreamHandler { Context = context }; + + // Act — establishment succeeds: pipeline wraps first MoveNextAsync then yields all items + var results = new List(); + await foreach (var item in decorator.Execute( + query, + (q, ct) => handler.ExecuteAsync(q, ct), + default)) + { + results.Add(item); + } + + // Assert — all handler items pass through the decorator unchanged + results.ShouldBe(MultiItemStreamHandler.Items); + } + } +} From c47f430ab7308d1a0f63f68320470652500d3db2 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 06:19:07 +0100 Subject: [PATCH 26/37] test: Document that retry before first item uses fresh stream with no duplicate emission (T023) Co-Authored-By: Claude Sonnet 4.6 --- ...st_item_should_retry_without_duplicates.cs | 69 +++++++++++++++++++ .../TransientlyFailingStreamHandler.cs | 59 ++++++++++++++++ 2 files changed, 128 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_fails_before_first_item_should_retry_without_duplicates.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/TransientlyFailingStreamHandler.cs diff --git a/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_fails_before_first_item_should_retry_without_duplicates.cs b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_fails_before_first_item_should_retry_without_duplicates.cs new file mode 100644 index 00000000..7d79e25a --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_fails_before_first_item_should_retry_without_duplicates.cs @@ -0,0 +1,69 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Policies.Handlers; +using Polly; +using Polly.Registry; +using Polly.Retry; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests.Decorators +{ + public class StreamResilienceRetryBeforeFirstItemTests + { + [Fact] + public async Task When_establishment_fails_before_first_item_should_retry_fresh_stream_with_no_duplicate_emission() + { + // Arrange — retry pipeline that handles the transient InvalidOperationException + const string pipelineName = "Retry"; + var registry = new ResiliencePipelineRegistry(); + registry.TryAddBuilder(pipelineName, (builder, _) => + builder.AddRetry(new RetryStrategyOptions + { + ShouldHandle = new PredicateBuilder().Handle(), + MaxRetryAttempts = 3, + Delay = TimeSpan.Zero + })); + + var context = new QueryContext { ResiliencePipeline = registry }; + var decorator = new UseResiliencePipelineStreamHandler + { + Context = context + }; + decorator.InitializeFromAttributeParams(new object[] { pipelineName }); + + // Handler fails on its first attempt (throws before yielding any item), then succeeds + var handler = new TransientlyFailingStreamHandler(failuresBeforeSuccess: 1); + var query = new MultiItemStreamQuery(); + + // Act — the Establish callback disposes the failed enumerator and retries with a fresh one + var results = new List(); + await foreach (var item in decorator.Execute( + query, + (q, ct) => handler.ExecuteAsync(q, ct), + default)) + { + results.Add(item); + } + + // Assert — full sequence emitted exactly once (retry starts a fresh enumerable, no duplicates) + results.ShouldBe(MultiItemStreamHandler.Items); + handler.Calls.ShouldBe(2, "handler body ran twice: once for the failure, once for the successful retry"); + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/TransientlyFailingStreamHandler.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/TransientlyFailingStreamHandler.cs new file mode 100644 index 00000000..459ab7d3 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/TransientlyFailingStreamHandler.cs @@ -0,0 +1,59 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A counting stream test double that throws on its + /// first failuresBeforeSuccess invocations (before yielding any item) and yields all + /// on subsequent invocations. Used to prove a retry + /// resilience pipeline retries a transient stream establishment failure to success with no + /// duplicate item emission. + /// + internal sealed class TransientlyFailingStreamHandler : IStreamQueryHandler + { + private readonly int _failuresBeforeSuccess; + + public TransientlyFailingStreamHandler(int failuresBeforeSuccess = 1) + { + _failuresBeforeSuccess = failuresBeforeSuccess; + } + + /// Gets the total number of times ExecuteAsync has been entered. + public int Calls { get; private set; } + + public IQueryContext Context { get; set; } + + public async IAsyncEnumerable ExecuteAsync( + MultiItemStreamQuery query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + Calls++; + if (Calls <= _failuresBeforeSuccess) + throw new InvalidOperationException("transient stream failure"); + + foreach (var item in MultiItemStreamHandler.Items) + { + cancellationToken.ThrowIfCancellationRequested(); + yield return item; + } + await Task.CompletedTask; + } + } +} From 334a5c979cadac266a2ed062b5fe3a471abe1751 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 06:20:41 +0100 Subject: [PATCH 27/37] test: Document that post-first-item faults propagate without retry (T024) Co-Authored-By: Claude Sonnet 4.6 --- ...aults_after_first_item_should_not_retry.cs | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/Decorators/When_stream_faults_after_first_item_should_not_retry.cs diff --git a/test/Paramore.Darker.Core.Tests/Decorators/When_stream_faults_after_first_item_should_not_retry.cs b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_faults_after_first_item_should_not_retry.cs new file mode 100644 index 00000000..97fe6342 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_faults_after_first_item_should_not_retry.cs @@ -0,0 +1,76 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Policies.Handlers; +using Polly; +using Polly.Registry; +using Polly.Retry; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests.Decorators +{ + public class StreamResiliencePostFirstItemFaultTests + { + [Fact] + public async Task When_stream_faults_after_first_item_should_propagate_without_retry_and_without_re_emitting_items() + { + // Arrange — retry pipeline configured to handle InvalidOperationException (but it must NOT + // retry faults that occur after the first item has already been yielded) + const string pipelineName = "Retry"; + var registry = new ResiliencePipelineRegistry(); + registry.TryAddBuilder(pipelineName, (builder, _) => + builder.AddRetry(new RetryStrategyOptions + { + ShouldHandle = new PredicateBuilder().Handle(), + MaxRetryAttempts = 3, + Delay = TimeSpan.Zero + })); + + var context = new QueryContext { ResiliencePipeline = registry }; + var decorator = new UseResiliencePipelineStreamHandler + { + Context = context + }; + decorator.InitializeFromAttributeParams(new object[] { pipelineName }); + + // FaultingStreamHandler yields ItemsBeforeFault items, then throws InvalidOperationException + var query = new FaultingStreamQuery(); + var handler = new FaultingStreamHandler { Context = context }; + + // Act — collect items until the mid-stream fault propagates + var results = new List(); + var exception = await Should.ThrowAsync(async () => + { + await foreach (var item in decorator.Execute( + query, + (q, ct) => handler.ExecuteAsync(q, ct), + default)) + { + results.Add(item); + } + }); + + // Assert — items before the fault were observed; exception propagated; no re-emission + // (if the pipeline had retried, results would contain the items repeated N times) + results.Count.ShouldBe(FaultingStreamHandler.ItemsBeforeFault, + "only items yielded before the fault should be present — no retry re-emission"); + exception.Message.ShouldBe(FaultingStreamHandler.ExceptionMessage, + "the original exception propagates unwrapped"); + } + } +} From 64502d7c1426dac77678d9a751b22ab8c18650a0 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 06:20:48 +0100 Subject: [PATCH 28/37] chore: Mark T020-T024 complete in tasks.md Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index f91917be..e6c5a3b8 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -298,7 +298,7 @@ - Implementation should: - Wrap the decorator's `await foreach`/`yield` in `try/catch` that records then rethrows -- [ ] **TEST + IMPLEMENT: T020 — `BuildStream` writes a span event per pipeline step** +- [x] **TEST + IMPLEMENT: T020 — `BuildStream` writes a span event per pipeline step** - **USE COMMAND**: `/test-first when building a stream pipeline with a span should write a query event per pipeline step` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_building_stream_pipeline_with_span_should_write_event_per_step.cs` @@ -313,12 +313,12 @@ ## Phase 6 — Resilience for streams (behaviour — highest risk) -- [ ] **STRUCTURAL: T021 — Stream resilience attribute** +- [x] **STRUCTURAL: T021 — Stream resilience attribute** - Add `UseResiliencePipelineStreamAttribute(int step, string policy) : StreamQueryHandlerAttribute` (`src/Paramore.Darker/Policies/Attributes/`) returning the stream resilience decorator type; **no** `useTypePipeline` parameter (ADR §3a — untyped pipeline only). Must compile. -- [ ] **TEST + IMPLEMENT: T022 — Resilience decorator yields items on the happy path** +- [x] **TEST + IMPLEMENT: T022 — Resilience decorator yields items on the happy path** - **USE COMMAND**: `/test-first when a stream uses the resilience pipeline decorator and establishment succeeds should yield all items` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_resilience_establishment_succeeds_should_yield_all_items.cs` @@ -333,7 +333,7 @@ `[EnumeratorCancellation]`, yield only after the pipeline succeeds; reuse the async decorator's `InitializeFromAttributeParams` pipeline-resolution logic -- [ ] **TEST + IMPLEMENT: T023 — Retry before the first item does not duplicate emission** +- [x] **TEST + IMPLEMENT: T023 — Retry before the first item does not duplicate emission** - **USE COMMAND**: `/test-first when establishment fails before the first item should retry a fresh stream with no duplicate emission` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_establishment_fails_before_first_item_should_retry_without_duplicates.cs` @@ -346,7 +346,7 @@ - Rely on first-`MoveNextAsync`-inside-the-pipeline (ADR §3a); no item leaves the decorator until the pipeline succeeds -- [ ] **TEST + IMPLEMENT: T024 — Faults after the first item are not retried** +- [x] **TEST + IMPLEMENT: T024 — Faults after the first item are not retried** - **USE COMMAND**: `/test-first when a stream faults after the first item should propagate without retry and without re-emitting items` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_faults_after_first_item_should_not_retry.cs` From cdcf6408f40a6b591f636b8cdd6f7537b1f4800f Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 07:09:16 +0100 Subject: [PATCH 29/37] test: Document that each failed establishment attempt disposes its enumerator (T025) Co-Authored-By: Claude Sonnet 4.6 --- ...s_should_dispose_each_failed_enumerator.cs | 73 +++++++++++++++++++ .../TestDoubles/DisposalCountingEnumerable.cs | 62 ++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_retries_should_dispose_each_failed_enumerator.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/DisposalCountingEnumerable.cs diff --git a/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_retries_should_dispose_each_failed_enumerator.cs b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_retries_should_dispose_each_failed_enumerator.cs new file mode 100644 index 00000000..5e7bb041 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_retries_should_dispose_each_failed_enumerator.cs @@ -0,0 +1,73 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Policies.Handlers; +using Polly; +using Polly.Registry; +using Polly.Retry; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests.Decorators +{ + public class StreamEstablishmentRetryDisposalTests + { + [Fact] + public async Task When_establishment_retries_N_times_should_dispose_the_enumerator_from_each_failed_attempt() + { + // Arrange — retry pipeline with 2 retries (3 total attempts), all of which fail + const int maxRetryAttempts = 2; + const int expectedDisposals = maxRetryAttempts + 1; // original attempt + retries + const string pipelineName = "Retry"; + + var registry = new ResiliencePipelineRegistry(); + registry.TryAddBuilder(pipelineName, (builder, _) => + builder.AddRetry(new RetryStrategyOptions + { + ShouldHandle = new PredicateBuilder().Handle(), + MaxRetryAttempts = maxRetryAttempts, + Delay = TimeSpan.Zero + })); + + var context = new QueryContext { ResiliencePipeline = registry }; + var decorator = new UseResiliencePipelineStreamHandler + { + Context = context + }; + decorator.InitializeFromAttributeParams(new object[] { pipelineName }); + + // Handler always fails before yielding any item — every establishment attempt throws + var handler = new TransientlyFailingStreamHandler(failuresBeforeSuccess: int.MaxValue); + var query = new MultiItemStreamQuery(); + var disposeCount = 0; + + // Act — all attempts fail; each failed enumerator must be disposed inside Establish's catch + await Should.ThrowAsync(async () => + { + await foreach (var _ in decorator.Execute( + query, + (q, ct) => new DisposalCountingEnumerable(handler.ExecuteAsync(q, ct), () => disposeCount++), + default)) + { + } + }); + + // Assert — one dispose per failed attempt: no enumerator is abandoned on retry + disposeCount.ShouldBe(expectedDisposals, + "each failed establishment attempt must dispose its enumerator to prevent resource leaks"); + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/DisposalCountingEnumerable.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/DisposalCountingEnumerable.cs new file mode 100644 index 00000000..f73f5727 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/DisposalCountingEnumerable.cs @@ -0,0 +1,62 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Collections.Generic; +using System.Threading; +using System.Threading.Tasks; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// Wraps an and invokes a callback each time + /// is called on a created enumerator. + /// Used to assert that every enumerator created during resilience retries is disposed. + /// + internal sealed class DisposalCountingEnumerable : IAsyncEnumerable + { + private readonly IAsyncEnumerable _inner; + private readonly Action _onDispose; + + public DisposalCountingEnumerable(IAsyncEnumerable inner, Action onDispose) + { + _inner = inner; + _onDispose = onDispose; + } + + public IAsyncEnumerator GetAsyncEnumerator(CancellationToken cancellationToken = default) + => new CountingEnumerator(_inner.GetAsyncEnumerator(cancellationToken), _onDispose); + + private sealed class CountingEnumerator : IAsyncEnumerator + { + private readonly IAsyncEnumerator _inner; + private readonly Action _onDispose; + + public CountingEnumerator(IAsyncEnumerator inner, Action onDispose) + { + _inner = inner; + _onDispose = onDispose; + } + + public T Current => _inner.Current; + + public ValueTask MoveNextAsync() => _inner.MoveNextAsync(); + + public async ValueTask DisposeAsync() + { + await _inner.DisposeAsync().ConfigureAwait(false); + _onDispose(); + } + } + } +} From e641de6012ed15ed3a4731bc7ac5f2ea8a9018b2 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 07:09:22 +0100 Subject: [PATCH 30/37] chore: Mark T025 complete in tasks.md Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index e6c5a3b8..d9875891 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -358,7 +358,7 @@ - Confirm the pipeline has exited once the first item is yielded (subsequent `MoveNextAsync` faults propagate) — ADR §3a -- [ ] **TEST + IMPLEMENT: T025 — Each failed establishment attempt disposes its enumerator** +- [x] **TEST + IMPLEMENT: T025 — Each failed establishment attempt disposes its enumerator** - **USE COMMAND**: `/test-first when establishment retries N times should dispose the enumerator from each failed attempt` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_establishment_retries_should_dispose_each_failed_enumerator.cs` From 09e70439eec70a7a24e4d7b9adc20e5ae24b7fdd Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 12:59:34 +0200 Subject: [PATCH 31/37] test: Document that fallback strategy substitutes an alternate stream at establishment (T026) Co-Authored-By: Claude Sonnet 4.6 --- ..._strategy_should_yield_alternate_stream.cs | 100 ++++++++++++++++++ .../EstablishmentFallbackStrategy.cs | 59 +++++++++++ 2 files changed, 159 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_fails_with_fallback_strategy_should_yield_alternate_stream.cs create mode 100644 test/Paramore.Darker.Core.Tests/TestDoubles/EstablishmentFallbackStrategy.cs diff --git a/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_fails_with_fallback_strategy_should_yield_alternate_stream.cs b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_fails_with_fallback_strategy_should_yield_alternate_stream.cs new file mode 100644 index 00000000..dd4bc6aa --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/Decorators/When_stream_establishment_fails_with_fallback_strategy_should_yield_alternate_stream.cs @@ -0,0 +1,100 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; +using Paramore.Darker.Core.Tests.TestDoubles; +using Paramore.Darker.Policies.Handlers; +using Polly; +using Polly.Registry; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests.Decorators +{ + public class StreamFallbackEstablishmentTests + { + [Fact] + public async Task When_establishment_fails_with_fallback_strategy_should_yield_alternate_stream() + { + // Arrange + const string pipelineName = "Fallback"; + var primaryDisposeCount = 0; + var alternateDisposeCount = 0; + + // Primary handler always fails before yielding any item + var primaryHandler = new TransientlyFailingStreamHandler(failuresBeforeSuccess: int.MaxValue); + + // Alternate source: the items the fallback stream should yield + static string[] AlternateItems() => new[] { "fallback-a", "fallback-b" }; + var countingAlternateSource = new DisposalCountingEnumerable( + AsyncSource(AlternateItems()), () => alternateDisposeCount++); + + // Resilience pipeline: when Establish faults, substitute the alternate (enumerator, moved) + // The untyped pipeline boxes TResult to object internally; the factory returns a boxed + // (IAsyncEnumerator, bool) that Polly unboxes for the decorator. + var registry = new ResiliencePipelineRegistry(); + registry.TryAddBuilder(pipelineName, (builder, _) => + builder.AddStrategy(_ => new EstablishmentFallbackStrategy(async ct => + { + var e = countingAlternateSource.GetAsyncEnumerator(ct); + var moved = await e.MoveNextAsync(); + return (object)(e, moved); + }))); + + var context = new QueryContext { ResiliencePipeline = registry }; + var decorator = new UseResiliencePipelineStreamHandler + { + Context = context + }; + decorator.InitializeFromAttributeParams(new object[] { pipelineName }); + + var query = new MultiItemStreamQuery(); + + // Act — primary fails; fallback substitutes the alternate stream at establishment + var results = new List(); + await foreach (var item in decorator.Execute( + query, + (q, ct) => new DisposalCountingEnumerable(primaryHandler.ExecuteAsync(q, ct), () => primaryDisposeCount++), + default)) + { + results.Add(item); + } + + // Assert — alternate stream items yielded (fallback path taken) + results.ShouldBe(AlternateItems()); + + // Primary enumerator disposed by Establish's catch before fallback fired: no primary leak + primaryDisposeCount.ShouldBe(1, + "primary enumerator must be disposed inside Establish's catch before the fallback fires"); + + // Alternate enumerator is the only one owned by the outer await using + alternateDisposeCount.ShouldBe(1, + "the fallback-substituted enumerator is disposed exactly once by the outer await using"); + } + + private static async IAsyncEnumerable AsyncSource( + string[] items, + [EnumeratorCancellation] CancellationToken ct = default) + { + foreach (var item in items) + { + ct.ThrowIfCancellationRequested(); + yield return item; + } + await Task.CompletedTask; + } + } +} diff --git a/test/Paramore.Darker.Core.Tests/TestDoubles/EstablishmentFallbackStrategy.cs b/test/Paramore.Darker.Core.Tests/TestDoubles/EstablishmentFallbackStrategy.cs new file mode 100644 index 00000000..9b3e1640 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/TestDoubles/EstablishmentFallbackStrategy.cs @@ -0,0 +1,59 @@ +// Copyright (c) 2025, Ian Cooper +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are permitted provided that the +// following conditions are met: +// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the +// Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING +// BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +using System; +using System.Threading; +using System.Threading.Tasks; +using Polly; + +namespace Paramore.Darker.Core.Tests.TestDoubles +{ + /// + /// A custom Polly v8 resilience strategy that acts as a fallback for stream establishment. + /// When the wrapped callback faults, it invokes and returns + /// its result as a successful outcome. Used to test that + /// UseResiliencePipelineStreamHandler correctly hands the fallback-supplied + /// (IAsyncEnumerator<T>, bool) to the outer await using without leaking + /// the (already-disposed) primary enumerator. + /// + /// + /// The untyped boxes execution results to object + /// internally, so inside + /// is always object when used with the untyped pipeline. The factory therefore returns + /// a boxed tuple that Polly unboxes back to (IAsyncEnumerator<T>, bool) for the + /// caller. + /// + internal sealed class EstablishmentFallbackStrategy : ResilienceStrategy + { + private readonly Func> _fallbackFactory; + + public EstablishmentFallbackStrategy(Func> fallbackFactory) + { + _fallbackFactory = fallbackFactory; + } + + protected override async ValueTask> ExecuteCore( + Func>> callback, + ResilienceContext context, + TState state) + { + var outcome = await callback(context, state); + if (outcome.Exception != null) + { + var fallbackResult = await _fallbackFactory(context.CancellationToken); + return Outcome.FromResult((TResult)fallbackResult); + } + return outcome; + } + } +} From 0c692de124361372ba300ce7342c74d657108121 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 12:59:56 +0200 Subject: [PATCH 32/37] chore: Mark T026 complete in tasks.md Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index d9875891..d6f7d4ad 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -370,7 +370,7 @@ - Ensure the in-pipeline `Establish` callback disposes the enumerator in its `catch` before rethrowing (the reference article omits this) — ADR §3a -- [ ] **TEST + IMPLEMENT: T026 — Fallback strategy substitutes an alternate stream at establishment** +- [x] **TEST + IMPLEMENT: T026 — Fallback strategy substitutes an alternate stream at establishment** - **USE COMMAND**: `/test-first when the resilience pipeline has a fallback strategy should substitute an alternate stream at establishment` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_stream_establishment_fails_with_fallback_strategy_should_yield_alternate_stream.cs` From 7a342cbc46243518893bdf377dbeee2f4f94ea4e Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 13:35:32 +0200 Subject: [PATCH 33/37] feat: wire stream handler registry into DI assembly scan (T027) Add ServiceCollectionStreamHandlerRegistry that registers stream handler types in DI alongside the sync/async registries; thread it through AddHandlersFromAssemblies and BuildQueryProcessor into HandlerConfiguration so ExecuteStream resolves handlers from the container after an assembly scan. Co-Authored-By: Claude Sonnet 4.6 --- .../ServiceCollectionDarkerHandlerBuilder.cs | 4 ++ .../ServiceCollectionExtensions.cs | 9 ++-- .../ServiceCollectionStreamHandlerRegistry.cs | 49 +++++++++++++++++++ ...ssembly_should_register_stream_handlers.cs | 38 ++++++++++++++ 4 files changed, 97 insertions(+), 3 deletions(-) create mode 100644 src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionStreamHandlerRegistry.cs create mode 100644 test/Paramore.Darker.Extensions.Tests/When_AddHandlersFromAssemblies_scans_assembly_should_register_stream_handlers.cs diff --git a/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionDarkerHandlerBuilder.cs b/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionDarkerHandlerBuilder.cs index 9dc5f0ac..19dd0131 100644 --- a/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionDarkerHandlerBuilder.cs +++ b/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionDarkerHandlerBuilder.cs @@ -10,16 +10,19 @@ internal class ServiceCollectionDarkerHandlerBuilder : IDarkerHandlerBuilder private readonly ServiceCollectionDecoratorRegistry _decoratorRegistry; private readonly ServiceCollectionHandlerRegistry _registry; private readonly ServiceCollectionHandlerRegistryAsync _registryAsync; + private readonly ServiceCollectionStreamHandlerRegistry _registryStream; public IServiceCollection Services { get; } public ServiceCollectionDarkerHandlerBuilder(ServiceCollectionHandlerRegistry registry, ServiceCollectionHandlerRegistryAsync registryAsync, + ServiceCollectionStreamHandlerRegistry registryStream, ServiceCollectionDecoratorRegistry decoratorRegistry, IServiceCollection services) { _registry = registry; _registryAsync = registryAsync; + _registryStream = registryStream; _decoratorRegistry = decoratorRegistry; Services = services; } @@ -30,6 +33,7 @@ public IDarkerHandlerBuilder AddHandlersFromAssemblies(params Assembly[] assembl _registry.RegisterFromAssemblies(assemblies); _registryAsync.RegisterFromAssemblies(assemblies); + _registryStream.RegisterFromAssemblies(assemblies); return this; } diff --git a/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionExtensions.cs b/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionExtensions.cs index 52065f0b..a7d1f726 100644 --- a/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionExtensions.cs +++ b/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionExtensions.cs @@ -21,18 +21,20 @@ public static IDarkerHandlerBuilder AddDarker(this IServiceCollection services, var handlerRegistry = new ServiceCollectionHandlerRegistry(services, options.HandlerLifetime); var handlerRegistryAsync = new ServiceCollectionHandlerRegistryAsync(services, options.HandlerLifetime); + var handlerRegistryStream = new ServiceCollectionStreamHandlerRegistry(services, options.HandlerLifetime); var decoratorRegistry = new ServiceCollectionDecoratorRegistry(services, options.HandlerLifetime); decoratorRegistry.RegisterDefaultDecorators(); - services.TryAdd(new ServiceDescriptor(typeof(IQueryProcessor), provider => BuildQueryProcessor(handlerRegistry, handlerRegistryAsync, provider, decoratorRegistry, options), options.QueryProcessorLifetime)); + services.TryAdd(new ServiceDescriptor(typeof(IQueryProcessor), provider => BuildQueryProcessor(handlerRegistry, handlerRegistryAsync, handlerRegistryStream, provider, decoratorRegistry, options), options.QueryProcessorLifetime)); - return new ServiceCollectionDarkerHandlerBuilder(handlerRegistry, handlerRegistryAsync, decoratorRegistry, services); + return new ServiceCollectionDarkerHandlerBuilder(handlerRegistry, handlerRegistryAsync, handlerRegistryStream, decoratorRegistry, services); } private static QueryProcessor BuildQueryProcessor( IQueryHandlerRegistry handlerRegistry, IQueryHandlerRegistryAsync handlerRegistryAsync, + IStreamQueryHandlerRegistry handlerRegistryStream, IServiceProvider provider, ServiceCollectionDecoratorRegistry decoratorRegistry, DarkerOptions options) @@ -48,7 +50,8 @@ private static QueryProcessor BuildQueryProcessor( return new QueryProcessor( new HandlerConfiguration( handlerRegistry, componentFactory, decoratorRegistry, componentFactory, - handlerRegistryAsync, componentFactory, decoratorRegistry, componentFactory), + handlerRegistryAsync, componentFactory, decoratorRegistry, componentFactory, + handlerRegistryStream), options.QueryContextFactory, policyRegistry, resiliencePipelineProvider, tracer, options.InstrumentationOptions); } diff --git a/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionStreamHandlerRegistry.cs b/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionStreamHandlerRegistry.cs new file mode 100644 index 00000000..a0ef05af --- /dev/null +++ b/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionStreamHandlerRegistry.cs @@ -0,0 +1,49 @@ +// The MIT License (MIT) +// Copyright (c) 2016 Ian Cooper +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// The above copyright notice and this permission notice shall be included in all +// copies or substantial portions of the Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +// THE SOFTWARE. + +using System; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.DependencyInjection.Extensions; + +namespace Paramore.Darker.Extensions.DependencyInjection +{ + /// + /// A that also registers each handler type + /// in a so the DI container can resolve it. + /// + internal sealed class ServiceCollectionStreamHandlerRegistry : StreamQueryHandlerRegistry + { + private readonly ServiceLifetime _lifetime; + private readonly IServiceCollection _services; + + public ServiceCollectionStreamHandlerRegistry(IServiceCollection services, ServiceLifetime lifetime) + { + _services = services; + _lifetime = lifetime; + } + + /// + public override void Register(Type queryType, Type resultType, Type handlerType) + { + _services.TryAdd(new ServiceDescriptor(handlerType, handlerType, _lifetime)); + + base.Register(queryType, resultType, handlerType); + } + } +} diff --git a/test/Paramore.Darker.Extensions.Tests/When_AddHandlersFromAssemblies_scans_assembly_should_register_stream_handlers.cs b/test/Paramore.Darker.Extensions.Tests/When_AddHandlersFromAssemblies_scans_assembly_should_register_stream_handlers.cs new file mode 100644 index 00000000..27ab8058 --- /dev/null +++ b/test/Paramore.Darker.Extensions.Tests/When_AddHandlersFromAssemblies_scans_assembly_should_register_stream_handlers.cs @@ -0,0 +1,38 @@ +using System.Collections.Generic; +using System.Threading.Tasks; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using Paramore.Darker.Extensions.DependencyInjection; +using Paramore.Darker.Core.Tests.Exported; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Extensions.Tests +{ + public class AddDarkerStreamHandlerRegistrationTests + { + [Fact] + public async Task When_AddHandlersFromAssemblies_scans_assembly_should_register_stream_handlers_so_ExecuteStream_yields_items() + { + // Arrange + var services = new ServiceCollection(); + services.AddSingleton(NullLoggerFactory.Instance); + services.AddDarker().AddHandlersFromAssemblies(typeof(ExportedStreamQueryHandler).Assembly); + + var provider = services.BuildServiceProvider(); + var queryProcessor = provider.GetRequiredService(); + var results = new List(); + + // Act + await foreach (var item in queryProcessor.ExecuteStream(new ExportedStreamQuery())) + { + results.Add(item); + } + + // Assert + results.ShouldHaveSingleItem(); + results[0].ShouldBe("exported-item"); + } + } +} From 4a02945c0a5a6c8ba094a023d43824dcc0f138bb Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 17:03:02 +0200 Subject: [PATCH 34/37] feat: add AddStreamHandlers explicit registration to DI builder (T028) Add AddStreamHandlers(Action) to IDarkerHandlerBuilder and implement it on ServiceCollectionDarkerHandlerBuilder, mirroring AddAsyncHandlers so callers can explicitly register stream handlers without an assembly scan. Co-Authored-By: Claude Sonnet 4.6 --- .../IDarkerHandlerBuilder.cs | 1 + .../ServiceCollectionDarkerHandlerBuilder.cs | 10 +++++ ..._explicitly_should_execute_stream_query.cs | 39 +++++++++++++++++++ 3 files changed, 50 insertions(+) create mode 100644 test/Paramore.Darker.Extensions.Tests/When_registering_stream_handlers_explicitly_should_execute_stream_query.cs diff --git a/src/Paramore.Darker.Extensions.DependencyInjection/IDarkerHandlerBuilder.cs b/src/Paramore.Darker.Extensions.DependencyInjection/IDarkerHandlerBuilder.cs index ce219232..ce7847a6 100644 --- a/src/Paramore.Darker.Extensions.DependencyInjection/IDarkerHandlerBuilder.cs +++ b/src/Paramore.Darker.Extensions.DependencyInjection/IDarkerHandlerBuilder.cs @@ -11,5 +11,6 @@ public interface IDarkerHandlerBuilder : IQueryProcessorExtensionBuilder IDarkerHandlerBuilder AddHandlersFromAssemblies(params Assembly[] assemblies); IDarkerHandlerBuilder AddHandlers(Action registerHandlers); IDarkerHandlerBuilder AddAsyncHandlers(Action registerHandlers); + IDarkerHandlerBuilder AddStreamHandlers(Action registerHandlers); } } \ No newline at end of file diff --git a/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionDarkerHandlerBuilder.cs b/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionDarkerHandlerBuilder.cs index 19dd0131..7cdd4774 100644 --- a/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionDarkerHandlerBuilder.cs +++ b/src/Paramore.Darker.Extensions.DependencyInjection/ServiceCollectionDarkerHandlerBuilder.cs @@ -58,6 +58,16 @@ public IDarkerHandlerBuilder AddAsyncHandlers(Action return this; } + public IDarkerHandlerBuilder AddStreamHandlers(Action registerHandlers) + { + if (registerHandlers == null) + throw new ArgumentNullException(nameof(registerHandlers)); + + registerHandlers(_registryStream); + + return this; + } + public IQueryProcessorExtensionBuilder RegisterDecorator(Type decoratorType) { if (decoratorType == null) throw new ArgumentNullException(nameof(decoratorType)); diff --git a/test/Paramore.Darker.Extensions.Tests/When_registering_stream_handlers_explicitly_should_execute_stream_query.cs b/test/Paramore.Darker.Extensions.Tests/When_registering_stream_handlers_explicitly_should_execute_stream_query.cs new file mode 100644 index 00000000..7a6ec49d --- /dev/null +++ b/test/Paramore.Darker.Extensions.Tests/When_registering_stream_handlers_explicitly_should_execute_stream_query.cs @@ -0,0 +1,39 @@ +using System.Collections.Generic; +using System.Threading.Tasks; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using Paramore.Darker.Extensions.DependencyInjection; +using Paramore.Darker.Core.Tests.Exported; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Extensions.Tests +{ + public class AddStreamHandlersExplicitRegistrationTests + { + [Fact] + public async Task When_registering_stream_handler_explicitly_via_builder_should_execute_stream_query_and_yield_items() + { + // Arrange + var services = new ServiceCollection(); + services.AddSingleton(NullLoggerFactory.Instance); + services.AddDarker() + .AddStreamHandlers(r => r.Register()); + + var provider = services.BuildServiceProvider(); + var queryProcessor = provider.GetRequiredService(); + var results = new List(); + + // Act + await foreach (var item in queryProcessor.ExecuteStream(new ExportedStreamQuery())) + { + results.Add(item); + } + + // Assert + results.ShouldHaveSingleItem(); + results[0].ShouldBe("exported-item"); + } + } +} From de7567fda765e2ea96419429a7dc6faaf80de752 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 19:32:23 +0200 Subject: [PATCH 35/37] test: document QueryProcessorBuilder stream registry wiring (T029) Verify that a processor built via the fluent QueryProcessorBuilder with a StreamQueryHandlerRegistry in HandlerConfiguration executes stream queries via ExecuteStream; behaviour was already covered by T006/T008 so no new implementation was required. Co-Authored-By: Claude Sonnet 4.6 --- ...am_registry_should_execute_stream_query.cs | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 test/Paramore.Darker.Core.Tests/When_QueryProcessorBuilder_configured_with_stream_registry_should_execute_stream_query.cs diff --git a/test/Paramore.Darker.Core.Tests/When_QueryProcessorBuilder_configured_with_stream_registry_should_execute_stream_query.cs b/test/Paramore.Darker.Core.Tests/When_QueryProcessorBuilder_configured_with_stream_registry_should_execute_stream_query.cs new file mode 100644 index 00000000..2dda6c09 --- /dev/null +++ b/test/Paramore.Darker.Core.Tests/When_QueryProcessorBuilder_configured_with_stream_registry_should_execute_stream_query.cs @@ -0,0 +1,52 @@ +using System.Collections.Generic; +using System.Threading.Tasks; +using Paramore.Darker.Builder; +using Paramore.Darker.Core.Tests.Exported; +using Shouldly; +using Xunit; + +namespace Paramore.Darker.Core.Tests +{ + public class QueryProcessorBuilderStreamTests + { + [Fact] + public async Task When_QueryProcessorBuilder_configured_with_stream_registry_should_execute_stream_query() + { + // Arrange + var streamRegistry = new StreamQueryHandlerRegistry(); + streamRegistry.Register(); + + var handlerFactory = new SimpleHandlerFactory(type => + { + if (type == typeof(ExportedStreamQueryHandler)) return new ExportedStreamQueryHandler(); + return null!; + }); + + var queryProcessor = QueryProcessorBuilder.With() + .Handlers(new HandlerConfiguration( + new QueryHandlerRegistry(), + handlerFactory, + new InMemoryDecoratorRegistry(), + new SimpleHandlerDecoratorFactory(type => null!), + new QueryHandlerRegistryAsync(), + handlerFactory, + new InMemoryDecoratorRegistry(), + new SimpleHandlerDecoratorFactory(type => null!), + streamRegistry)) + .InMemoryQueryContextFactory() + .Build(); + + var results = new List(); + + // Act + await foreach (var item in queryProcessor.ExecuteStream(new ExportedStreamQuery())) + { + results.Add(item); + } + + // Assert + results.ShouldHaveSingleItem(); + results[0].ShouldBe("exported-item"); + } + } +} From 155bf32ddf6fe0a070aea3ebc01bdfa48bb811d0 Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 20:53:52 +0200 Subject: [PATCH 36/37] chore: mark T027-T030 complete in tasks.md T027-T029 DI wiring complete; T030 verified green on net8.0/net9.0 Release build and AOT publish with no new warnings or errors. Co-Authored-By: Claude Sonnet 4.6 --- specs/012-streaming_results/tasks.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index d6f7d4ad..447b6906 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -388,7 +388,7 @@ ## Phase 7 — Dependency Injection & registration wiring (behaviour) -- [ ] **TEST + IMPLEMENT: T027 — `AddHandlersFromAssemblies` registers stream handlers for DI** +- [x] **TEST + IMPLEMENT: T027 — `AddHandlersFromAssemblies` registers stream handlers for DI** - **USE COMMAND**: `/test-first when AddDarker scans assemblies should register stream handlers so ExecuteStream resolves them from the container` - Test location: `test/Paramore.Darker.Extensions.Tests` - Test file: `When_AddHandlersFromAssemblies_scans_assembly_should_register_stream_handlers.cs` @@ -402,7 +402,7 @@ - Thread the stream registry into the `HandlerConfiguration` built in `ServiceCollectionExtensions.BuildQueryProcessor` (ADR §5.4) -- [ ] **TEST + IMPLEMENT: T028 — Explicit `AddStreamHandlers` registration** +- [x] **TEST + IMPLEMENT: T028 — Explicit `AddStreamHandlers` registration** - **USE COMMAND**: `/test-first when registering stream handlers explicitly via the builder should resolve and execute the stream query` - Test location: `test/Paramore.Darker.Extensions.Tests` - Test file: `When_registering_stream_handlers_explicitly_should_execute_stream_query.cs` @@ -414,7 +414,7 @@ - Add `AddStreamHandlers(Action)` to `IDarkerHandlerBuilder` / `ServiceCollectionDarkerHandlerBuilder` (mirror `AddAsyncHandlers`) -- [ ] **TEST + IMPLEMENT: T029 — Non-DI `QueryProcessorBuilder` wires the stream registry** +- [x] **TEST + IMPLEMENT: T029 — Non-DI `QueryProcessorBuilder` wires the stream registry** - **USE COMMAND**: `/test-first when a processor is built via QueryProcessorBuilder with a stream registry should execute a stream query` - Test location: `test/Paramore.Darker.Core.Tests` - Test file: `When_QueryProcessorBuilder_configured_with_stream_registry_should_execute_stream_query.cs` @@ -430,7 +430,7 @@ ## Phase 8 — Cross-target build, AOT, docs (verification) -- [ ] **VERIFY: T030 — Green on all target frameworks + AOT** +- [x] **VERIFY: T030 — Green on all target frameworks + AOT** - `dotnet build Darker.Filter.slnf -c Release` and `dotnet test Darker.Filter.slnf -c Release --no-build` pass on `netstandard2.0;net8.0;net9.0` (NFR3). - `Paramore.Darker.Tests.AOT` still builds/publishes (NFR4 — no new trim/AOT-hostile reflection From 060fb692e80c500467e0117c164aa0570c5e2fef Mon Sep 17 00:00:00 2001 From: Ian Cooper Date: Mon, 13 Jul 2026 21:56:19 +0200 Subject: [PATCH 37/37] docs: add streaming queries documentation (T031) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a 'Streaming Queries' section to the root README covering: ExecuteStream + await foreach usage, DI registration (scan and explicit), QueryProcessorBuilder wiring, UseResiliencePipelineStream attribute, and a semantics table (laziness, cancellation, exceptions, deferred config error, re-enumeration, caller context, legacy attribute restriction). Update specs/012-streaming_results/README.md with full implementation summary — all 31 tasks across 9 phases complete. Co-Authored-By: Claude Sonnet 4.6 --- README.md | 105 ++++++++++++++++++++++++++ specs/012-streaming_results/README.md | 23 +++++- specs/012-streaming_results/tasks.md | 2 +- 3 files changed, 126 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 7ad2edf8..8a91a6eb 100644 --- a/README.md +++ b/README.md @@ -115,3 +115,108 @@ IQueryProcessor queryProcessor = QueryProcessorBuilder.With() Instead of `Activator.CreateInstance`, you can pass any factory `Func` to construct handlers and decorators. > **Note:** The `Paramore.Darker.SimpleInjector` and `Paramore.Darker.LightInject` packages have been removed as of V5. If you use a third-party DI container, use its built-in adapter for `Microsoft.Extensions.DependencyInjection` and integrate with Darker via the `Paramore.Darker.Extensions.DependencyInjection` package instead. + +## Streaming Queries + +Darker supports streaming queries that yield results incrementally as `IAsyncEnumerable`, +so large result sets or real-time feeds are produced on demand rather than buffered into memory. + +### Define a stream query and handler + +```csharp +using Paramore.Darker; +using System.Collections.Generic; +using System.Threading; + +// TResult is the item type, not the enumerable. +public sealed class GetOrdersStream : IStreamQuery +{ + public string CustomerId { get; } + public GetOrdersStream(string customerId) => CustomerId = customerId; +} + +public sealed class GetOrdersStreamHandler : StreamQueryHandler +{ + public override async IAsyncEnumerable ExecuteAsync( + GetOrdersStream query, + [System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken = default) + { + await foreach (var order in _repository.StreamByCustomerAsync(query.CustomerId, cancellationToken)) + yield return order; + } +} +``` + +### Execute with `await foreach` + +```csharp +await foreach (var order in queryProcessor.ExecuteStream(new GetOrdersStream("C42"), cancellationToken)) +{ + // items arrive as the handler produces them — no buffering + Process(order); +} +``` + +### Registration with DI (assembly scan) + +`AddHandlersFromAssemblies` picks up `IStreamQueryHandler<,>` implementations automatically alongside +sync and async handlers: + +```csharp +services.AddDarker() + .AddHandlersFromAssemblies(typeof(GetOrdersStreamHandler).Assembly); +``` + +### Registration with DI (explicit) + +```csharp +services.AddDarker() + .AddStreamHandlers(r => r.Register()); +``` + +### Registration without DI + +```csharp +var streamRegistry = new StreamQueryHandlerRegistry(); +streamRegistry.Register(); + +IQueryProcessor queryProcessor = QueryProcessorBuilder.With() + .Handlers(new HandlerConfiguration( + syncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + asyncRegistry, handlerFactory, decoratorRegistry, decoratorFactory, + streamRegistry)) + .InMemoryQueryContextFactory() + .Build(); +``` + +### Resilience (Polly v8) + +Use `[UseResiliencePipelineStream]` — **not** `[RetryableQuery]` or `[FallbackPolicy]`, which apply +only to single-result handlers and throw a `ConfigurationException` on a stream handler: + +```csharp +public sealed class GetOrdersStreamHandler : StreamQueryHandler +{ + [UseResiliencePipelineStream(1, "MyRetryPipeline")] + public override async IAsyncEnumerable ExecuteAsync(GetOrdersStream query, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { ... } +} +``` + +Resilience covers **stream establishment and the first item only**. Once the first item has been +yielded to the caller, the pipeline has exited and subsequent faults propagate un-retried. A `Timeout` +strategy therefore bounds *getting the stream started*, not total enumeration time. `Hedging` is not +supported for streams. These are intentional semantics, not limitations to be worked around. + +### Documented semantics + +| Behaviour | Detail | +|---|---| +| **Laziness** | The framework never buffers the sequence; items are produced on demand. Custom decorators must also avoid buffering (e.g. `ToListAsync`). | +| **Cancellation** | Pass a `CancellationToken` to `ExecuteStream`; cancelling mid-stream stops enumeration and propagates `OperationCanceledException`. | +| **Exceptions mid-stream** | Faults during enumeration propagate with their original stack trace — no `TargetInvocationException` wrapper. | +| **Configuration errors** | A missing or mismatched handler surfaces as `ConfigurationException` from the caller's **first `await foreach` iteration**, not from the `ExecuteStream` call itself (deliberate — resolving eagerly would leak the handler if the caller never enumerates). | +| **Re-enumeration** | Each `await foreach` over the same `IAsyncEnumerable` re-executes the handler with a fresh pipeline. The stream is cold, not cached. To iterate twice over the same data, buffer it yourself (`await ToListAsync()`). | +| **Caller-supplied context** | A `queryContext` passed to `ExecuteStream` is scoped to a **single enumeration**. Concurrent or repeated enumeration is only safe when the processor creates the context (pass `null`). | +| **Legacy attributes** | `[RetryableQuery]` and `[FallbackPolicy]` do **not** apply to streams. Use `[UseResiliencePipelineStream]` for stream resilience. Applying a mismatched attribute throws `ConfigurationException`. | diff --git a/specs/012-streaming_results/README.md b/specs/012-streaming_results/README.md index 04a498e6..91822150 100644 --- a/specs/012-streaming_results/README.md +++ b/specs/012-streaming_results/README.md @@ -18,12 +18,29 @@ result set. Requirements, design, and scope to be established in the workflow be - [x] Design / ADR (`/spec:design`) — ✅ ADR 0019 `Accepted` - [x] Adversarial Review — 3 rounds (6 + 4 + 3 findings) all resolved in ADR 0019 (see `review-design.md`) - [x] Task Breakdown (`/spec:tasks`) — ✅ `tasks.md` created (31 tasks across 9 phases) -- [ ] Implementation (`/spec:implement` or `/spec:ralph-implement`) +- [x] Implementation (`/spec:implement`) — ✅ T001–T031 complete + +## Implementation Summary + +All 31 tasks across 9 phases are complete: + +| Phase | Tasks | Description | +|-------|-------|-------------| +| 0 | T001–T003 | Structural foundations: `IAsyncEnumerable` target support, stream query/handler/decorator contracts | +| 1 | T004–T005 | Handler registry: `StreamQueryHandlerRegistry` + assembly scan | +| 2 | T006–T008 | Pipeline build + processor entry point: `BuildStream`, `ExecuteStream` | +| 3 | T009–T014 | Core correctness: laziness, cancellation, exceptions, lifetime, cross-path guard, deferred config error | +| 4 | T015–T017 | Decorator pipeline: ordered chain, mismatch validation, re-enumeration | +| 5 | T018–T020 | Logging/telemetry: stream lifecycle logging, fault recording, span events per step | +| 6 | T021–T026 | Resilience: `UseResiliencePipelineStreamHandler`, retry/no-duplicate, post-first-item propagation, enumerator disposal, fallback | +| 7 | T027–T029 | DI wiring: assembly scan, explicit `AddStreamHandlers`, `QueryProcessorBuilder` | +| 8 | T030–T031 | Verification + docs: cross-target Release build, AOT publish, user documentation | ## Documents | Phase | File | Status | |-------|------|--------| -| Requirements | `requirements.md` | ⬜ Not started | +| Requirements | `requirements.md` | ✅ Approved | | Design | `docs/adr/0019-streaming-query-pipeline.md` | ✅ Accepted | -| Tasks | `tasks.md` | ✅ Created | +| Tasks | `tasks.md` | ✅ All complete | +| User docs | `README.md` (root) — "Streaming Queries" section | ✅ Added | diff --git a/specs/012-streaming_results/tasks.md b/specs/012-streaming_results/tasks.md index 447b6906..d1ed55a6 100644 --- a/specs/012-streaming_results/tasks.md +++ b/specs/012-streaming_results/tasks.md @@ -437,7 +437,7 @@ beyond existing `BuildAsync`). - Not a `/test-first` task — runs the existing suite; fix regressions surfaced. -- [ ] **DOCS: T031 — Document streaming usage and semantics** +- [x] **DOCS: T031 — Document streaming usage and semantics** - Update README / user docs with: `ExecuteStream` + `await foreach` usage; DI registration; and the **documented semantics** — deferred config error on first `MoveNextAsync` (ADR §4); resilience covers **establishment + first item only**, `Timeout` bounds start-up, `Hedging` unsupported (ADR §3a);