Skip to content

Stabilize CachedResponseBodyTests.Copy_SingleSegment - #68110

Open
irfanajaffer wants to merge 2 commits into
dotnet:mainfrom
irfanajaffer:61670_TimeoutIssue
Open

Stabilize CachedResponseBodyTests.Copy_SingleSegment#68110
irfanajaffer wants to merge 2 commits into
dotnet:mainfrom
irfanajaffer:61670_TimeoutIssue

Conversation

@irfanajaffer

Copy link
Copy Markdown
Contributor

Stabilize CachedResponseBodyTests.Copy_SingleSegment

Description

This PR addresses flakiness in CachedResponseBodyTests.Copy_SingleSegment.

The test could intermittently fail with System.OperationCanceledException during CI execution. The failure was determined to originate from the test implementation rather than the CachedResponseBody production code.

The test uses a shared CancellationTokenSource with a 5-second timeout for both the producer and consumer operations. Under normal execution the copy and read operations complete successfully. However, under heavy CI load or thread-pool contention, the timeout can expire while the operations are still making forward progress. When this occurs, either PipeReader.ReadAsync(...) or CachedResponseBody.CopyToAsync(...) observes cancellation and throws OperationCanceledException, causing the test to fail despite the underlying functionality behaving correctly.

To make the test more resilient, timeout-driven cancellation has been removed from the copy and receive operations and replaced with an explicit hang guard using Task.WaitAsync(...). This preserves protection against hangs while allowing valid in-flight work to complete regardless of temporary scheduling delays.

The updated implementation explicitly:

  • Removes cancellation-based failure paths from the producer and consumer operations.
  • Preserves timeout protection through a dedicated hang guard.
  • Allows genuine hangs to surface as TimeoutException instead of injecting cancellation into otherwise healthy operations.
  • Eliminates the race condition between task completion and timeout expiration.

With these changes in place, the test no longer depends on a timeout-based cancellation mechanism and is resilient to transient CI scheduling delays.


Validation / Investigation

As part of the investigation:

  1. The failure was analyzed and determined to be caused by timeout-triggered cancellation rather than a defect in CachedResponseBody.
  2. The test flow was reviewed and the shared CancellationTokenSource was identified as the source of the intermittent OperationCanceledException.
  3. The timeout logic was replaced with a dedicated hang guard using Task.WaitAsync(...).
  4. Additional stress runs were performed to verify that the test remains stable under varying scheduling conditions.

While the issue is timing-dependent and primarily manifests under CI contention, the updated implementation removes the cancellation race entirely by allowing the producer and consumer to complete naturally while enforcing an external completion deadline.


Changes

  1. Removed timeout-based CancellationTokenSource usage from producer and consumer operations.
  2. Updated the test to execute copy and receive operations using CancellationToken.None.
  3. Replaced cancellation-driven timeout behavior with Task.WaitAsync(...) hang-guard protection.
  4. Added debugger-aware timeout handling to prevent interference during local debugging sessions.
  5. Ensured genuine hangs continue to fail deterministically while eliminating false failures caused by cancellation timing races.

Fixes #61670.

@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Jul 30, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Thanks for your PR, @irfanajaffer. Someone from the team will get assigned to your PR shortly and we'll get it reviewed.

@irfanajaffer
irfanajaffer marked this pull request as ready for review July 31, 2026 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retry Microsoft.AspNetCore.ResponseCaching.Tests.CachedResponseBodyTests.Copy_SingleSegment

1 participant