Fix aggregation metadata session not reconnecting after downstream loss (#312) - #750
Merged
Conversation
…ss (#312) Reconnect the aggregation server's downstream session on any not-good keep-alive status instead of gating on OutstandingRequestCount + DefunctRequestCount >= 3. When the downstream server restarts or the network drops, the keep-alive read fails synchronously (BadConnectionClosed) before the request is registered, so those counters never rise and the reconnect logic never fired. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
The aggregation server's internal "metadata" session never recovered after a downstream server was restarted or the network connection was lost (issue #312). Once the keep-alive went "late", the aggregator kept retrying but never re-established the session.
The root cause is in
Client_KeepAlive(Workshop/Aggregation/Server/AggregationNodeManager.cs). Reconnection was gated onsession.OutstandingRequestCount + session.DefunctRequestCount >= 3. When the downstream drops, the keep-alive read fails synchronously (e.g.BadConnectionClosed) before the request is ever registered, so those counters never increment and the gate never opened. TheSessionReconnectHandlerwas therefore never started.This change starts reconnection immediately on any not-good keep-alive status (
ServiceResult.IsNotGood(e.Status)), keeping only the!session.SessionId.IsNullguard. The existing stale-session cleanup inGetClientSessionremains as a backstop.Note: the original report also described a stack-level secure-channel "renew instead of recreate" failure (
BadTcpSecureChannelUnknown) against stack1.4.368. This sample now runs on stack2.0.158-preview, where that behavior has been reworked, so no samples change is needed for that half.Related Issues
Types of changes
What types of changes does your code introduce?
Checklist
Further comments
The removed count-based gate was the specific mechanism the issue reporter identified as never firing. Reconnecting on a bad keep-alive status matches the pattern used by the reference client samples. The Aggregation Server project builds cleanly with this change (remaining warnings are pre-existing and unrelated).