[release-7.4] Fix an old exclude failed bug - #13869
Open
spraza wants to merge 2 commits into
Open
Conversation
In removeKeysFromFailedServer, the writes that unassign a dropped shard from the dest servers (serverKeysFalse) and the writes that reassign it to the new team as an empty range were pushed into a single actors vector and awaited together. krmSetRangeCoalescing performs a read-modify-write on the serverKeys map, so when a server appears in both the dest set and the new team, running the two batches concurrently lets the coalescing reads observe stale state and produce an incorrect serverKeys assignment. Await the unassign batch first, then run the empty-range assignment in a separate batch, ensuring the serverKey map assignment is correct. Ported from upstream/main PR apple#13645 (8bd7e44). Conflicts resolved in: fdbserver/MoveKeys.actor.cpp (release-7.4 still uses the wait()/actor-compiler style rather than main's co_await; shardId needed to become state since it now crosses the new wait() boundary). Same resolution as the release-7.3 backport, PR apple#13662.
spraza
marked this pull request as ready for review
August 14, 2026 17:05
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
ploxiln
approved these changes
Aug 14, 2026
saintstack
approved these changes
Aug 14, 2026
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Contributor
Result of foundationdb-pr-clang-arm on Linux RHEL 9
|
Contributor
Result of foundationdb-pr-macos-m1 on macOS 14.x
|
This comment has been minimized.
This comment has been minimized.
Contributor
Result of foundationdb-pr-clang on Linux RHEL 9
|
Contributor
Result of foundationdb-pr on Linux RHEL 9
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Contributor
Result of foundationdb-pr-cluster-tests on Linux RHEL 9
|
Contributor
Result of foundationdb-pr-macos on macOS 14.x
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port #13645 to release-7.4.
Problem: In removeKeysFromFailedServer, the writes that unassign a dropped shard from the dest servers (serverKeysFalse) and the writes that reassign it to the new team as an empty range were pushed into a single actors vector and awaited together. krmSetRangeCoalescing performs a read-modify-write on the serverKeys map, so when a server appears in both the dest set and the new team, running the two batches concurrently lets the coalescing reads observe stale state and produce an incorrect serverKeys assignment.
Solution: await the unassign batch first, then run the empty-range assignment in a separate batch, ensuring the serverKey map assignment is correct.
Same fix already backported to release-7.3 in #13662. release-7.4 still used the wait()/actor-compiler style rather than main's co_await, and shardId needed to become state since it now crosses the new wait() boundary -- same conflict resolution as the 7.3 backport.
100K: 20260813-171407-praza-port-7.4-pr13645-817f-e3951c728b485de6 pass=100000 fail=0 ended=100000 max_runs=100000