part: allocate watch channels lazily - #178
Merged
Merged
Conversation
|
Only allocate the watch channels when requested.
Split Tree and Txn Get/Prefix into no-watch defaults and
explicit GetWatch/PrefixWatch methods, and route ordinary StateDB
index operations through no-watch paths while preserving existing
watch invalidation semantics.
StateDB benchmarks:
goos: linux
goarch: arm64
pkg: github.com/cilium/statedb
│ a.txt │ b.txt │
│ sec/op │ sec/op vs base │
DB_Modify-6 570.2µ ± ∞ ¹ 528.1µ ± ∞ ¹ ~ (p=0.095 n=5)
DB_GetInsert-6 633.9µ ± ∞ ¹ 586.5µ ± ∞ ¹ ~ (p=0.095 n=5)
DB_RandomInsert-6 613.6µ ± ∞ ¹ 515.3µ ± ∞ ¹ ~ (p=0.095 n=5)
DB_SequentialInsert-6 564.4µ ± ∞ ¹ 501.2µ ± ∞ ¹ ~ (p=0.151 n=5)
DB_RandomLookup-6 34.62µ ± ∞ ¹ 33.48µ ± ∞ ¹ -3.30% (p=0.016 n=5)
DB_SequentialLookup-6 27.89µ ± ∞ ¹ 26.04µ ± ∞ ¹ -6.63% (p=0.008 n=5)
DB_Prefix_SecondaryIndex-6 112.2µ ± ∞ ¹ 111.2µ ± ∞ ¹ ~ (p=0.151 n=5)
geomean 201.7µ 186.2µ -7.66%
¹ need >= 6 samples for confidence interval at level 0.95
│ a.txt │ b.txt │
│ B/op │ B/op vs base │
DB_Modify-6 533.5Ki ± ∞ ¹ 431.1Ki ± ∞ ¹ -19.18% (p=0.008 n=5)
DB_GetInsert-6 517.8Ki ± ∞ ¹ 415.5Ki ± ∞ ¹ -19.76% (p=0.008 n=5)
DB_RandomInsert-6 509.9Ki ± ∞ ¹ 407.7Ki ± ∞ ¹ -20.06% (p=0.008 n=5)
DB_SequentialInsert-6 509.9Ki ± ∞ ¹ 407.7Ki ± ∞ ¹ -20.06% (p=0.008 n=5)
DB_RandomLookup-6 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
DB_SequentialLookup-6 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ (p=1.000 n=5) ²
DB_Prefix_SecondaryIndex-6 122.0Ki ± ∞ ¹ 122.0Ki ± ∞ ¹ +0.03% (p=0.008 n=5)
geomean ³ -11.82% ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean
Reconciler benchmark before (best of three runs):
1000000 objects reconciled in 1.35 seconds (batch size 1000)
Throughput 740668.25 objects per second
888MB total allocated, 6015187 in-use objects, 338MB bytes in use
2.00user 0.21system 0:01.50elapsed 147%CPU (0avgtext+0avgdata 617884maxresident)k
152inputs+0outputs (1major+203587minor)pagefaults 0swaps
Reconciler benchmark after (best of three runs):
1000000 objects reconciled in 1.32 seconds (batch size 1000)
Throughput 760075.67 objects per second
687MB total allocated, 6015138 in-use objects, 239MB bytes in use
2.17user 0.20system 0:01.45elapsed 162%CPU (0avgtext+0avgdata 475600maxresident)k
0inputs+0outputs (0major+136382minor)pagefaults 0swaps
AIL:3
Signed-off-by: Jussi Maki <jussi@isovalent.com>
Reconciliation status is committed before metrics are published, and refresh jobs register their health nodes independently. Wait for both states in the multi-reconciler test so it does not depend on goroutine scheduling under the race detector. AIL:3 Signed-off-by: Jussi Maki <jussi@isovalent.com>
Add a benchmark that replaces 1,000 values per committed transaction in a tree with per-node watches. This exercises path cloning and watch notification collection. AIL:3 Signed-off-by: Jussi Maki <jussi@isovalent.com>
Collect watch states in a slice instead of a map. Tree traversal visits
each cloned node once. Lazy watches may share a state across physical
nodes; closing a state is idempotent, so duplicates are safe. Clear
the slice before reuse so closed channels and their subtrees are
not retained.
goos: linux
goarch: arm64
pkg: github.com/cilium/statedb/part
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
_Insert-6 90.81µ ± 5% 87.97µ ± 2% ~ (p=0.315 n=10)
PatchWatchReplace-6 142.30µ ± 8% 81.95µ ± 3% -42.41% (p=0.000 n=10)
│ before.txt │ after.txt │
│ objects/sec │ objects/sec vs base │
_Insert-6 11.01M ± 5% 11.37M ± 2% ~ (p=0.315 n=10)
PatchWatchReplace-6 7.030M ± 7% 12.202M ± 3% +73.58% (p=0.000 n=10)
│ before.txt │ after.txt │
│ B/op │ B/op vs base │
_Insert-6 78.66Ki ± 0% 77.81Ki ± 0% -1.07% (p=0.000 n=10)
PatchWatchReplace-6 136.52Ki ± 0% 63.81Ki ± 0% -53.26% (p=0.000 n=10)
│ before.txt │ after.txt │
│ allocs/op │ allocs/op vs base │
_Insert-6 3.060k ± 0% 3.058k ± 0% -0.07% (p=0.000 n=10)
PatchWatchReplace-6 2.033k ± 0% 2.011k ± 0% -1.08% (p=0.000 n=10)
AIL:3
Signed-off-by: Jussi Maki <jussi@isovalent.com>
joamaki
force-pushed
the
pr/joamaki/lazy-watches
branch
from
August 25, 2026 09:59
1084d0c to
8c26bd9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Only allocate the watch channels when requested.
Split Tree and Txn Get/Prefix into no-watch defaults and explicit GetWatch/PrefixWatch methods, and route ordinary StateDB index operations through no-watch paths while preserving existing watch invalidation semantics.
StateDB benchmarks:
Reconciler benchmark before (best of three runs):
Reconciler benchmark after (best of three runs):
AIL:3