part: add bitmap Node64 and Node128 - #180
Merged
Merged
Conversation
|
pippolo84
approved these changes
Aug 25, 2026
pippolo84
left a comment
Member
There was a problem hiding this comment.
The addition of the two intermediate sizes Node64 and Node128 is indeed effective. Nice!
Add a benchmark for small committed transactions against a prepopulated random tree whose second-level nodes have a mean fanout of 96. This isolates the path-copying cost of transactions containing 1, 2, 4, 8, or 16 updates. Signed-off-by: Jussi Maki <jussi@isovalent.com>
Replace Node48's 256-byte reverse index with packed, sorted child
arrays indexed by a 256-bit bitmap and popcount. Add Node64 and
Node128 stages between Node16 and Node256, including symmetric
promotion and demotion paths.
On 64-bit Go, the node sizes and allocator classes are:
raw size size class slack
Node48 680B 704B 24B
Node64 584B 640B 56B
Node128 1096B 1152B 56B
Node256 2088B 2304B 216B
Thus Node64 has 16 more child slots than Node48 while using a smaller
allocator class. Node128 delays the much larger Node256 allocation
until fanout exceeds 128.
part benchmarks (10 runs, 300ms benchtime):
goos: linux
goarch: arm64
pkg: github.com/cilium/statedb/part
│ before │ after │
│ sec/op │ sec/op vs base │
_Uint64Map_Sequential_Insert-6 520.3µ ± 4% 446.2µ ± 3% -14.24% (p=0.000 n=10)
_Uint64Map_Sequential_Txn_Insert-6 71.33µ ± 1% 75.09µ ± 1% +5.27% (p=0.000 n=10)
_Uint64Map_Random_Insert-6 624.1µ ± 3% 413.7µ ± 1% -33.72% (p=0.000 n=10)
_Uint64Map_Random_Txn_Insert-6 111.5µ ± 2% 117.3µ ± 2% +5.16% (p=0.000 n=10)
_Insert_RootOnlyWatch-6 74.87µ ± 3% 75.92µ ± 3% ~ (p=0.105 n=10)
_Get-6 16.12µ ± 1% 16.22µ ± 1% +0.60% (p=0.027 n=10)
SmallWriteTxn/updates_1-6 2.241µ ± 1% 1.961µ ± 1% -12.47% (p=0.000 n=10)
SmallWriteTxn/updates_2-6 3.487µ ± 2% 2.760µ ± 1% -20.85% (p=0.000 n=10)
SmallWriteTxn/updates_4-6 5.899µ ± 2% 4.495µ ± 4% -23.80% (p=0.000 n=10)
SmallWriteTxn/updates_8-6 10.614µ ± 2% 7.796µ ± 6% -26.55% (p=0.002 n=10)
SmallWriteTxn/updates_16-6 20.08µ ± 1% 14.23µ ± 3% -29.12% (p=0.000 n=10)
geomean 31.53µ 26.93µ -14.59%
│ before │ after │
│ B/op │ B/op vs base │
_Uint64Map_Sequential_Insert-6 2.106Mi ± 0% 1.716Mi ± 0% -18.54% (p=0.000 n=10)
_Uint64Map_Sequential_Txn_Insert-6 84.33Ki ± 0% 88.58Ki ± 0% +5.04% (p=0.000 n=10)
_Uint64Map_Random_Insert-6 2.402Mi ± 0% 1.352Mi ± 0% -43.70% (p=0.000 n=10)
_Uint64Map_Random_Txn_Insert-6 116.2Ki ± 1% 114.6Ki ± 1% -1.42% (p=0.000 n=10)
_Insert_RootOnlyWatch-6 69.83Ki ± 0% 74.02Ki ± 0% +6.00% (p=0.000 n=10)
_Get-6 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10)
SmallWriteTxn/updates_1-6 4.681Ki ± 0% 3.548Ki ± 0% -24.20% (p=0.000 n=10)
SmallWriteTxn/updates_2-6 6.994Ki ± 0% 4.732Ki ± 0% -32.34% (p=0.000 n=10)
SmallWriteTxn/updates_4-6 11.594Ki ± 0% 7.090Ki ± 0% -38.85% (p=0.000 n=10)
SmallWriteTxn/updates_8-6 20.69Ki ± 0% 11.75Ki ± 0% -43.21% (p=0.000 n=10)
SmallWriteTxn/updates_16-6 38.52Ki ± 0% 20.89Ki ± 0% -45.76% (p=0.000 n=10)
The small write transaction allocation counts are unchanged.
Reconciler benchmark, 1,000,000 objects and incremental batch size
1000. Parent and changed binaries were alternated; results are the
best throughput of three runs per batch size:
batch before obj/s after obj/s change before alloc after alloc change
1 160071.01 183551.30 +14.67% 9946MB 8237MB -17.18%
10 620465.78 594884.80 -4.12% 1793MB 1649MB -8.03%
100 445345.12 532524.79 +19.57% 970MB 969MB -0.10%
1000 663607.42 697113.34 +5.05% 888MB 901MB +1.46%
AIL:3
Signed-off-by: Jussi Maki <jussi@isovalent.com>
joamaki
force-pushed
the
pr/joamaki/node64-node128
branch
from
August 25, 2026 11:38
075436f to
fb4ff86
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace Node48's 256-byte reverse index with packed, sorted child arrays indexed by a 256-bit bitmap and popcount. Add Node64 and Node128 stages between Node16 and Node256, including symmetric promotion and demotion paths. With these node sizes we'll have smaller allocations and cloning of the nodes is faster as there's less to copy. The Node64+Node128 is slightly less optimal for lookups than having just Node48 but it significantly reduces allocations.
Allocator sizes
On 64-bit Go:
Thus Node64 (with bitmaps) has 16 more child slots than Node48 while using a smaller allocator class. Node128 delays the much larger Node256 allocation until fanout exceeds 128.
Benchmarks
partbenchmarks10 runs, 300 ms benchtime:
The small write transaction allocation counts are unchanged.
Reconciler benchmark
1,000,000 objects and incremental batch size 1000. Parent and changed binaries were alternated; results are the best throughput of three runs per batch size:
AIL:3