Skip to content

Make the chunk packet deflate level configurable - #203

Merged
EverNife merged 1 commit into
CrucibleMC:stagingfrom
EverNife:feature/chunk-send-deflate
Sep 1, 2026
Merged

Make the chunk packet deflate level configurable#203
EverNife merged 1 commit into
CrucibleMC:stagingfrom
EverNife:feature/chunk-send-deflate

Conversation

@EverNife

Copy link
Copy Markdown
Member

Stacks on the other two: this branch merges feature/fast-chunk-send and
fix/tile-entities-in-range, so until those land the diff here shows their commits too.
Only the last commit is new work.

Problem

The deflate level for chunk packets has been hardcoded since Spigot lowered it from 6 to 4, with a
comment saying a higher one slows things down too much and nothing saying what it costs to go lower.
It is a straight CPU-for-bandwidth trade made on the netty threads, and which of the two is scarce
is a property of the host, not of the server software.

Change

crucible.optimization.chunkCompressionLevel sets the level, and the default moves from 4 to 2.
The level is read when a netty thread builds its pooled deflater, so editing it takes effect on the
next restart — the deflater is not reconfigured per packet.

The output buffer is also sized with zlib's compressBound instead of input + 100. Low levels are
reachable now, and at level 0 deflate stores rather than compresses, so its output is larger than
its input; deflate() would handle a short buffer by writing what fits and returning quietly, and
the packet would ship a truncated stream with no error anywhere. Out-of-range config values are
clamped rather than thrown, since the call site is the packet encoder and an exception there kills
the connection instead of logging.

Evidence

Every level compressing the identical payload, so the comparison is over the same bytes rather
than two different flights over the world. Both runs are real packets written to the socket for a
HeadlessMC Forge 1.7.10 client, compressed on the netty threads by writePacketData.

Vanilla world — 89 packets, 441 chunks, 21,96MB raw

level deflated ratio ms/packet vs level 4 (bytes) vs level 4 (cpu)
0 21,97MB 1,00x 0,073 +2658,3% 0,06x
1 1,08MB 20,43x 0,633 +35,0% 0,50x
2 1,02MB 21,61x 0,691 +27,6% 0,55x
3 998,4KB 22,53x 0,884 +22,4% 0,70x
4 815,5KB 27,58x 1,264 1,00x
5 776,6KB 28,96x 1,563 -4,8% 1,24x
6 761,2KB 29,55x 2,276 -6,7% 1,80x
7 748,4KB 30,05x 2,974 -8,2% 2,35x
8 735,9KB 30,56x 7,568 -9,8% 5,99x
9 734,2KB 30,63x 18,201 -10,0% 14,40x

Modded world — 36 packets, 177 chunks, 7,91MB raw

Crucible with 15 mods loaded, CustomNPC-Plus among them.

level deflated ratio ms/packet vs level 4 (bytes) vs level 4 (cpu)
0 7,92MB 1,00x 0,059 +2931,1% 0,06x
1 364,5KB 22,24x 0,502 +36,3% 0,49x
2 344,3KB 23,54x 0,562 +28,7% 0,55x
3 328,6KB 24,66x 0,703 +22,9% 0,69x
4 267,4KB 30,31x 1,021 1,00x
5 253,8KB 31,93x 1,323 -5,1% 1,30x
6 249,1KB 32,53x 1,882 -6,8% 1,84x
7 244,9KB 33,10x 2,343 -8,4% 2,29x
8 240,7KB 33,67x 5,680 -10,0% 5,56x
9 240,2KB 33,74x 13,199 -10,2% 12,92x

Why the default moves to 2

The two runs agree on the shape, which is the useful part: modding the server changed the absolute
sizes but not the curve.

Going up from 4 is close to worthless. Level 9 buys 10% fewer bytes for 14x the CPU, and levels
8 and 9 produce output within 0,2% of each other — the second is pure waste. Whatever argument
existed for 6 does not survive the measurement.

Going down is where the trade is real. Level 2 costs 28% more bytes and gives back 45% of the
CPU. On the modded run that is 267KB to 344KB for a 177-chunk join, against 1,02ms to 0,56ms of
netty time per packet. Chunk streaming is bursty — it is concentrated in the seconds after a join or
a teleport, exactly when a server is least able to spare CPU — and bandwidth is the resource most
hosts have to spare. Level 1 is barely cheaper than 2 (0,50x versus 0,55x) for another 8% of bytes,
so 2 is the better end of that step.

Anyone whose constraint is the other way round sets the value back to 4, or higher, and restarts.

@gravit0

gravit0 commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

We can also try introducing the zstd compression function instead of deflate, which is disabled by default. zstd compresses data better and more efficiently

@EverNife

Copy link
Copy Markdown
Member Author

We can also try introducing the zstd compression function instead of deflate, which is disabled by default. zstd compresses data better and more efficiently

It seems to be a good idea indeed.
Tested here 'zstd / lz4' might need a client side mod. But zlib-ng / libdeflate might work.

As its ortogonal to the actual change, i might test on another PR later

The level has been hardcoded since Spigot lowered it from 6 to 4, with a
comment saying a higher one slows things down too much and nothing saying what
it costs to go lower. It is a straight CPU for bandwidth trade made on the
netty threads, and which side is scarce is a property of the host, not of the
server software.

crucible.optimization.chunkCompressionLevel now sets it for the bulk packet,
read when a netty thread builds its pooled deflater - a restart-time decision,
so nothing is reconfigured per packet. The single chunk packet keeps its fixed
4; its buffer is sized at exactly the input, so it has no room for a level that
does not shrink. Out of range values are clamped rather than thrown, since the
call site is the packet encoder and an exception there kills the connection
instead of logging. -1 passes through unclamped: it is zlib's own default
level, not a value under the floor, and folding it into NO_COMPRESSION would
answer a request for the default with no compression at all.

The output buffer is sized with zlib's compressBound instead of input + 100.
Low levels are reachable now, and at level 0 deflate stores rather than
compresses and its output is larger than its input; deflate() writes what fits,
returns, and the packet ships a truncated stream with no error anywhere - the
client ignores the inflate return as well, so it arrives as empty chunks rather
than as a failure. input + 100 runs out at eight full chunks in one packet.
deflate() is now checked for finished(), so a bound that ever stopped covering
its output would fail loudly instead of shipping that same silent truncation.

The default moves to 2 on measurements taken over real chunk traffic to a
connected client, every level compressing the identical payload. Going up from
4 is close to worthless: level 9 costs 14x the CPU for 10% fewer bytes, and 8
and 9 land within 0.2% of each other. Going down is where the trade is real: 2
gives back 45% of the CPU for 28% more bytes, and chunk streaming is bursty -
concentrated in the seconds after a join or a teleport, exactly when a server
is least able to spare CPU. Two runs, vanilla and a 15-mod server, agree on the
shape.
@EverNife
EverNife force-pushed the feature/chunk-send-deflate branch from dd700b8 to 6e3e5ae Compare September 1, 2026 20:09
@EverNife
EverNife merged commit 47a2c67 into CrucibleMC:staging Sep 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants