Make the chunk packet deflate level configurable - #203
Merged
Conversation
Contributor
|
We can also try introducing the zstd compression function instead of deflate, which is disabled by default. zstd compresses data better and more efficiently |
Member
Author
It seems to be a good idea indeed. As its ortogonal to the actual change, i might test on another PR later |
The level has been hardcoded since Spigot lowered it from 6 to 4, with a comment saying a higher one slows things down too much and nothing saying what it costs to go lower. It is a straight CPU for bandwidth trade made on the netty threads, and which side is scarce is a property of the host, not of the server software. crucible.optimization.chunkCompressionLevel now sets it for the bulk packet, read when a netty thread builds its pooled deflater - a restart-time decision, so nothing is reconfigured per packet. The single chunk packet keeps its fixed 4; its buffer is sized at exactly the input, so it has no room for a level that does not shrink. Out of range values are clamped rather than thrown, since the call site is the packet encoder and an exception there kills the connection instead of logging. -1 passes through unclamped: it is zlib's own default level, not a value under the floor, and folding it into NO_COMPRESSION would answer a request for the default with no compression at all. The output buffer is sized with zlib's compressBound instead of input + 100. Low levels are reachable now, and at level 0 deflate stores rather than compresses and its output is larger than its input; deflate() writes what fits, returns, and the packet ships a truncated stream with no error anywhere - the client ignores the inflate return as well, so it arrives as empty chunks rather than as a failure. input + 100 runs out at eight full chunks in one packet. deflate() is now checked for finished(), so a bound that ever stopped covering its output would fail loudly instead of shipping that same silent truncation. The default moves to 2 on measurements taken over real chunk traffic to a connected client, every level compressing the identical payload. Going up from 4 is close to worthless: level 9 costs 14x the CPU for 10% fewer bytes, and 8 and 9 land within 0.2% of each other. Going down is where the trade is real: 2 gives back 45% of the CPU for 28% more bytes, and chunk streaming is bursty - concentrated in the seconds after a join or a teleport, exactly when a server is least able to spare CPU. Two runs, vanilla and a 15-mod server, agree on the shape.
EverNife
force-pushed
the
feature/chunk-send-deflate
branch
from
September 1, 2026 20:09
dd700b8 to
6e3e5ae
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The deflate level for chunk packets has been hardcoded since Spigot lowered it from 6 to 4, with a
comment saying a higher one slows things down too much and nothing saying what it costs to go lower.
It is a straight CPU-for-bandwidth trade made on the netty threads, and which of the two is scarce
is a property of the host, not of the server software.
Change
crucible.optimization.chunkCompressionLevelsets the level, and the default moves from 4 to 2.The level is read when a netty thread builds its pooled deflater, so editing it takes effect on the
next restart — the deflater is not reconfigured per packet.
The output buffer is also sized with zlib's
compressBoundinstead ofinput + 100. Low levels arereachable now, and at level 0 deflate stores rather than compresses, so its output is larger than
its input;
deflate()would handle a short buffer by writing what fits and returning quietly, andthe packet would ship a truncated stream with no error anywhere. Out-of-range config values are
clamped rather than thrown, since the call site is the packet encoder and an exception there kills
the connection instead of logging.
Evidence
Every level compressing the identical payload, so the comparison is over the same bytes rather
than two different flights over the world. Both runs are real packets written to the socket for a
HeadlessMC Forge 1.7.10 client, compressed on the netty threads by
writePacketData.Vanilla world — 89 packets, 441 chunks, 21,96MB raw
Modded world — 36 packets, 177 chunks, 7,91MB raw
Crucible with 15 mods loaded, CustomNPC-Plus among them.
Why the default moves to 2
The two runs agree on the shape, which is the useful part: modding the server changed the absolute
sizes but not the curve.
Going up from 4 is close to worthless. Level 9 buys 10% fewer bytes for 14x the CPU, and levels
8 and 9 produce output within 0,2% of each other — the second is pure waste. Whatever argument
existed for 6 does not survive the measurement.
Going down is where the trade is real. Level 2 costs 28% more bytes and gives back 45% of the
CPU. On the modded run that is 267KB to 344KB for a 177-chunk join, against 1,02ms to 0,56ms of
netty time per packet. Chunk streaming is bursty — it is concentrated in the seconds after a join or
a teleport, exactly when a server is least able to spare CPU — and bandwidth is the resource most
hosts have to spare. Level 1 is barely cheaper than 2 (0,50x versus 0,55x) for another 8% of bytes,
so 2 is the better end of that step.
Anyone whose constraint is the other way round sets the value back to 4, or higher, and restarts.