stream: cut per-chunk overhead in web streams pipelines

[anonrig]

What

Reduces per-chunk overhead across common web streams pipelines (fetch body decompression/upload, TextDecoderStream/TextEncoderStream transcoding, Readable.toWeb/Writable.toWeb piping), targeting workloads that move large payloads in small (e.g. 4 KiB) chunks.

Three changes:

1. CompressionStream/DecompressionStream: process chunks without threadpool round trips.
   Previously these classes wrapped a zlib stream.Duplex in the web streams adapters, so every written chunk was dispatched to the threadpool and its completion observed on a later event loop turn. For a 64 MiB body in 4 KiB chunks that is 16384 threadpool round trips, which dominates the cost of the stream. The classes are now built on a TransformStream that drives the raw zlib/brotli handle synchronously (writeSync), processing inputs larger than 64 KiB in slices with an event-loop turn in between so huge chunks cannot block the loop for their full duration. Output is emitted in up to 64 KiB chunks, either zero-copy or right-sized copies.

2. TextEncoderStream: fast-path chunk encoding.
   The encode-and-enqueue algorithm was a literal transcription of the spec's per-code-unit loop (single-character string extraction, charCodeAt, string accumulator append for every code unit). Its only observable effects are the surrogate hand-off at chunk boundaries and U+FFFD replacement, which TextEncoder.encode() already performs, so the loop is replaced with explicit chunk-boundary handling plus a single encode() call.

3. Readable.toWeb(): avoid copying buffers that are the sole view of their ArrayBuffer.
   Chunks were always copied before being enqueued to avoid exposing Buffer pool slices. Buffers that own their entire ArrayBuffer (as fs streams produce for every read) cannot alias pooled memory and are now exposed zero-copy.

Why

Pipelines like fetch() → DecompressionStream → TextDecoderStream → for await, fs.createReadStream() → CompressionStream → fetch POST, and fs → TextDecoderStream → TextEncoderStream → fs spend the majority of their time in per-chunk machinery rather than in the codecs doing the actual work.

(Benchmark results from an isolated machine to be added after the full validation run completes; preliminary measurements show ~13x on the TextEncoderStream stage and ~3x on the DecompressionStream stage for 4 KiB chunks.)

Testing

• New/existing: test/parallel/test-whatwg-webstreams-compression.js, test-webstreams-compression-bad-chunks.js, test-webstreams-decompression-reject-trailing.js, test-webstreams-compression-buffer-source.js, test-compression-decompression-stream.js, test-whatwg-webstreams-encoding.js, adapters tests.
• WPT: compression, encoding, streams suites.
• Adds benchmark/webstreams/compression.js and benchmark/webstreams/encoding.js.

  

[cursor]

Large chunks yield without copying input

High Severity

Inputs larger than kInputSliceSize are processed in slices with an await between them, but each slice is only a view of the original chunk. After transform yields, other tasks can detach or mutate that ArrayBuffer before later slices run, so remaining data is compressed from invalid or changed memory. The WPT detach cases (compression-with-detach, decompression-with-detach) expect the write to keep working after the source is detached.

Additional Locations (1)

• lib/internal/webstreams/compression.js#L227-L235

 

^{Reviewed by Cursor Bugbot for commit 8019292. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 936186f. Configure here.}