Db/mapped source lever

[derekwbrown]

Stop V8 retaining a private copy of the app source (mapped-source lever)

DRAFT — not ready to merge. More commits are coming; see "What is still missing" below. The most
important of these is hash verification of the mapped bytes, which is a hard prerequisite for shipping
this at all.

The problem

js2bin stores the app as base64(brotli(source)) inside the binary. At every process start the default
bootstrap base64-decodes it, brotli-decompresses it into a large Buffer, .toString()s that into a JS
string, and then concatenates that into a template literal in order to prepend the cluster preamble.

The resulting string is what V8 keeps alive for the entire process lifetime, because it needs the source
text to lazily compile functions that have not run yet. For Cribl's ~26 MB bundle that is ~26 MB of
private heap in every process, plus a boot peak of roughly three times that while the copies
coexist.

On a two-process Cribl Edge node that is a little over 20% of the node's total memory footprint, and it
is pure overhead — the same bytes, duplicated per process, never mutated.

The fix

Keep the payload compressed in the binary, but materialise the decompressed source to a file once,
map it read-only, and hand V8 an external one-byte string over the mapping. The bytes become
file-backed and shared between every process instead of private to each, and the binary does not grow.

Two bootstraps are added, both opt-in at --ci time:

• JS2BIN_MAPPED_SOURCE=1 → src/_third_party_main_mapped_source.js. The lever described above.
  Requires the new native binding. Falls back to the historical in-heap path on any failure rather
  than refusing to boot.
• JS2BIN_UNCOMPRESSED_SOURCE=1 → src/_third_party_main_uncompressed_source.js. Stores the
  payload as raw source instead. Node's own builtin loader already hands V8 a file-backed external
  one-byte string, so this needs no C++ at all — but the binary grows by the source size, and the win
  is lost under overlay because an overlay delivers a fresh bundle that goes back through the in-heap
  path.

Plus the native support:

• src/patch/22.22.2/node_mapped_source.cc.patch — adds internalBinding('js2bin') .mapFileAsExternalString(path), returning a v8::String::NewExternalOneByte over a read-only file
  mapping (Windows CreateFileMapping/MapViewOfFile, POSIX mmap). Public V8 API only, no Node
  internal helpers, to keep the patch small and portable across Node versions.
• src/patch/22.22.2/node_binding_mapped_source.cc.patch — adds V(js2bin) to the compile-time
  binding registry. This is required: NODE_BINDING_CONTEXT_AWARE_INTERNAL alone does not register
  a binding on Node 22, which cost a debugging cycle (No such binding: js2bin).
• src/NodeBuilder.js — the env-var flags, bootstrap selection, raw payload return, and the
  comment-padding fill for the uncompressed variant.

The native patches are applied unconditionally, so every binary carries the capability even though
only the mapped bootstrap uses it.

Measured result

Windows Server 2019, Cribl Edge, two processes, 1020 s settle, three settled runs per arm, both arms
on the same box built from the same source tag, differing only in cribl.exe:

clean19, total Private Working Set	run 1	run 2	run 3	mean
stock v4.19.0	248.2	246.6	249.8	248.2
mapped v4.19.0	193.5	193.6	198.5	195.2

−53.0 MB PWS (−21.4%), boot −42 ms, no change in binary size.

Reproduced on current dev (bundle 6.2% larger): stock ~256.8, mapped 202.1, so the win grows to
~54.7 MB — it tracks the source size, as the mechanism predicts.

Working Set moves only −10.2 MB while private drops 53.0, and both processes gain ~24.7 MB of shared
memory. That is the signature of pages being re-accounted as file-backed, not freed — which is exactly
what this change is supposed to do.

Heap snapshots confirm the mechanism in isolation: strings drop 35.54 → 9.30 MiB (−26.24) while code,
object shapes, arrays, closures and node count are all identical to within noise. The retained string is
precisely one copy of the source.

Why the win is slightly larger than the retained source

At v4.19.0 the source is 24.77 MiB, so two processes should give −49.5 MB against a measured −53.0 —
the mechanism explains ~93%, leaving ~3.5 MB.

The likely remainder is resident heap slack. The old path transiently materialises ~50 MB of strings
per process (decompressed Buffer → .toString() → template concatenation); V8 grows its heap to hold
them, and those pages stay resident and private after GC, because Private Working Set charges committed
resident pages rather than live set. The mapped path never builds that peak, so it never grows the heap.

This is inferred, not measured. A heap-total-vs-PWS comparison on the two arms would settle it.

Two invariants that must not be "tidied up"

1. Never slice the mapped string. A SlicedString over an external parent can be flattened by V8 at
   compile time, silently reinstating the private copy this exists to remove.
2. Never concatenate it. This is why the cluster preamble runs as real code in the bootstrap instead
   of being textually prepended, and why the uncompressed placeholder is padded with a trailing line
   comment rather than NULs.

Both are easy to undo by accident during a cleanup pass, and both fail silently — the binary still works,
it just quietly costs the memory again.

About the second commit

82262e4 is titled "NEEDS REVIEW". That title is now stale — it has been reviewed and it should be
kept. It re-anchors node.cc.patch's _third_party_main hook and moves it after the
env->worker_context() check. The original message worried that the reordering was an unjustified
behaviour change to existing js2bin functionality. It is not; it is a bug fix:

• Upstream Node 22.22.2 contains no _third_party_main hook at all — verified against pristine
  node.cc from the 22.22.2 tarball, where the only match in StartExecution is the worker_context()
  check. js2bin injects the whole block, so where it goes is js2bin's choice, not a modification of
  Node's own control flow.
• Worker threads run through StartExecution too — that is precisely why upstream's
  worker_context() check lives there. Anything injected above it therefore also fires for worker
  threads, and Exists("_third_party_main") is true in every Environment because the builtin is
  compiled into the binary.
• So with the original placement, any new Worker() inside a js2bin binary re-enters the app
  bootstrap instead of internal/main/worker_thread: the worker never loads its designated script and
  instead boots a second copy of the whole app in-thread.
• Cribl reaches this. The bundle carries LOCALCOMPUTE_ENTRY_ADAPTER_SOURCE, worker glue that the
  app-platform LocalCompute engine runs via new Worker(bundleFilePath, { workerData: {...} }).

Reviewers may reasonably ask for it as a separate PR, since it fixes a pre-existing bug and is not part
of the lever. Happy to split it.

Cross-platform status

The implementation is cross-platform by construction, but has only ever been built and measured on
Windows:

• the C++ has a complete POSIX branch (open/fstat/mmap(PROT_READ, MAP_PRIVATE)/close) behind
  #if defined(_WIN32), with correctly gated headers;
• the patches apply unconditionally, so a Linux or macOS --ci build already attempts to compile it;
• the bootstrap JS is platform-agnostic (path.join, os.tmpdir(), fs);
• v8::String::NewExternalOneByte is public and identical across platforms.

The POSIX path has never been compiled. Two things to know before anyone relies on it:

1. On Linux this win is invisible in RSS. Linux RSS counts shared pages in full in every process that
   maps them, so two processes sharing one 26 MB mapping still report 26 MB each. It shows only in
   USS/PSS — the analogue of Windows Private Working Set. Measuring a sharing win with RSS will
   report zero.
2. The missing hash verification is materially worse on POSIX. The cache path is predictable
   (js2bin-src-<app>-<payloadLength>.js) and os.tmpdir() is world-writable /tmp, so an
   unprivileged local user could pre-create that file and have their source compiled and executed by a
   root-run process. Windows' per-account temp directories blunt this; on Linux it is a straightforward
   local privilege escalation.

What is still missing — further commits to come

This branch is a measured spike, not a shippable feature. Still to come:

1. Hash verification (the blocker). Embed a SHA-256 of the uncompressed source in the signed binary
   and verify the mapped bytes before compiling. That, and not filesystem permissions, is what makes
   the cache file's writability irrelevant: tampering yields a mismatch and a refusal to run. ~26 MB of
   SHA-256 is tens of milliseconds. Nothing here should ship before this lands, and it is doubly
   required for any POSIX build.
2. Remove the visible path once mapped — FILE_FLAG_DELETE_ON_CLOSE on Windows, unlink-after-mmap
   on POSIX — so no readable copy of the application source is left on disk.
3. A fixed, known cache directory instead of os.tmpdir(), which is per-account and therefore
   produces one copy per account (an interactive run and the service account currently write two), plus
   ACL hardening on that directory.
4. Key the cache filename on the source digest rather than the compressed payload's length.
5. Overlay composition. The bootstrap selection is
   uncompressedSource ? … : mappedSource ? … : enableOverlay ? … : default, so JS2BIN_MAPPED_SOURCE=1
   replaces the overlay bootstrap rather than composing with it — the binary is not overlay-capable at
   all. The concept survives overlay (the payload stays compressed, and an overlay simply replaces that
   payload), but shipping it overlay-capable means merging the mapping logic into
   _third_party_main_overlay.js. Unbuilt and unmeasured.
6. Build and measure on Linux and macOS, per the section above.

Testing done

• A wiped tree (rm -rf build/node-v22.22.2) applies every patch cleanly — node.cc patched twice,
  node_binding.cc patched, no failed hunks, no reversed-patch detection — and the resulting binary
  boots: 2 processes, {"status":"healthy"}, zero MODULE_NOT_FOUND / Cannot find module /
  unhandledRejection.
• Confirmed at runtime that the lever actually engages rather than silently falling back: no
  js2bin: falling back to in-heap source on the debug channel, and the mapped cache file is present
  with the expected size.
• The measurement above, three settled runs per arm, on one box, with the arms differing only in
  cribl.exe (verified by hash).

[derekwbrown]

Overlay composition resolved — this was the last remaining unknown. 587bd7e

The mapped bootstrap used to displace the overlay bootstrap, so JS2BIN_MAPPED_SOURCE=1 silently produced a binary that wasn't overlay-capable at all, even with --enable-overlay passed and the keys embedded. The two are orthogonal — overlay decides which source runs, mapped-source decides how it reaches V8 — and they now compose.

Two parts weren't a copy-paste merge, and both are worth a reviewer's eye:

• The cache key was a latent bug. It keyed on the embedded payload's length. Under overlay the compiled source can be a different payload of the same length, so a node could map and execute the wrong source. It's now a SHA-256 of whichever payload is active — still computable without decompressing, so an existing cache file still short-circuits the work. (Not the integrity check; that still has to be a digest embedded in the signed binary.)
• The two failure modes are kept distinct. A decompression failure of an overlay bundle demotes to the embedded payload, as before. A mapping failure must not, or an unrelated filesystem or binding problem would silently discard a validly signed overlay.

Verified on a purpose-built binary across 7 scenarios — no overlay, valid signed overlay, wrong signing key, valid signature over a tampered body, zero-length signature, encrypted with the right key, encrypted with the wrong key. Two different test apps, so the compiled source is identifiable from stdout rather than inferred, and every scenario also asserts nothing fell back to the in-heap path (a silent fallback would otherwise read as a pass). All 7 pass, the two payloads land on distinct cache files, and every overlay rejection still fires. The overlay region of the new file is byte-identical to _third_party_main_overlay.js, checked mechanically, so overlay behaviour is unchanged.

And the memory win survives overlay. Real Cribl binary, overlay-capable, 3 settled runs on Windows: 199.9 MB private working set vs 202.1 MB for the non-overlay mapped build — indistinguishable within the spreads, i.e. overlay-capability costs nothing.

Still to do — all knowns, no unknowns left: hash verification of the mapped bytes (the blocker; nothing ships before it), delete-on-close so no readable copy of the source is left on disk, a fixed ACL-hardened cache directory instead of os.tmpdir(), digest-keyed cache invalidation, and building/measuring on Linux and macOS (the POSIX branch is written but has never been compiled).

Two side findings outside this PR's scope:

• js2bin --ci is broken on Windows. runCommand spawns vcbuild.bat without shell: true, and Node ≥18.20 refuses to spawn .bat/.cmd without a shell (CVE-2024-27980 mitigation) → spawn EINVAL. This is not the NoDefaultCurrentDirectoryInExePath issue I previously blamed it on — I unset that and still got EINVAL. I worked around it by driving vcbuild.bat directly, but js2bin probably wants the shell: true fix.
• A latent hazard for this feature: Node stores a builtin as UTF-16 if its source has any non-ASCII byte. A single non-ASCII byte in the app bundle would flip the payload builtin to two-byte and silently break the one-byte-external-string premise the whole lever rests on. Cribl's bundle is 100% ASCII today; this deserves a build-time assertion.