Merge pull request #5 from clojurewasm/develop/zig-016-migration

Migrate to Zig 0.16.0 + zwasm v1.11.0

Author: chaploud

.claude/CLAUDE.md

@@ -1,6 +1,6 @@
 # ClojureWasm
 
-Full-scratch Clojure implementation in Zig 0.15.2. Behavioral compatibility target.
+Full-scratch Clojure implementation in Zig 0.16.0. Behavioral compatibility target.
 Reference: ClojureWasmBeta (via add-dir). Design: `.dev/future.md`. Memo: `.dev/memo.md`.
 
 ## Language Policy
@@ -287,10 +287,10 @@ Notes: `"JVM interop"`, `"builtin (upstream is pure clj)"`, `"stub"`, `"UPSTREAM
 See `.claude/rules/java-interop.md` (auto-loads on .clj/analyzer/builtin edits).
 Do NOT skip features that look JVM-specific — try Zig equivalents first.
 
-## Zig 0.15.2 Pitfalls
+## Zig 0.16.0 Pitfalls
 
 Check `.claude/references/zig-tips.md` first, then Zig stdlib at
-`/opt/homebrew/Cellar/zig/0.15.2/lib` or Beta's `docs/reference/zig_guide.md`.
+`/opt/homebrew/Cellar/zig/0.16.0/lib` or Beta's `docs/reference/zig_guide.md`.
 
 ## References
 

.claude/references/zig-tips.md

@@ -1,4 +1,4 @@
-# Zig 0.15.2 Tips & Pitfalls
+# Zig 0.16.0 Tips & Pitfalls
 
 Common mistakes and workarounds discovered during development.
 
@@ -23,15 +23,15 @@ try list.append(allocator, 42);        // allocator passed per call
 
 ```zig
 var buf: [4096]u8 = undefined;
-var writer = std.fs.File.stdout().writer(&buf);
+var writer = std.Io.File.stdout().writer(io_default.get(), &buf);
 const stdout = &writer.interface;
 // ... write ...
 try stdout.flush();  // don't forget
 ```
 
 ## Use std.Io.Writer (type-erased) instead of anytype for writers
 
-In 0.15.2, `std.Io.Writer` is the new type-erased writer.
+In 0.16.0, `std.Io.Writer` is the new type-erased writer.
 `GenericWriter` and `fixedBufferStream` are deprecated.
 
 Prefer `*std.Io.Writer` over `anytype` for writer parameters.

.dev/CONTRIBUTING.md

@@ -30,7 +30,7 @@ the project direction.
 
 ### Prerequisites
 
-- [Zig 0.15.2](https://ziglang.org/download/) (exact version required)
+- [Zig 0.16.0](https://ziglang.org/download/) (exact version required)
 - macOS Apple Silicon (primary development platform)
 
 ### Build & Test

.dev/archive/zig-016-migration.md

@@ -0,0 +1,170 @@
+# Zig 0.15.2 → 0.16.0 Migration — Working Document
+
+**Status**: In progress (branch `develop/zig-016-migration`)
+**Baseline**: commit `8bfbf5b` (`pre-zig-016` in `bench/history.yaml`)
+**Target zwasm**: v1.11.0 (released)
+
+This is a **temporary** working doc. Delete after Phase 7 completion (or
+move learnings into `.dev/decisions.md` D## entry and `.dev/zig-tips.md`).
+
+## Phase -1: zwasm Dependency Audit
+
+### Existing -Dwasm Infrastructure (already in place)
+
+`build.zig` already supports `-Dwasm=true|false` (default true) with full
+conditional gating. Zig source files using zwasm are already wrapped in
+`if (enable_wasm) ...` patterns. **No new build-side gating needed**.
+
+- `build.zig:10` — `-Dwasm` flag definition
+- `build.zig:17` — propagated to `build_options.enable_wasm`
+- `build.zig:22-37` — `zwasm_mod` / `zwasm_native_mod` conditional dep
+- `build.zig:44,59,84` — conditional `addImport("zwasm", ...)`
+- `build.zig:115` — `wasm32-wasi` target does NOT depend on zwasm (correct)
+
+Source-side gates already present:
+
+- `src/runtime/wasm_types.zig:20` — `const zwasm = if (enable_wasm) @import("zwasm") else struct {};`
+- `src/lang/lib/cljw_wasm.zig:16` — `.enabled = wasm_types.enable_wasm`
+  (NamespaceDef level — `cljw.wasm` namespace is unregistered when disabled)
+
+### Verified working under `-Dwasm=false` on Zig 0.15.2
+
+- `zig build -Dwasm=false` → exit 0 ✓
+- `zig build test -Dwasm=false` → exit 0 ✓ (Zig unit tests auto-skip)
+- `zig build -Doptimize=ReleaseSafe -Dwasm=false` → exit 0 ✓
+
+### What FAILS under `-Dwasm=false` (needs Phase 0 work)
+
+#### 1. E2E Wasm tests (test/e2e/wasm/, 6 files)
+
+```
+test/e2e/wasm/01_basic_test.clj
+test/e2e/wasm/02_tinygo_test.clj
+test/e2e/wasm/03_host_functions_test.clj
+test/e2e/wasm/04_module_objects_test.clj
+test/e2e/wasm/05_wit_test.clj
+test/e2e/wasm/06_multi_module_test.clj
+```
+
+All start with `(require '[cljw.wasm :as wasm])` → fail with "Could not
+locate cljw.wasm on load path" because the namespace is not registered.
+
+#### 2. Test runners that unconditionally invoke wasm tests/benchmarks
+
+| Runner | What breaks |
+|---|---|
+| `test/run_all.sh` | step "e2e tests (wasm)" calls `bash test/e2e/run_e2e.sh` (no dir filter) → all e2e dirs incl. wasm |
+| `test/e2e/run_e2e.sh` | no `--no-wasm` flag; finds all `*_test.clj` recursively |
+| `bench/wasm_bench.sh` | runs wasm benchmarks via TinyGo .wasm modules — needs cljw.wasm |
+| `bench/run_bench.sh` | runs benchmarks 21-25, 28-31 (9 wasm benchmarks) under `bench/benchmarks/` |
+
+#### 3. Wasm benchmarks (bench/benchmarks/)
+
+```
+21_wasm_load     22_wasm_call    23_wasm_memory  24_wasm_fib     25_wasm_sieve
+28_wasm_tgo_fib  29_wasm_tgo_tak 30_wasm_tgo_arith 31_wasm_tgo_sieve
+```
+
+### Source files referencing WasmModule type (for migration awareness)
+
+Already-gated, but require io threading in Phase 2:
+
+- `src/runtime/wasm_types.zig` — main bridge
+- `src/runtime/wasm_wit_parser.zig` — WIT parser, uses @embedFile (no io)
+- `src/runtime/value.zig` — `.wasm_module` variant
+- `src/runtime/dispatch.zig` — invokeWasmFn dispatch
+- `src/runtime/gc.zig` — WasmModule finalizer registry
+- `src/lang/lib/cljw_wasm.zig` — NamespaceDef
+- `src/lang/lib/cljw_wasm_builtins.zig` — wasm/load, wasm/fn impl
+- `src/engine/vm/vm.zig`, `src/engine/evaluator/tree_walk.zig` — call sites
+- `src/app/repl/nrepl.zig:1427` — `#<WasmModule>` formatter
+- `src/app/deps.zig` — `cljw/wasm-deps` config parsing (test data only)
+
+## Phase 0: Plan
+
+Reduced scope thanks to existing infrastructure:
+
+1. **Add `--no-wasm` flag to test runners**:
+   - `test/run_all.sh` — skip "e2e tests (wasm)" step when `--no-wasm`
+   - `test/e2e/run_e2e.sh` — skip `wasm/` directory when `--no-wasm` (or `WASM_DISABLED=1` env)
+   - `bench/wasm_bench.sh` — early exit with friendly message when `--no-wasm`
+   - `bench/run_bench.sh` — filter out wasm_* benchmarks when `--no-wasm`
+
+2. **Update build.zig.zon**: `minimum_zig_version = "0.16.0"` (will be done as
+   part of Phase 0 commit, even though we still build with 0.15.2 during the
+   actual code migration phases — `.zon` is just metadata until we actually
+   bump zig).
+
+   Actually: defer this to first 0.16-only commit so we can keep building
+   with 0.15.2 during preparatory commits.
+
+3. **Update zwasm dep tag**: defer to Phase 6 (currently v1.9.1, target v1.11.0).
+   Until Phase 6, build with `-Dwasm=false` so the v1.9.1 zwasm dep is never resolved.
+
+4. **Update `.dev/baselines.md`**: relax binary size cap (≤5.0MB → provisional
+   ≤5.5MB during migration, finalize in Phase 7).
+
+5. **Doc/CI sweep**: grep "0.15.2", "Zig 0.15", update to "Zig 0.16.0":
+   - `.claude/CLAUDE.md`
+   - `.dev/baselines.md`, `.dev/decisions.md`, `.dev/references/*.md`
+   - `README.md`
+   - `flake.nix`, `flake.lock` (if present)
+   - `.github/workflows/*.yml` (if present)
+   - `scripts/*.sh`
+
+## Decision: Gating mechanism for test runners
+
+Use **`--no-wasm` flag** on each runner (matches existing `--quick`,
+`--tree-walk` patterns). Avoid env vars to keep behavior explicit.
+
+`test/run_all.sh` will pass `--no-wasm` down to `run_e2e.sh` when invoked
+with `--no-wasm`, and skip `wasm_bench.sh` entirely.
+
+## Open questions for Phase 6 (deferred)
+
+- Does zwasm v1.11.0 export the same module interface as v1.9.1?
+  (`zwasm.WasmModule`, `zwasm.Capabilities`, `zwasm.ImportEntry`, etc.)
+- Are there breaking API changes in zwasm v1.10.0 → v1.11.0 we'd need
+  to absorb at the `wasm_types.zig` bridge?
+- Action: read `~/Documents/MyProducts/zwasm/CHANGELOG.md` v1.10.0 + v1.11.0
+  notes when entering Phase 6.
+
+## Phase 7: Atomic Toolchain Flip (deferred)
+
+Once code migration is complete and tests are green on Zig 0.16.0,
+flip all toolchain pins and version-mention strings in a single commit.
+Doing this earlier creates a window where neither 0.15.2 nor 0.16.0 builds
+cleanly.
+
+Files to update:
+
+| File | Lines | Change |
+|---|---|---|
+| `build.zig.zon` | 11 | `.minimum_zig_version = "0.16.0"` |
+| `flake.nix` | 9, 20, 23, 27, 31, 35, 46, 58 | URLs and comments → 0.16.0 |
+| `flake.lock` | 71 | regenerate via `nix flake update zig-overlay` |
+| `.github/workflows/ci.yml` | 16, 74, 117 | `version: 0.16.0` |
+| `.github/workflows/nightly.yml` | 15, 59 | `version: 0.16.0` |
+| `.github/workflows/release.yml` | 32 | `version: 0.16.0` |
+| `README.md` | 5, 34 | badge + install link |
+| `.claude/CLAUDE.md` | 3, 290, 293 | intro + "Pitfalls" section header + path hint |
+| `.claude/references/zig-tips.md` | 1, 34 | title + body content |
+| `.dev/baselines.md` | 4 | "Zig 0.15.2" → "Zig 0.16.0" platform line |
+| `.dev/CONTRIBUTING.md` | 33 | install requirement |
+| `.dev/references/setup-orbstack.md` | 19, 30 | install + version check |
+| `.dev/references/ubuntu-testing-guide.md` | 56 | describe 0.16-specific behavior if changed |
+| `docs/differences.md` | 10 | runtime row |
+| `.dev/future.md` | 365 | check if still relevant |
+
+DO NOT touch:
+- `.dev/archive/**` — historical phase notes
+- `.dev/decisions.md` D## entries that reference 0.15.2 — these are immutable history
+  (D## about ArenaAllocator.free, @call always_tail, etc. — those decisions remain valid context)
+
+After flip:
+- Re-run `bash test/run_all.sh` (no --no-wasm) on Zig 0.16.0
+- OrbStack Ubuntu validation: `--seed 0` still required? Re-test
+- Update binary size baseline to actual measured value
+- Add D## entry in `.dev/decisions.md` for the migration
+- Add F## in `.dev/checklist.md` for the libc strip follow-up (zwasm W46 equivalent)
+- Delete this file (`.dev/zig-016-migration.md`)

.dev/baselines.md

@@ -1,27 +1,31 @@
 # Non-Functional Baselines
 
-Measured on: 2026-02-25 (v0.4.0 + GPA leak fix + JIT register fix)
-Platform: macOS ARM64 (Apple M4 Pro), Zig 0.15.2
+Measured on: 2026-04-27 (Zig 0.16.0 migration complete; HTTP / nREPL / line
+editor / `cljw build` runtime stubs remain — see Phase 7 follow-ups in
+.dev/checklist.md F##).
+Platform: macOS ARM64 (Apple M4 Pro), Zig 0.16.0
 Binary: ReleaseSafe
 
 ## Profiles
 
 | Profile | Binary | Startup | RSS | Notes |
 |---------|--------|---------|-----|-------|
-| wasm=true (default) | 4.76MB | 4.5ms | 7.9MB | Full feature set |
+| wasm=true (default) | 4.12MB | 4.1ms | 8.2MB | Full feature set, libc linked |
 | wasm=false | (not measured) | — | — | No zwasm dependency |
 
 ## Thresholds
 
-All-Zig migration complete (Phases A-F, C.1). Binary size threshold RESTORED.
-Binary grew ~0.5MB due to embedded Clojure multiline strings (pprint, spec.alpha).
-Phase E optimization target: reduce back toward 4.3MB.
+Post-migration baselines (matched against pre-zig-016 history.yaml entry —
+no benchmark regressed beyond noise; lazy_chain actually improved).
+Binary is currently smaller than 0.15.2 because http_server / nrepl / the
+fancy line editor / `cljw build` were stubbed during the migration; restoring
+them under std.Io.net will likely add several hundred KB back.
 
 | Metric              | Baseline   | Threshold  | Margin | How to measure                              |
 |---------------------|------------|------------|--------|---------------------------------------------|
-| Binary size         | 4.76 MB    | 5.0 MB     | +5%    | `ls -la zig-out/bin/cljw` (after ReleaseSafe build) |
-| Startup time        | 4.5 ms     | 6.0 ms     | 1.3x   | `hyperfine -N --warmup 5 --runs 10 './zig-out/bin/cljw -e nil'` |
-| RSS (light)         | 7.9 MB     | 10 MB      | +27%   | `/usr/bin/time -l ./zig-out/bin/cljw -e nil 2>&1 \| grep 'maximum resident'` |
+| Binary size         | 4.12 MB    | 5.5 MB     | +33%   | `ls -la zig-out/bin/cljw` (after ReleaseSafe build) — slack for stub-restoration + libc |
+| Startup time        | 4.1 ms     | 6.0 ms     | 1.5x   | `hyperfine -N --warmup 5 --runs 10 './zig-out/bin/cljw -e nil'` |
+| RSS (light)         | 8.2 MB     | 10 MB      | +22%   | `/usr/bin/time -l ./zig-out/bin/cljw -e nil 2>&1 \| grep 'maximum resident'` |
 | Benchmark (any)     | see below  | 1.2x       | +20%   | Per-benchmark: `bash bench/run_bench.sh --bench=NAME --runs=10 --warmup=5` |
 
 ## `cljw build` Artifact Baselines (2026-02-20)

.dev/checklist.md

@@ -32,3 +32,15 @@ Target Phase references: see `.dev/roadmap.md` Phase Tracker + Open Checklist It
 | F104 | Profile-guided optimization (extend IC)     | 89     | Extend inline caching beyond monomorphic                                 |
 | F105 | JIT compilation (expand beyond ARM64 PoC)   | 90     | ARM64 hot-loop JIT done (Phase 37.4, D87). Future: x86_64 port, expand beyond integer loops. |
 | F120 | Native SIMD optimization (CW internals)     | 89     | Investigate Zig `@Vector` for CW hot paths. Profile first.               |
+
+## Open follow-ups from the Zig 0.16.0 migration (D111)
+
+| ID   | Item                                                       | Trigger / notes                                                                |
+|------|------------------------------------------------------------|--------------------------------------------------------------------------------|
+| F140 | Restore HTTP server (`cljw.http/run-server`) on `std.Io.net` | Server / Stream / Connection were stubbed in `lang/builtins/http_server.zig` (D111). Reimplement accept loop on `std.Io.net.Server`, plumb `io` through handler dispatch, restore Ring request/response building. Original logic preserved in git history pre-`40d2f20`. |
+| F141 | Restore HTTP client (`cljw.http/get|post|put|delete`)        | `std.http.Client` now has a `.io` field (D111). Wire `io_default.get()` and unstub `doHttpRequest`.                                              |
+| F142 | Restore nREPL server                                       | Whole `src/app/repl/nrepl.zig` (~1818 lines) collapsed to a stub during D111. Needs the same `std.Io.net` + accept loop work as F140 plus `std.posix.poll` replacement; sessions / mutex use `io_default` helpers. |
+| F143 | Restore raw-mode line editor                               | `src/app/repl/line_editor.zig` not yet ported (still on `std.fs.File` + `std.io.fixedBufferStream`). `runRepl` falls through to `runReplSimple` until this is done. |
+| F144 | Restore `cljw build` self-bundling                         | `std.fs.selfExePath` + `std.fs.openFileAbsolute` were removed in 0.16. Reimplement via argv[0] + `std.c.realpath` (or `_NSGetExecutablePath` / `/proc/self/exe`) and migrate file write loop. Stub in `runner.zig handleBuildCommand`. |
+| F145 | OrbStack Ubuntu re-validation under Zig 0.16.0             | `--seed 0` workaround was discovered on 0.15.2; re-test on 0.16.0 (Random.zig line numbers may have shifted). Run full `bash test/run_all.sh` + `bash bench/run_bench.sh` on Linux ARM64 + x86_64. |
+| F146 | Strip libc back out (`link_libc = false`)                  | zwasm v1.11.0 enables libc to satisfy the `std.posix.*` removals (D111). cf. zwasm W46. Once `std.Io` and the std.c usages in CW (`getcwd`, `getenv`, `realpath`, `mprotect`, `write`) all get pure-zig equivalents, drop libc to recover the pre-migration ~290 KB on Linux. |

.dev/decisions.md

@@ -938,3 +938,49 @@ instance state). Wasm linear memory remains separately managed per spec.
 at Engine construction. Requires zwasm D128 to be implemented first.
 
 Related: zwasm D128, cw-new D13.
+
+## D111: Zig 0.15.2 → 0.16.0 Migration
+
+**Date**: 2026-04-27
+**Status**: Done
+**Decision**: Migrate the entire ClojureWasm tree from Zig 0.15.2 to 0.16.0,
+together with bumping zwasm to v1.11.0 (the first 0.16-compatible tag).
+Centralize the new `std.Io` model behind a process-wide accessor module
+`runtime/io_default.zig` so existing module-level mutexes, time helpers,
+env lookups, and sleeps don't have to thread `io` through every call site.
+
+**Why now**: Zig 0.16 reshapes `std.Io` (Mutex/Condition/sleep/Timestamp
+all take `io: Io`), removes `std.fs.cwd` (replaced by `std.Io.Dir`), removes
+`std.posix.{getenv,write,isatty}`, and changes `pub fn main()` to
+`pub fn main(init: std.process.Init)`. Staying on 0.15.2 indefinitely
+forfeits stdlib improvements and forces zwasm to maintain a parallel branch.
+
+**Approach**:
+
+- *zwasm-first vs detach-then-reattach*: chose to upgrade zwasm to v1.11.0
+  from the start (rejected the original "detach + Phase 6 reattach" plan).
+  Reason: v1.11.0 is already 0.16-ready, so keeping zwasm in saved a whole
+  reattach phase and let wasm e2e/bridge tests stay green throughout.
+- *io_default module*: production entry points (main, cache_gen) call
+  `io_default.set(init.io)` at startup, so all module-level mutexes /
+  Condition variables / nanoTimestamp / sleep / getenv pick up the real
+  cancelable io. Tests fall through to a process-wide
+  `std.Io.Threaded.init_single_threaded` default, except for the few that
+  need real spawn semantics (shell tests) which install a local Threaded.
+- *libc linkage*: zwasm v1.11.0 enables `link_libc = true` by default
+  (D135 in zwasm). CW inherits the libc-linked binary; we use std.c.getenv
+  / std.c.realpath / std.c.write / std.c.mprotect / std.c.getcwd in places
+  where stdlib equivalents were removed. Stripping libc back out is a
+  follow-up (F##; cf. zwasm's W46 sequence).
+- *temporary stubs*: HTTP server, nREPL, fancy line editor, and `cljw build`
+  rely on `std.net` / `std.posix.poll` / raw-mode termios / `std.fs.selfExePath`
+  — all gone or reshaped in 0.16. The full rewrite to `std.Io.net` + Smith
+  fuzzing is non-trivial and was scoped out of this migration. Each is
+  stubbed with a clear runtime error and tracked as a separate F## item.
+
+**Verification**: 1324/1324 unit tests, 83/83 cljw test namespaces, 6/6 wasm
+e2e, deps.edn e2e all green on macOS aarch64. Bench history records
+`pre-zig-016` and `post-zig-016` entries; no individual benchmark regressed
+beyond noise; lazy_chain actually improved.
+
+Related: zwasm D135 (Vm.io infra), Phase 7 follow-ups in `.dev/checklist.md`.

.dev/future.md

@@ -362,7 +362,7 @@ Two tracks that do not fully converge. GC and bytecode diverge.
 
 - MarkSweepGc works on wasm32-wasi as-is (GPA→WasmPageAllocator, PoC validated)
 - Free-pool recycling ideal for Wasm (memory grows only, never shrinks)
-- WasmGC not usable: Zig 0.15.2 can't emit WasmGC instructions (struct.new, i31ref)
+- WasmGC not usable: Zig 0.16.0 can't emit WasmGC instructions (struct.new, i31ref)
 - Dynamic languages on Wasm (Python, Ruby) all use self-managed GC in linear memory
 - No comptime GC switching needed for MVP — same MarkSweepGc on both tracks