feat: expand gateway glibc compatibility

[pimlock]

Problem Statement

OpenShell gateway release binaries currently do not have a confirmed Rocky/RHEL 8-compatible GNU/Linux ABI floor.

Rocky Linux 8 and RHEL 8 ship glibc 2.28 with enterprise backports. Users running gateway on those hosts need openshell-gateway to start without requiring newer glibc symbols. The v0.0.63 generic GNU x86_64 gateway artifact was dynamically linked and required symbols above that floor:

• GLIBC_2.29: log, log2, exp, pow, exp2
• GLIBC_2.29: posix_spawn_file_actions_addchdir_np
• GLIBC_2.30: pthread_cond_clockwait
• GLIBC_2.30: gettid

Follow-up local builds showed that newer Rust/std paths can avoid some of these with weak optional lookups or fallbacks, but the CI path is still not settled. In particular, the current cargo-zigbuild gateway build with bundled Z3 has hit C/C++ wrapper/toolchain issues in CI before reaching the final GLIBC symbol verification stage.

Proposed Design

Expand gateway glibc compatibility so the released GNU/Linux gateway can explicitly target Rocky/RHEL/Alma/Oracle Linux 8 class systems with glibc 2.28.

The final build path should:

• produce openshell-gateway release artifacts for GNU/Linux with no required GLIBC_* symbols above GLIBC_2.28;
• keep a direct verifier in CI, for example:

strings openshell-gateway | rg -o 'GLIBC_[0-9]+\.[0-9]+' | sort -Vu
objdump -T openshell-gateway | rg 'GLIBC_2\.(29|30|3[1-9])'

• smoke-test the resulting binary on a Rocky/RHEL 8-compatible runtime, ideally with LD_BIND_NOW=1 so missing eager bindings fail at startup;
• document which release artifacts are expected to support glibc 2.28.

Alternatives Considered

1. Keep cargo-zigbuild with an explicit glibc 2.28 target and unbundle Z3

Build the gateway with an explicit target such as:

cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.28 -p openshell-server --bin openshell-gateway

Then stop bundling Z3 into the gateway binary and provide Z3 as a runtime package dependency instead.

Pros:

• Keeps the desired glibc floor explicit in the build target.
• Avoids the current z3-sys bundled C/C++ build path that is interacting poorly with cargo-zigbuild in CI.
• May align with system package managers for Debian/RPM-style gateway packages.

Cons / open questions:

• We need to confirm whether Z3 is still required at build time for Rust crate metadata/link discovery, even if it is not bundled.
• Release package manifests would need to declare the correct Z3 runtime dependency for every package format we produce.
• Runtime dependency availability and ABI compatibility would vary by distro.
• Tarball users would need clear instructions or a bundled sidecar strategy, since tarballs do not install system packages.

2. Use an older glibc sysroot on the existing CI image

A sysroot is a directory containing the target platform's headers and libraries. During linking, the compiler/linker can be told to use that directory instead of the build host's /usr/include, /lib, and /usr/lib.

This lets us use a newer build container/toolchain while still producing a binary with an older glibc ABI floor. For example, run modern Rust/clang/cargo in an Ubuntu 24.04 or Fedora image, but link against a Rocky/RHEL 8 sysroot containing glibc 2.28:

RUSTFLAGS="
  -C linker=clang
  -C link-arg=--sysroot=/opt/sysroots/rocky8
  -C link-arg=-L/opt/sysroots/rocky8/usr/lib64
  -C link-arg=-L/opt/sysroots/rocky8/lib64
"
cargo build --release -p openshell-server --bin openshell-gateway --target x86_64-unknown-linux-gnu

Deno uses a related approach in denoland/deno_sysroot_build: the sysroot provides older-versioned libraries to the linker while the build itself does not run inside a chroot/sysroot. See https://github.com/denoland/deno_sysroot_build.

Pros:

• Makes the ABI floor explicit through the sysroot contents rather than through the ambient CI host.
• Allows modern build tooling while linking against Rocky/RHEL 8-era glibc.
• Should fix accidental bindings of old APIs to newer symbol versions, such as log@GLIBC_2.29 or pthread_create@GLIBC_2.34.
• May avoid a full old-image build while keeping CI maintainability.

Cons / open questions:

• The sysroot must be built, versioned, checksummed, and refreshed intentionally.
• C/C++ dependencies such as Z3 still need careful compiler/linker configuration against the sysroot.
• A sysroot will not invent APIs missing from glibc 2.28. Code still needs fallbacks for newer APIs such as:

posix_spawn_file_actions_addchdir_np  # glibc 2.29
pthread_cond_clockwait                # glibc 2.30
gettid                                # glibc 2.30

• We may need linker shims for specific libm symbols if they otherwise bind to newer symbol versions.

3. Introduce musl as an alternative artifact

Provide an additional musl/static gateway artifact for environments that prefer not to depend on host glibc at all.

Pros:

• Avoids the host glibc ABI floor problem for users who can run a musl-linked gateway.
• We already use static musl successfully for the supervisor image path.
• Could be useful as an escape hatch while GNU/glibc compatibility work continues.

Cons / open questions:

• Fully switching the primary gateway artifact to musl is not a good default because musl builds are harder to FIPS validate than glibc builds.
• Native TLS, DNS, libc behavior, and dependency behavior need separate validation.
• This should likely be additive, not a replacement for GNU/glibc artifacts.

Agent Investigation

Context from the glibc compatibility investigation:

• Rocky/RHEL 8 glibc is based on 2.28, with enterprise backports.
• The v0.0.63 openshell-gateway-x86_64-unknown-linux-gnu.tar.gz artifact required GLIBC_2.29 and GLIBC_2.30 symbols.
• Local experiments showed that a 2.28-floor build can start on Rocky 8 with LD_BIND_NOW=1, but the current CI path still needs a robust build strategy.
• Rust/std weak optional symbol behavior appears acceptable for some symbols:
  • posix_spawn_file_actions_addchdir_np has a fallback path when unavailable.
  • gettid can fall back to the raw syscall when the glibc wrapper is unavailable.
  • pthread_cond_clockwait disappeared from the rebuilt binary; current Rust std Linux condvars use a futex path / older pthread symbols.
• The bundled Z3 path is the main unresolved build-system complication for the gateway GNU/Linux release build.

Definition of Done

• Decide on the primary build strategy for GNU/Linux gateway glibc 2.28 compatibility.
• Resolve the bundled/unbundled Z3 build and runtime packaging story.
• Add CI verification that fails if openshell-gateway requires symbols above GLIBC_2.28.
• Add a Rocky/RHEL 8 startup smoke test for the built artifact.
• Update release/package documentation to describe the supported glibc floor and any Z3 runtime dependency.
• Decide whether to add a separate musl gateway artifact as an alternative.

[pimlock]

Related issue, where we introduced zigbuild linker to target lower glibc: #1456

[pimlock]

z3 packaging spike update

We tried the unbundled/system-z3 route for the gateway in the glibc 2.28 compatibility PR.

What worked:

• The Linux gateway artifact can be built with an explicit GNU zig target and without bundled-z3.
• The resulting gateway links dynamically to z3, and the glibc verifier kept the binary at the intended GLIBC_2.28 floor.
• Debian packaging worked with Depends: libz3-4.
• Snap already stages libz3-4, and Homebrew already declares depends_on "z3".

What failed:

• RPM package smoke failed in %check when running the staged gateway:

  openshell-gateway: error while loading shared libraries: libz3.so.4: cannot open shared object file: No such file or directory
  

• Adding Fedora z3-libs/z3-devel was not sufficient. The failed run used Fedora 44 and installed z3-libs-4.16.0-1.fc44, but the gateway artifact had been linked in the Debian/Ubuntu-style build environment and required libz3.so.4.

Conclusion:

• Unbundling z3 makes the "portable" Linux GNU gateway artifact distro-sensitive again. zigbuild controls the glibc floor, but it does not make third-party shared-library SONAMEs portable across Debian/RPM ecosystems.

Options considered:

• Build the RPM gateway inside Fedora so it links against Fedora z3. Rejected for this PR because the straightforward RPM build path uses Fedora's current glibc/toolchain, which would lose the explicit glibc 2.28 compatibility contract unless we also teach the RPM build to use cargo zigbuild and verify symbols.
• Ship the matching z3 shared runtime next to the gateway and wire RPATH/RUNPATH. Possible, but then OpenShell owns z3 runtime provenance and security updates, and we may also need to account for z3's own C++ runtime dependencies.
• Go back to bundled/vendored z3 for the gateway. This looks like the cleaner release contract if we can make it work with explicit glibc targeting: one gateway artifact, no distro z3 SONAME dependency, and verify-glibc-symbols.sh 2.28 remains the guardrail.

Next spike:

• Try the z3 vendored/existing bundled-z3 path again, but specifically determine whether we can target x86_64-unknown-linux-gnu.2.28 explicitly rather than relying on Zig's current default for bare GNU targets.

[pimlock]

Bundled Z3 follow-up

We tried the bundled/vendored Z3 path again after the distro libz3.so.4 approach failed for RPM packaging.

Successful package-smoke run with bundled Z3: https://github.com/NVIDIA/OpenShell/actions/runs/27658312503

Current approach:

• Build the gateway with the existing bundled-z3 feature so z3-sys builds and statically links Z3 instead of leaving a runtime libz3.so dependency.
• Keep the gateway GNU target explicit as x86_64-unknown-linux-gnu.2.28 / aarch64-unknown-linux-gnu.2.28 so the glibc floor is intentional rather than relying on cargo-zigbuild defaults.
• Add Zig C/C++ wrapper setup for z3-sys CMake builds. cargo zigbuild accepts Rust-style glibc-suffixed targets, but Zig C/C++ expects the vendorless form, e.g. x86_64-linux-gnu.2.28. The wrapper strips leaked --target=* flags from CMake/cc-rs and invokes Zig with the normalized target.
• Export a matching CMAKE_TOOLCHAIN_FILE_* override so z3-sys CMake configuration uses the wrapper instead of cargo-zigbuild's generated temporary CMake toolchain.
• Keep the runtime checks that fail if the gateway binary dynamically depends on libz3, and keep verify-glibc-symbols.sh 2.28 as the final guardrail.

Cache issue found during the spike:

• The first successful bundled-Z3 run restored an exact Swatinem/rust-cache hit that contained stale z3-sys CMake output pointing at cargo-zigbuild's generated wrapper/toolchain paths.
• The helper now removes only stale z3-sys CMake build directories whose cached files mention cargo-zigbuild/old zigcc/zigcxx wrapper paths. Corrected z3-sys CMake output is preserved.
• However, GitHub Actions caches are effectively immutable for an exact key. The successful run restored an exact cache hit and the post-cache step reported Cache up-to-date, so it did not save the corrected Z3 build output over the old key.
• To make future runs actually reuse the corrected Z3 build, the branch now adds a key: component based on hashFiles('tasks/scripts/setup-zig-cc-wrapper.sh') for the gateway cache paths. That gives the wrapper-aware build a new exact cache key and automatically busts again if the wrapper changes.

Net result: bundled Z3 works for the package-smoke path, keeps the gateway portable without a distro libz3 runtime dependency, and preserves the explicit glibc 2.28 target. The cache fix should make the first run on the new key seed a clean cache; later runs should be able to reuse the corrected z3-sys build output.

[pimlock]

Follow-up on the bundled/vendored Z3 path and cache behavior:

• The bundled-Z3 package-smoke probe passed in Release Dev run 27658312503, so we kept the gateway self-contained instead of adding a dynamic libz3.so.4 runtime dependency to every packaging surface.
• For caching, the important correction is that Swatinem/rust-cache needed the wrapper hash in shared-key; adding a separate key: input showed up in action input logs but did not change the computed primary cache key for this workflow.
• Run 27660740650 seeded the corrected cache on commit 5d836b406624bf78dba49bbc66fdded2324a800a. The gateway build steps were still cold-ish: about 12m26s for amd64 and 12m30s for arm64.
• Run 27661473950 reran the same commit and restored the new zig-wrapper-... cache keys with full matches. The gateway build steps dropped to about 1m09s for amd64 and 1m14s for arm64.

The current branch therefore uses:

shared-key: gateway-binary-gnu-${{ matrix.arch }}-zig-wrapper-${{ hashFiles('tasks/scripts/setup-zig-cc-wrapper.sh') }}

The wrapper also removes stale z3-sys CMake build directories only when their cached CMake files still reference old cargo-zigbuild wrapper paths, so a corrected cache is preserved and reused.