Skip to content

remove oras - #166

Open
bcumming wants to merge 44 commits into
eth-cscs:mainfrom
bcumming:no-oras
Open

remove oras#166
bcumming wants to merge 44 commits into
eth-cscs:mainfrom
bcumming:no-oras

Conversation

@bcumming

@bcumming bcumming commented Jun 30, 2026

Copy link
Copy Markdown
Member

Remove uenv's runtime dependency on the external oras binary and replaces it with a self-contained, native OCI registry client written in C++ (src/oci/). After this change uenv is a fully static binary with no runtime shell-outs to oras, tar, gzip, or curl — all registry, archive, and hashing operations happen in-process. Images pushed by the new client are byte-for-byte compatible with oras, so artifacts remain interoperable in both directions.

oras was an external process uenv shelled out to for push/pull/copy/inspect. That made deployments depend on a separate binary, complicated error handling, and left the tool unable to be fully static. This PR brings all of that functionality in-house.

New features:

src/oci/ — native OCI registry client. A deliberately self-contained module (depends only on src/util/ + external libs; enforced by a grep in CI/docs) that talks the OCI distribution API over HTTP:

  • client (pimpl'd), pull, push, copy, manifest handling, and referrers-API support (with fallback for registries that lack it)
  • auth — bearer-token flow with re-authentication of expired tokens, and a docker-credential helper instead of silently falling back to anonymous access
  • Strongly-typed domain vocabulary (digest, reference, tag, manifest) with its own lexer-based parse.*, so near-synonymous registry terms can't be mixed up
  • Extensively documented in src/oci/readme.md

New src/util/ building blocks, kept in util so oci never reaches up into uenv:

  • sha.* — in-process SHA-256 (blobs are hashed while streaming)
  • url.* — a first-class URL type with consistent end-to-end handling
  • archive.* — in-process tar pack/unpack via libarchive (replaces tar/gzip), oras-compatible
  • Extended curl.* (streaming, upload progress), shared parse.* scaffolding, and lexer/string additions

Dependencies now vendored as static meson subprojects: zlib, openssl (curl's TLS backend), libarchive. Dropped the wrapdb fallback and the old curl runtime deps.

Bug fixes:

  • Verify the digest of downloaded meta archives; stop buffering them in memory
  • Downloads go to a temp file so a failed download leaves no half-written state
  • Wrong-typed JSON in registry responses no longer aborts via an uncaught type_error
  • Image copy no longer silently drops attached metadata
  • Treat missing credentials on delete as a hard error
  • Fixed a race in download-progress reporting

Tests:

  • New unit suites: oci_auth, oci_client, oci_digest, oci_manifest, oci_parse, oci_registry, oci_tag, plus sha, url, archive, curl, subprocess
  • New registry BATS suite exercising push/pull end-to-end against a throwaway local zot registry, with registry_ctl / listing_mock helpers (self-skips when no zot binary is present)

bcumming and others added 30 commits June 29, 2026 15:22
…ring them in memory

pull_meta fetched the meta tar.gz with the in-memory get_blob, which
checks only the HTTP status and performs no sha256 verification, then
wrote the buffer back out to a temp file for tar. The meta payload
(env.json, views) is later sourced into user environments, and it was
the only downloaded artifact that skipped verification.

Stream the layer straight to the staging path with get_blob_to_file,
which hashes during download and rejects a digest mismatch. This also
removes a redundant RAM+disk round trip of the whole blob.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant