build(bootstrap): manage uv with mise - #1134
Conversation
|
Before on a fresh checkout on a system with an out of range version of UV: After: After this |
Bootstrap assumed a uv on PATH inside pyproject.toml's `required-version = ">=0.9.14,<0.10.0"`, with README and AGENTS.md telling people to install a matching one by hand. Pin uv in mise.toml alongside node and pnpm, add `UV := $(MISE_EXEC) uv`, and route every uv invocation in the Makefile through it so `make test-unit`, `make bootstrap-python` and friends use the pinned version regardless of what is on PATH. NMP_SKIP_MISE=1 still falls back to bare `uv`. Default PYTHON_VERSION to 3.12. It was 3.11, which no longer satisfies `requires-python = ">=3.12,<3.15"`, so `uv python install` fetched an interpreter the project cannot use while `uv venv` quietly picked a different one. pnpm and uv are both fetched through the GitHub API, which allows 60 unauthenticated requests per hour per IP — enough to fail a bootstrap on a shared NAT or a CI runner. Reuse a GITHUB_TOKEN from the environment or the gh CLI when one is available, and explain the rate limit when the install fails instead of surfacing a bare 403. Signed-off-by: mschwab <mschwab@nvidia.com>
caedda3 to
aa33a34
Compare
|
Routing every uv call through `mise exec --` broke CI: the Python jobs install uv with astral-sh/setup-uv and never install mise, so `make test-unit-ci`, `test-integration-ci`, `update-licenses`, `vendor`, `generate-cli-commands` and `check-copyright-headers` all died with env: '/home/runner/.local/bin/mise': No such file or directory make: *** [Makefile:387: test-unit-ci] Error 127 MISE falls back to $(HOME)/.local/bin/mise whether or not anything is there, so MISE_EXEC prefixed a binary that does not exist. Set NMP_SKIP_MISE=1 for the workflow. setup-uv already resolves uv from pyproject.toml's required-version — the same constraint mise.toml's `uv = "0.9"` was chosen to satisfy — and actions/setup-node pins Node for the web jobs, so nothing here needs the mise-managed toolchain. The pin mise adds is for developer machines, where a global uv outside required-version can win on PATH; CI has no such uv. Only ci.yaml needs this. ngc-metadata.yaml and insights-testbed.yml call `uv run` directly rather than through make, so MISE_EXEC never applies, and publish-fern-docs.yaml's `make docs-check` is npm-only. Signed-off-by: mschwab <mschwab@nvidia.com>
Routing uv through `mise exec --` means targets that call uv now depend on mise being installed, and only the bootstrap chain installs it. Run `make test-unit` on a fresh clone and you get /bin/sh: /Users/you/.local/bin/mise: No such file or directory make: *** [test-unit] Error 127 which says nothing about what to do. Document the ordering in SETUP.md with the symptom, the fix, and the NMP_SKIP_MISE=1 opt-out. Also drop the advice this branch obsoletes. AGENTS.md told Cursor Cloud users to `pip install 'uv>=0.9.14,<0.10.0'` before bootstrapping and README.md listed a matching uv as a prerequisite for the source checkout; mise now supplies it, and a global uv outside the range no longer has to be downgraded. Signed-off-by: mschwab <mschwab@nvidia.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe PR pins uv 0.9, routes Makefile Python commands through mise, updates Python to 3.12, improves mise authentication handling, and configures CI and documentation for the shared toolchain. ChangesPython toolchain execution
Sequence Diagram(s)sequenceDiagram
participant Makefile
participant GitHubCLI
participant mise
participant uv
Makefile->>GitHubCLI: request gh auth token when GITHUB_TOKEN is unset
GitHubCLI-->>Makefile: return GitHub token
Makefile->>mise: install pinned tools
mise-->>Makefile: provide configured toolchain
Makefile->>uv: run Python commands through MISE_EXEC
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Makefile`:
- Line 20: Update the Makefile commands invoking uv venv and uv sync to pass
--python $(PYTHON_VERSION), ensuring the configured interpreter controls
environment creation and synchronization. Also add verify-python-version as a
prerequisite of the bootstrap-python target.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: e5f734d8-9a32-40b0-801a-6c4faa32c019
📒 Files selected for processing (3)
.github/workflows/ci.yamlMakefilemise.toml
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@README.md`:
- Around line 42-43: Update the README prerequisites wording around the make
bootstrap guidance to distinguish required source-development tools, especially
Git and GNU Make, from the pinned uv, Node.js, and pnpm versions installed by
bootstrap. Do not state that the preceding prerequisites apply only to PyPI
installation; clarify that bootstrap installs tool versions but still requires
Git and GNU Make.
In `@SETUP.md`:
- Around line 104-107: Update the fenced code block in SETUP.md containing the
`/bin/sh` and `make: *** [test-unit] Error 127` output to specify the `text`
language tag, preserving the displayed command output unchanged.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 97fc8649-f3d2-4f27-b9e4-db9e4b7a8b90
📒 Files selected for processing (3)
AGENTS.mdREADME.mdSETUP.md
`uv venv` and `uv sync` were choosing an interpreter on their own. `requires-python = ">=3.12,<3.15"` accepts 3.12, 3.13 and 3.14, so `verify-python-version` would find or install PYTHON_VERSION and then `uv venv` built the environment against something else entirely: $ uv python find 3.12 $ uv venv --seed --allow-existing $ .venv/bin/python --version Python 3.13.3 Pass `--python $(PYTHON_VERSION)` to both, which pins it (3.12.12 in the same check), and make `bootstrap-python` depend on `verify-python-version` so the interpreter exists before the sync rather than only when someone runs the verify target directly. Only the guardrails benchmark job runs `make bootstrap-python`, and it already pins python-version 3.12; the 3.12/3.13 wheel matrices build without it, so nothing in CI changes interpreter. Also correct the README claim that the listed prerequisites apply only to the PyPI install — Git and GNU Make are still needed from source — and give the Error 127 block in SETUP.md a language tag (MD040). Signed-off-by: mschwab <mschwab@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
Makefile (1)
168-179: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winAdd the mise prerequisite to
.venv/bin/python.This target has no prerequisite, but its recipe invokes
$(UV). UnlessNMP_SKIP_MISE=1, this expands tomise exec -- uv. A directmake .venv/bin/pythoncan therefore run beforeverify-miseinstalls mise.Add
verify-python-versionorverify-miseas a prerequisite.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Makefile` around lines 168 - 179, Add verify-python-version or verify-mise as a prerequisite to the .venv/bin/python target so direct invocations validate the required mise/UV setup before executing the recipe. Preserve the existing virtual-environment creation logic and BOOTSTRAP_CREATE_VENV guard.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Makefile`:
- Line 178: Update the venv setup recipe around the existing “uv venv” command
to detect whether .venv/bin/python reports a different version than
PYTHON_VERSION. When the versions differ, remove or clear the existing
environment and recreate it with uv venv --clear; otherwise preserve the current
environment setup behavior.
---
Outside diff comments:
In `@Makefile`:
- Around line 168-179: Add verify-python-version or verify-mise as a
prerequisite to the .venv/bin/python target so direct invocations validate the
required mise/UV setup before executing the recipe. Preserve the existing
virtual-environment creation logic and BOOTSTRAP_CREATE_VENV guard.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: e52683c6-e1b7-43ec-90c5-e30ec86dc7ed
📒 Files selected for processing (3)
MakefileREADME.mdSETUP.md
🚧 Files skipped from review as they are similar to previous changes (2)
- README.md
- SETUP.md
The target had no prerequisites but its recipe calls $(UV), which expands to `mise exec -- uv` unless NMP_SKIP_MISE is set. A direct `make .venv` — or `make bootstrap-plugins`, which depends on it — could therefore run before anything installed mise. Make `verify-python-version` an order-only prerequisite. A normal one would force the recipe on every invocation, since a phony prerequisite is always newer than the target file, and re-seed an existing venv each time; order-only runs the check but leaves an up-to-date venv alone. That also makes the `uv python find || uv python install` line inside the recipe redundant with the one in verify-python-version, so drop it. Signed-off-by: mschwab <mschwab@nvidia.com>
`.venv/bin/python` existing says nothing about which interpreter built it, so as a file target make skipped the recipe and a stale venv survived a PYTHON_VERSION change: $ .venv/bin/python -V # PYTHON_VERSION = 3.12 Python 3.13.3 $ make .venv make: Nothing to be done for `.venv'. Make it phony and compare the interpreter instead. On a match the recipe exits early, so an up-to-date venv is never rebuilt; on a mismatch it recreates with `--clear` and says what changed. `--clear` matters because `--allow-existing` leaves the old tree behind — a 3.13 venv rebuilt at 3.12 keeps `lib/python3.13` next to `lib/python3.12`. `bootstrap-python` already self-healed through `uv sync --python`; this covers `make .venv` and `make bootstrap-plugins` reached on their own. Signed-off-by: mschwab <mschwab@nvidia.com>
Wrapper targets escaped mise. `stainless`, `lint`, `lint-fix` and `vendor-nemo-platform-ext` hand off to a script or a sub-make that calls bare `uv` — 9 of the 13 scripts behind `lint-all.sh` do, as do `sdk/stainless.sh` and `packages/nemo_platform_ext/Makefile`. So `make update-sdk`, `make update-cli`, `make lint` and `make lint-fix` still broke against a global uv outside `required-version`, which is the case this branch exists to fix. Route the four through $(MISE_EXEC); scripts already invoked via $(UV) are fine, since `mise exec` puts the pinned uv on PATH for the whole subprocess tree. Patch-level PYTHON_VERSION never matched. The probe reported major.minor and compared it literally, so PYTHON_VERSION=3.13.3 against a 3.13.3 venv read as "3.13 != 3.13.3" and cleared the environment on every invocation — including, with BOOTSTRAP_LOCAL_PLUGIN_DIRS set, the freshly synced one immediately before installing local plugins. Compare the full version and treat PYTHON_VERSION as a selector, so 3.13 matches any 3.13.x and 3.13.3 matches exactly. A damaged .venv could not be repaired. `--clear` was gated on the interpreter probe succeeding, so a .venv whose python was missing or non-executable ran `uv venv` with neither --clear nor --allow-existing and failed with "A directory already exists". Gate it on the directory instead. Signed-off-by: mschwab <mschwab@nvidia.com>
Summary
Extends the mise-managed toolchain from #1109 to cover uv.
pyproject.tomlrequiresuv>=0.9.14,<0.10.0, but nothing installed a matching uv —README.mdpinned an installer URL andAGENTS.mdtold Cursor Cloud users topip install 'uv>=0.9.14,<0.10.0'after hitting the mismatch. Same gap mise already closes for Node.js and pnpm.uv = "0.9"inmise.toml. Resolves to 0.9.30, inside therequired-versionrange, and mise verifies GitHub artifact attestations plus a checksum on install.UV := $(MISE_EXEC) uvand route all 48 uv invocations in theMakefilethrough it, somake bootstrap-python,make test-unit,make update-sdkand the rest use the pinned uv regardless of PATH.NMP_SKIP_MISE=1still falls back to bareuv.verify-python-versionandbootstrap-pythonnow depend onverify-mise.PYTHON_VERSIONto 3.12. It was 3.11, which no longer satisfiesrequires-python = ">=3.12,<3.15", souv python installfetched an interpreter the project can't use whileuv venvsilently picked a different one.GITHUB_TOKENfrom the environment or theghCLI duringmise install, and explain the GitHub rate limit on failure instead of surfacing a bare 403.NMP_SKIP_MISE=1for the CI workflow, which provisions its own toolchain. Detail below.SETUP.md, and drop the now-obsoletepip install 'uv>=0.9.14,<0.10.0'advice fromAGENTS.mdand the uv prerequisite fromREADME.md's source-checkout path.Global uv version mismatches
This also fixes the case where uv is already installed globally at a version outside
required-version.AGENTS.mddocuments the symptom today — uv 0.11.x failsuv syncwith a version mismatch, and the advised fix is to downgrade the machine-wide install, which breaks other projects.Because every uv invocation goes through
mise exec --, the mise-managed uv takes precedence over whatever is first on PATH, so no downgrade is needed. Verified with a stub uv reporting 0.11.7 ahead of everything on PATH:Why CI opts out
MISEfalls back to$(HOME)/.local/bin/misewhether or not anything is there, so on a machine without mise,MISE_EXECprefixes every uv call with a binary that does not exist. CI is exactly that machine: the Python jobs install uv throughastral-sh/setup-uvand never install mise, which reachesverify-miseonly viabootstrap-studio. The first push of this branch failed four jobs on it:test-unit-ci,test-integration-ci(unit / integration / e2e), andupdate-licenses,vendor,generate-cli-commands,check-copyright-headers(Lint all).Setting
NMP_SKIP_MISE=1at the workflow level is the fix rather than installing mise in CI, because CI is already pinned from the same source of truth —setup-uvis configured withversion-file: pyproject.toml, so it resolves uv from the samerequired-versionconstraint thatuv = "0.9"was chosen to satisfy, andactions/setup-nodepins Node for the web jobs. What mise adds is protection against a global uv outsiderequired-versionwinning on PATH, which is a developer-machine problem; a fresh runner has no such uv. Installing mise per job would also pull node and pnpm onto Python-only jobs and spend GitHub API requests against the 60/hr unauthenticated ceiling for no pin that isn't already there.Only
ci.yamlneeds it.ngc-metadata.yamlandinsights-testbed.ymlcalluv rundirectly rather than through make, soMISE_EXECnever applies, andpublish-fern-docs.yaml'smake docs-checkis npm-only.On the GitHub rate limit
pnpm and uv are both fetched through the GitHub API (
aqua:pnpm/pnpm,aqua:astral-sh/uv), which allows 60 unauthenticated requests per hour per IP. That is enough to fail a bootstrap behind a shared NAT, and it affects the pnpm install from #1109 as well — earlier runs simply had headroom. Observed on a VM with the limit exhausted:Node is unaffected — it comes from nodejs.org.
Download mirrors (
url_replacements) don't help here, since the failing calls are metadata and attestation endpoints rather than release-asset downloads. Disabling aqua verification would cut some calls but trades away the integrity check. A token is the cheap fix:gh auth logincovers most developers. The token is read inside the recipe shell, so it never enters make's variable space ormake -noutput.Test plan
macOS (arm64):
make verify-miseinstalls uv 0.9.30;mise exec -- uv --version→uv 0.9.30.make verify-python-versionexits 0 and installs Python 3.12.12 through the mise-managed uv.uv lock --checkagainst the real workspace resolves 582 packages, lockfile in sync.make -n test-unit-ciexpands to"…/mise" exec -- uv run --frozen pytest …; withNMP_SKIP_MISE=1it expands to bareuv run. Same for the other five targets that failed CI.ghfallback completes the install;make -n verify-miseshows thegh auth tokencall but never the token.Ubuntu 26.04 (aarch64) VM, no node/pnpm/uv/mise installed:
CI:
Known limitation:
maketargets assume bootstrap ranOnly the bootstrap chain installs mise (
verify-mise). Onmainall fourMISE_EXECusers reach it; this branch adds 34 that don't — thetest-*targets,update-licenses,vendor,refresh-openapi,generate-cli-*,check-licenses,docs-*,bootstrap-plugins,run,benchmark-guardrails. On a machine without mise they fail withError 127instead of falling back:make bootstrapis the documented first step, so the normal path is unaffected — this bites someone who already has uv and jumps straight tomake test-unit. Documented inSETUP.mdwith the symptom and the fix rather than adding averify-miseprerequisite to all 34, which would runmise install --yeson everymake test-*.Known gap: pre-commit still uses the global uv
Five hooks in
.pre-commit-config.yamlarelanguage: systemand resolveuvoff PATH —uv-lock,copyright-fix,config-reference-docs, and thety/openapi-generatorhooks via the scripts they call. A global uv outsiderequired-versionstill blocks any commit touchingpyproject.toml. Pre-existing rather than a regression here, and fixing it properly means a sharedscript/uvshim that both the Makefile and pre-commit resolve through, which is a bigger change than this branch should carry. Tracked in #1161.Notes
#1109 has merged, so this branch was rebased onto
mainand now targetsmaindirectly.The
GITHUB_TOKENfallback and the rate-limit diagnostic also cover the pnpm install added in #1109.Summary by CodeRabbit
Chores
Documentation