build: migrate CI and service images from Poetry to uv by RonShakutai · Pull Request #2152 · data-privacy-stack/presidio

RonShakutai · 2026-07-09T17:47:10Z

Change Description

Migrates the repository's CI and service-image dependency installation from Poetry to uv, with committed lockfiles. This is the first, self-contained step toward a fully uv-based repo (a fuller migration — build backend, remaining Dockerfile variants, docs — is tracked as follow-up work).

Why: CI installed dependencies with poetry install and no committed lock, so Poetry ran a universal resolve of the whole optional-dependencies graph on every run. That resolve backtracked for many minutes — effectively hanging — on the analyzer's --all-extras graph (dominated by the langextract extra's deep transitive tree). The same pathology hit the local-build/E2E job, where docker compose build sat on "Resolving dependencies" for ~40 min. Reproduced locally: Poetry >6 min, never completed; uv resolves the same graph in ~2s.

What changed

Packaging (the 5 CI-tested packages):

Development tooling moved from Poetry's [tool.poetry.group.dev.dependencies] to the standard PEP 735 [dependency-groups] table — pyproject.toml is now the single source of truth for dev tools.
Each package ships a committed uv.lock (**/uv.lock added to CODEOWNERS; contributor docs require regenerating it when deps change).
The [project] metadata is otherwise unchanged, and the poetry-core build backend is retained for now, so published wheels/sdists are identical.

CI (.github/workflows/ci.yml):

uv is installed via pip install uv==0.11.6 (the astral-sh/setup-uv action isn't on the org allow-list — it caused a workflow startup_failure).
Dependencies install with uv sync --locked --group dev — installs exactly the locked graph and fails on pyproject/uv.lock drift (no resolution or lockfile mutation in CI). This replaces the manual uv pip install pytest … line.
Tests, spaCy downloads and diff-cover run inside the synced venv via uv run --no-sync; the wheel-build step keeps using the system interpreter.
uv's cache is persisted across runs (actions/cache keyed on uv.lock, trimmed with uv cache prune --ci) on the /mnt disk.
Job-level timeout-minutes safety net so a bad resolve fails fast instead of hanging.

Service images (analyzer / anonymizer / image-redactor):

Dockerfiles install with uv sync --locked --no-default-groups --extra server --no-install-project into UV_PROJECT_ENVIRONMENT=/usr/local, so images use the same locked graph as CI. The dependency layer is split from the source COPY for caching.
entrypoint.sh runs gunicorn directly (deps are on the system PATH; no poetry run).

Verified: all 22 test-matrix jobs (incl. Python 3.14) and the local-build/E2E job pass; the E2E image build dropped from a ~40 min hang to ~2.5 min. Service containers start healthy (/health → 200).

Security note: the committed lockfiles surface some pre-existing transitive advisories (transformers, cryptography) that the range-only manifests hid from scanners. Those are handled in a stacked security PR (#2158), not here.

Issue reference

N/A

Checklist

I have reviewed the contribution guidelines
I agree to follow this project's Code of Conduct
I confirm that I have the right to submit this contribution and that it does not knowingly contain proprietary or confidential code.
My code includes unit tests (CI/packaging-only change)
All unit tests and lint checks pass locally
My PR contains documentation updates / additions if required (CHANGELOG, CODEOWNERS, copilot-instructions)

Copilot

Pull request overview

This PR hardens the CI workflow against indefinite hangs during Poetry dependency resolution by adding explicit timeouts and ensuring Poetry runs non-interactively, improving reliability of the test matrix runs across Presidio components.

Changes:

Add a job-level timeout (timeout-minutes: 60) for the matrix test job to prevent unbounded runtime.
Add a step-level timeout (timeout-minutes: 30) specifically for the “Install dependencies” step to cap Poetry resolution time.
Add --no-interaction to poetry install to avoid CI waiting on prompts.

github-actions · 2026-07-09T17:57:19Z

Coverage report (presidio-anonymizer)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
presidio-anonymizer/presidio_anonymizer
anonymizer_engine.py
presidio-anonymizer/presidio_anonymizer/operators
custom.py
Project Total

_{This report was generated by python-coverage-comment-action}

github-actions · 2026-07-09T17:57:52Z

Coverage report (presidio-image-redactor)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
presidio-image-redactor/presidio_image_redactor
dicom_image_pii_verify_engine.py
document_intelligence_ocr.py
image_analyzer_engine.py
Project Total

_{This report was generated by python-coverage-comment-action}

github-actions · 2026-07-09T17:58:58Z

Coverage report (presidio-cli)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
presidio-cli/presidio_cli
cli.py
Project Total

_{This report was generated by python-coverage-comment-action}

github-actions · 2026-07-09T17:59:33Z

Coverage report (presidio-structured)

This PR does not seem to contain any modification to coverable code.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

The analyzer test job runs `poetry install --all-extras` with no committed lock, so Poetry performs a universal resolve of the full optional-dependencies graph on every run. That resolve backtracks for many minutes — effectively hanging — driven by the langextract extra's deep, loosely-bounded transitive tree. Reproduced locally: Poetry >6 min (never completed); uv ~2s for the same 158-package resolution. Switch the test jobs to uv: - Install uv via astral-sh/setup-uv. - Create a per-component venv with `uv venv --seed` (seed keeps pip for the wheel-build step) and expose it via $GITHUB_PATH / $VIRTUAL_ENV. - `uv pip install -e '.<extras>'` plus the test tooling that previously came from the Poetry dev group (uv does not read that group). - Add a job-level `timeout-minutes: 60` (and 30 on the install step) so a bad resolve fails fast instead of hanging. No package metadata changed — pyproject.toml is untouched; this only swaps the CI install tooling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Use "uv run --no-sync" for spaCy model downloads, pytest, and diff-cover instead of exposing the venv on $GITHUB_PATH. "uv sync" builds the project's .venv (resolving the analyzer --all-extras graph in seconds where Poetry's lockless universal resolve hangs), and the subsequent steps execute inside it idiomatically. The --all-extras selection stays in the job matrix, unchanged from main. Wheel-build keeps using the system interpreter, matching the previous Poetry behaviour. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

The astral-sh/setup-uv action is not on the org's allowed-actions list, so adding it made the CI workflow fail at startup (startup_failure) before any job ran. Install a pinned uv from PyPI with the already present setup-python instead; this needs no new third-party action and keeps the resolver benefits that motivated the Poetry -> uv switch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

spaCy's `download` command shells out to `python -m pip install`, but uv-created virtualenvs ship without pip, so runtime/first-use model downloads (e.g. the presidio-cli conftest fetching en_core_web_lg) failed with SystemExit: 1. Install pip alongside the test tooling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 15 out of 20 changed files in this pull request and generated 2 comments.

Address review feedback: - ci.yml: use ${{ env.UV_CACHE_DIR }} for the cache action path instead of hardcoding /mnt/uv-cache, so the cache and uv stay in sync if the dir changes. - copilot-instructions: update the "Technology Stack" bullet to name uv as the dependency/install tool (poetry-core retained only as build backend), matching the uv-based dev commands already documented. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 15 out of 20 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 15 out of 20 changed files in this pull request and generated 1 comment.

Trim the verbose explanatory comments in the test job to 1-3 lines each, and drop the dependency-review `allow-ghsas` entry for the transformers advisories. That suppression is a security concern rather than part of the Poetry->uv migration, so it moves to the stacked security PR alongside the .trivyignore. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 15 out of 20 changed files in this pull request and generated no new comments.

Remove `cache: 'pip'` (keyed on pyproject.toml) from the test job's setup-python: it is leftover Poetry-era config. Project dependencies now use the uv cache, and the only remaining pip installs (uv, tomli, build tools) are tiny and unrelated to pyproject.toml. Also condense the unreleased CHANGELOG entry to a single concise line; the implementation detail lives in the PR description. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 15 out of 20 changed files in this pull request and generated 2 comments.

+          # `build` needs tomli on Python < 3.11 to read pyproject.toml, and it
+          # isn't in the hashed requirements file; pre-install it (marker-guarded)
+          # so --require-hashes finds it already satisfied.
+          python -m pip install "tomli==2.4.1; python_version < '3.11'"
          pip install --require-hashes -r ${{ github.workspace }}/.github/pipelines/requirements-build.txt


Copilot

Pull request overview

Copilot reviewed 15 out of 20 changed files in this pull request and generated 1 comment.

Copilot AI review requested due to automatic review settings July 9, 2026 17:47

Copilot started reviewing on behalf of RonShakutai July 9, 2026 17:55 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 9, 2026 18:22

RonShakutai changed the title ~~fix(ci): add timeout-minutes to prevent Poetry hanging~~ fix(ci): revert analyzer to Python <3.14 to fix Poetry resolve hang Jul 9, 2026

Copilot started reviewing on behalf of RonShakutai July 9, 2026 18:23 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread presidio-analyzer/pyproject.toml Outdated

Copilot AI review requested due to automatic review settings July 9, 2026 20:46

Copilot started reviewing on behalf of RonShakutai July 9, 2026 20:47 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread presidio-analyzer/pyproject.toml Outdated

Copilot AI review requested due to automatic review settings July 12, 2026 08:04

RonShakutai force-pushed the ronshakutai/fix-ci-poetry-timeout branch from a029bf8 to 7340a02 Compare July 12, 2026 08:04

Copilot started reviewing on behalf of RonShakutai July 12, 2026 08:04 View session

RonShakutai changed the title ~~fix(ci): revert analyzer to Python <3.14 to fix Poetry resolve hang~~ ci: use uv for test dependency installation (fixes Poetry resolve hang) Jul 12, 2026