Modernize Python repos: pyproject.toml + uv + semantic-release

[feanil]

Context

The openedx org is standardizing on modern Python tooling across all maintained Python
packages. This issue tracks the work of applying that standard to repos that still use
legacy tooling.

Why:

• pip-tools is being retired in favour of uv, which is faster and produces a single
  reproducible uv.lock lockfile (no more per-environment .txt files)
• pyproject.toml (PEP 621/735) consolidates package metadata and dependency
  declarations in one file, eliminating setup.py, setup.cfg, and requirements/*.txt
• python-semantic-release automates version bumps and PyPI publishing from conventional
  commit messages, removing manual release steps

Reference implementation: openedx/sample-plugin- The files to
model after are backend/pyproject.toml, backend/tox.ini, backend/Makefile, and
.github/workflows/backend-ci.yml.

Supporting infrastructure:

• edx_lint write_uv_constraints (feat: add write_uv_constraints command to manage uv constraint-dependencies edx-lint#537) — syncs global version
  constraints from edx-lint into each repo's [tool.uv].constraint-dependencies

---

Which repos qualify?

A repo should be modernized if it meets all of the following:

• Is a Python package published (or intended to be published) to PyPI
• Uses pip-compile / requirements/*.txt for dependency management
• Uses setup.py or setup.cfg for package metadata (or a pyproject.toml that
  still uses dynamic = [..., "dependencies"] pointing to a requirements file)

---

Work per repo

---

1. Consolidate package metadata into pyproject.toml

Move all package metadata into a single pyproject.toml using PEP 621. This means
replacing setup.py/setup.cfg, declaring a static dependencies list under
[project], and using setuptools-scm for version management from git tags.

Reference: backend/pyproject.toml ([project], [build-system], and [tool.setuptools_scm] sections)

• Migrate metadata from setup.cfg/setup.py into [project]
• Declare dependencies as a static list (not dynamic from a requirements file)
• Configure setuptools-scm for version discovery
• Delete setup.py and setup.cfg

Note: The sample-plugin houses the python library in a subdirectory and so has
to set the root setting in tool.setuptools_scm to ... This should not be
necessary for any other repo and should be omitted.

---

2. Switch dependency management from pip-compile to uv

Replace all requirements/*.in + requirements/*.txt files with PEP 735
[dependency-groups] in pyproject.toml and a single uv.lock lockfile. The
standard groups are test-base, test, quality, doc, ci, and dev — with
additional version-matrix groups as needed. Update tox.ini to use tox-uv's
uv-venv-lock-runner, update the Makefile to use uv lock/uv sync, and update
CI to install uv and run uv run tox.

Reference: backend/pyproject.toml ([dependency-groups] and [tool.uv] sections), backend/tox.ini, backend/Makefile, .github/workflows/backend-ci.yml

• Add [dependency-groups] to pyproject.toml covering test, quality, doc, ci, and dev
• Add [tool.edx_lint].uv_constraints for any repo-specific version overrides,
  then run edx_lint write_uv_constraints to populate [tool.uv].constraint-dependencies
• Generate uv.lock and commit it
• Delete the requirements/ directory
• Update tox.ini to use tox-uv>=1 and uv-venv-lock-runner with dependency_groups
• Update Makefile targets (upgrade, compile-requirements, requirements)
• Update CI to install uv via astral-sh/setup-uv, install deps via uv sync --group ci,
  and run tests via uv run tox

---

3. Add semantic-release

Configure python-semantic-release so that pushing a conventional commit to main
automatically cuts a version, tags it, and publishes to PyPI. Add commitlint to enforce
conventional commit format on PRs.

Reference: backend/pyproject.toml ([tool.semantic_release] section), .github/workflows/release.yml

Note: The sample plugin overrides minor_tags because it wants to publish new
minor versions on docs changes. That is not needed in our other libraries. They
should use the default value unless you know for sure you need to use something
different.

• Add [tool.semantic_release] config to pyproject.toml with a build_command
  that sets SETUPTOOLS_SCM_PRETEND_VERSION at build time
• Add a release.yml workflow that runs CI then publishes to PyPI via OIDC
• Add a commitlint.yml workflow to enforce conventional commits on PRs
• Confirm PyPI trusted publisher (OIDC) is configured for the repo

---

Notes

• [tool.uv].constraint-dependencies is machine-managed. Never edit it directly.
  Repo-specific version overrides belong in [tool.edx_lint].uv_constraints.
• uv sync does not put tools on PATH. Use uv run tox in CI, not bare tox.
• A new framework version may require bumping requires-python. If the new version
  drops an older Python, bump accordingly and remove that Python from the tox envlist.
• OEP-67 ADR documenting this migration is in progress (link TBD once merged).

[xitij2000]

@feanil Sorry if this was already discussed and I'm jumping in too late, but I'd like to recommend hatch instead of uv as the build backend. It supports build environments which will allow replacing tox as well. It's a pypa project so it follows PEP standards closely and can still use uv for installing packages.
Sorry if this has already been discussed and eliminated.

[feanil]

@xitij2000 My understanding is that the build backend is still setuptools and we're just using uv for dependency management. Hatch seems promising as a setuptools replacement but I haven't evaluated it too deeply yet because I didn't want to add complexity to the switch from setup.py to pyproject.toml by also changing the build backend at the same time. Let me know if I misunderstood something about the ecosystem here.

[farhan]

• related to OEP-67: Add uv and pyproject.toml as backend standards, archive OEP-18 openedx-proposals#784

[bradenmacdonald]

Reference implementation: openedx/sample-plugin- The files to model after are backend/pyproject.toml, backend/tox.ini, backend/Makefile, and .github/workflows/backend-ci.yml.

I just want to point out that sample-plugin/backend is not using a src/ directory layout, and I would highly recommend that we try to adopt that where feasible, for these reasons and this reason (editable installs of repos like opaque-keys will not be usable by important tools like mypy because they're not using a src/ directory, but repos like openedx-core that do use a src directory work fine).

[feanil]

@bradenmacdonald thanks, that seems like another general improvement we should make across all of our libraries as we modernize.

[feanil]

PR for that refactor in the sample-plugin repo: openedx/sample-plugin#32 if anyone has a sec to review.

[salman2013]

Uses setup.py/setup.cfg — pip-compile / requirements/*.txt — PUBLISHED on PyPI

• django-wiki - https://github.com/openedx/django-wiki
• XBlock - https://github.com/openedx/XBlock
• event-tracking - https://github.com/openedx/event-tracking
• edx-ora2 - https://github.com/openedx/edx-ora2
• acid-block - https://github.com/openedx/acid-block
• xblock-sdk - https://github.com/openedx/xblock-sdk
• xblock-image-explorer - https://github.com/openedx/xblock-image-explorer
• opaque-keys - https://github.com/openedx/opaque-keys
• i18n-tools - https://github.com/openedx/i18n-tools
• edx-submissions - https://github.com/openedx/edx-submissions
• xblock-drag-and-drop-v2 - https://github.com/openedx/xblock-drag-and-drop-v2
• edx-val - https://github.com/openedx/edx-val
• openedx-webhooks - https://github.com/openedx/openedx-webhooks
• xblock-google-drive - https://github.com/openedx/xblock-google-drive
• edx-milestones - https://github.com/openedx/edx-milestones
• edx-search - https://github.com/openedx/edx-search
• edx-lint - https://github.com/openedx/edx-lint
• auth-backends - https://github.com/openedx/auth-backends
• edx-rest-api-client - https://github.com/openedx/edx-rest-api-client
• edx-proctoring - https://github.com/openedx/edx-proctoring
• edx-organizations - https://github.com/openedx/edx-organizations
• ccx-keys - https://github.com/openedx/ccx-keys
• django-pyfs - https://github.com/openedx/django-pyfs
• xblock-lti-consumer - https://github.com/openedx/xblock-lti-consumer
• edx-drf-extensions - https://github.com/openedx/edx-drf-extensions
• edx-django-sites-extensions - https://github.com/openedx/edx-django-sites-extensions
• django-user-tasks - https://github.com/openedx/django-user-tasks
• django-config-models - https://github.com/openedx/django-config-models
• edx-enterprise - https://github.com/openedx/edx-enterprise
• web-fragments - https://github.com/openedx/web-fragments
• edx-celeryutils - https://github.com/openedx/edx-celeryutils
• help-tokens - https://github.com/openedx/help-tokens
• RecommenderXBlock - https://github.com/openedx/RecommenderXBlock
• DoneXBlock - https://github.com/openedx/DoneXBlock
• edx-ace - https://github.com/openedx/edx-ace
• edx-enterprise-data - https://github.com/openedx/edx-enterprise-data
• completion - https://github.com/openedx/completion
• edx-toggles - https://github.com/openedx/edx-toggles
• edx-django-utils - https://github.com/openedx/edx-django-utils
• xss-utils - https://github.com/openedx/xss-utils
• code-annotations - https://github.com/openedx/code-annotations
• edx-rbac - https://github.com/openedx/edx-rbac
• edx-when - https://github.com/openedx/edx-when
• openedx-calc - https://github.com/openedx/openedx-calc
• openedx-chem - https://github.com/openedx/openedx-chem
• staff-graded-xblock - https://github.com/openedx/staff-graded-xblock
• super-csv - https://github.com/openedx/super-csv
• edx-bulk-grades - https://github.com/openedx/edx-bulk-grades
• api-doc-tools - https://github.com/openedx/api-doc-tools
• xblock-free-text-response - https://github.com/openedx/xblock-free-text-response
• xblock-in-video-quiz - https://github.com/openedx/xblock-in-video-quiz
• enmerkar-underscore - https://github.com/openedx/enmerkar-underscore
• pytest-repo-health - https://github.com/openedx/pytest-repo-health
• license-manager - https://github.com/openedx/license-manager
• taxonomy-connector - https://github.com/openedx/taxonomy-connector
• event-routing-backends - https://github.com/openedx/event-routing-backends
• olxcleaner - https://github.com/openedx/olxcleaner
• openedx-events - https://github.com/openedx/openedx-events
• openedx-filters - https://github.com/openedx/openedx-filters
• openedx-core - https://github.com/openedx/openedx-core
• codejail-includes - https://github.com/openedx/codejail-includes
• event-bus-kafka - https://github.com/openedx/event-bus-kafka
• django-require - https://github.com/openedx/django-require
• openedx-atlas - https://github.com/openedx/openedx-atlas
• FeedbackXBlock - https://github.com/openedx/FeedbackXBlock
• xblock-skill-tagging - https://github.com/openedx/xblock-skill-tagging
• openedx-ledger - https://github.com/openedx/openedx-ledger
• edx-enterprise-subsidy-client - https://github.com/openedx/edx-enterprise-subsidy-client
• tutor-contrib-aspects - https://github.com/openedx/tutor-contrib-aspects
• event-bus-redis - https://github.com/openedx/event-bus-redis
• platform-plugin-aspects - https://github.com/openedx/platform-plugin-aspects
• xblocks-contrib - https://github.com/openedx/xblocks-contrib
• forum - https://github.com/openedx/forum
• enterprise-integrated-channels - https://github.com/openedx/enterprise-integrated-channels
• crowdsourcehinter - https://github.com/openedx/crowdsourcehinter
• codejail - https://github.com/openedx/codejail
• openedx-authz - https://github.com/openedx/openedx-authz

Uses setup.py/setup.cfg + requirements/*.txt — NOT on PyPI

• credentials-themes - https://github.com/openedx/credentials-themes
• mockprock - https://github.com/openedx/mockprock
• edx-repo-health - https://github.com/openedx/edx-repo-health
• openedx-webhooks-data-schema - https://github.com/openedx/openedx-webhooks-data-schema
• enterprise-catalog - https://github.com/openedx/enterprise-catalog
• enterprise-access - https://github.com/openedx/enterprise-access
• enterprise-subsidy - https://github.com/openedx/enterprise-subsidy
• xapi-db-load - https://github.com/openedx/xapi-db-load
• codejail-service - https://github.com/openedx/codejail-service
• openedx-user-groups - https://github.com/openedx/openedx-user-groups
• cc2olx - https://github.com/openedx/cc2olx

Uses setup.py/setup.cfg — NO requirements/*.txt — NOT on PyPI

• pr_watcher_notifier - https://github.com/openedx/pr_watcher_notifier