OpenStack QE

openstack-qe provides Kubernetes-hosted smoke and regression validation for an OpenStack cloud.

Current test lanes cover:

• minimal instance boot validation for the fastest first proof
• API checks for auth, endpoint discovery, image lookup, and network prerequisites
• instance lifecycle checks for boot, addressing, and console output access
• Tobiko-backed SSH checks for guest reachability and command execution
• Barbican, Glance, Neutron, Octavia, and Magnum basics aligned with existing QE scenarios

The project runs through Argo Workflows and uses mounted OpenStack credentials.

Why This Approach

This project uses pytest as the primary framework for OpenStack end-to-end workflows that need:

• Nova boot-from-disk and boot-from-volume coverage across the available images
• Nova lifecycle checks such as live migration, cold migration, resize, snapshot, and rebuild
• connectivity checks that prove ping fails before a security-group rule and succeeds after it
• FLEX-specific validation for static vendor data and cloud-init behavior, plus heavier Magnum, Octavia, and Blazar scenarios outside the smallest daily lane

Detailed rationale and trade-offs: docs/WHY_APPROACH.md

Requirement List

Representative examples from the requirement set:

• Nova boot coverage across both boot-from-disk and boot-from-volume paths for every available image.
• Nova lifecycle validation such as live migration, cold migration, resize, snapshot, and rebuild flows.
• Connectivity behavior checks that prove ping fails before the right security-group rule and succeeds after it.
• FLEX-specific validation for static vendor data and cloud-init interaction.
• Service basics for Neutron, Glance, Cinder, Swift, Octavia, Barbican, Magnum, Heat, Blazar, and Zaqar.

For the explained summary and the full requirement list, see docs/REQUIREMENT_LIST.md.

Layout

• runner/: CLI entrypoint, config loading, result publishing, and shared OpenStack helpers
• tests/api/: minimal OpenStack API smoke checks
• tests/instance/: boot-and-cleanup validation
• tests/tobiko/: SSH-only Tobiko validation
• deploy/: namespace, RBAC, PVC, config, and secret contracts
• workflows/: Argo workflow templates and schedules
• gitops/: Argo CD app-of-apps manifests and the tracked runner image reference
• bootstrap/: bastion-host bootstrap scripts for seeding Argo CD and repo credentials
• docs/: operator notes, suite definitions, and behavior coverage design

Deployment Model

The deployment path is pull-based:

1. GitHub Actions runs lightweight checks.
2. On main, GitHub Actions builds and pushes an immutable GHCR image.
3. GitHub Actions updates the tracked runner image in Git.
4. Argo CD, running in the cluster, pulls the changed Git state and syncs it.
5. Argo Workflows runs the QE suites using the synced runner image.

Argo CD pulls manifest changes from Git and syncs them in the cluster. Bootstrap steps are documented in bootstrap/README.md.

Quick Start

1. Create a Python 3.10+ virtual environment (required).

python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip

2. Install dependencies:

pip install -e '.[dev,tobiko]'

Note for zsh: the quotes are required so the shell does not treat [] as a glob pattern. If python --version is below 3.10, installation will fail because the project declares requires-python = ">=3.10" in pyproject.toml.

3. Export the required runtime variables:

export OS_CLIENT_CONFIG_FILE=/path/to/clouds.yaml
export OS_CLOUD=mycloud
export QE_IMAGE=ubuntu-22.04
export QE_FLAVOR=m1.small
export QE_FIXED_NETWORK=private
export QE_BOOT_MODE=auto
export QE_FLOATING_NETWORK=public
export QE_SSH_USERNAME=ubuntu
export QE_SSH_KEY_PATH=/path/to/private-key
export QE_PARALLEL_WORKERS=1

QE_BOOT_MODE supports auto, disk, and volume. auto uses volume-backed boot automatically when the selected flavor has zero disk.

4. Run the smallest possible instance test:

python -m runner.cli --suite minimal

This suite only checks that one instance can be created and reaches ACTIVE.

5. Run the smoke suite:

python -m runner.cli --suite smoke

6. Run the smoke suite with 5 tests in parallel:

python -m runner.cli --suite smoke --parallel 5

You can also set QE_PARALLEL_WORKERS=5 to make parallel execution the default.

To run one specific test instead of the whole suite, pass the pytest nodeid after the suite selection. Example:

python -m runner.cli --suite minimal tests/minimal/test_boot_minimal.py::test_instance_boots_successfully

Artifacts are written under artifacts/ by default:

• junit-<suite>.xml
• pytest-<suite>.log
• summary-<suite>.json
• summary-<suite>.txt
• status-<suite>.txt
• diagnostics/<suite>/: failure-focused artifacts such as server fault details and console logs

During a run, the console prints each test as it starts so it is clearer what is being validated. The final summary includes:

• total, passed, failed, error, and skipped counts
• one line per test with its result
• all artifact and log file paths saved for the run

If you want the complete unfiltered pytest and runner output, open artifacts/console-<suite>.log.

Runtime Contract

The service expects:

• clouds.yaml mounted into the container
• a private SSH key secret mounted for guest reachability checks
• a PVC or object storage-compatible mount for workflow artifacts
• Argo parameters for suite selection and runner image selection
• optional service-specific QE flags and inputs for Barbican, Octavia, and Magnum checks

Notes

• The repo owns its own build, test, and deployment pipeline.
• Argo CD syncs manifests in the cluster; Argo Workflows executes the in-cluster test runs.
• Test selection is organized by suites such as smoke, nightly, and manual.
• Artifacts are designed for CI, Argo, and operator troubleshooting workflows.