feat(cli): add --device-tag argument for per-device filtering

[ChristopherJHart]

Description

Adds a --device-tag CLI argument (also settable via NAC_TEST_DEVICE_TAG env var) that filters test execution to only devices whose tags list in the data model contains the specified value. This enables a single set of test scripts to serve dual purposes — scoped to specific devices (e.g., --device-tag migration) or fabric-wide (no flag) — without duplicating test code.

Although SD-WAN is the catalyst for this change and the first architecture where the pattern is implemented (driven by an immediate customer need for post-migration validation), the intent is for each architecture to adopt this mechanism to filter operational tests down to a user-defined subset of devices. The CLI argument and env var plumbing are architecture-agnostic; only the per-architecture base class/resolver needs to read the env var and filter accordingly.

Related Issue(s)

• Companion PR: feat(sdwan): add NAC_TEST_DEVICE_TAG filtering to device discovery nac-test-pyats-common#39

Type of Change

• New feature (non-breaking change that adds functionality)

Test Framework Affected

• PyATS
• Robot Framework
• Both
• N/A (not test-framework specific)

Network as Code (NaC) Architecture Affected

• Catalyst SD-WAN (SDWAN Manager / vManage)
• All architectures

SD-WAN is the first implementation. The CLI/orchestrator plumbing is architecture-agnostic — future architectures add filtering by reading NAC_TEST_DEVICE_TAG in their resolver or base class.

Platform Tested

• macOS (version tested: Darwin 25.5.0)

Key Changes

• CLI (nac_test/cli/main.py): New --device-tag option with NAC_TEST_DEVICE_TAG envvar support
• CombinedOrchestrator (nac_test/combined_orchestrator.py): Accepts and stores device_tag, forwards to PyATSOrchestrator
• PyATSOrchestrator (nac_test/pyats_core/orchestrator.py): Injects NAC_TEST_DEVICE_TAG into subprocess env for API tests; sets env var before D2D device discovery
• DeviceExecutor (nac_test/pyats_core/execution/device/device_executor.py): Injects NAC_TEST_DEVICE_TAG into D2D subprocess env

Testing Done

• Unit tests added/updated
• Integration tests performed
• Manual testing performed:
  • PyATS tests executed successfully
  • Robot Framework tests executed successfully
  • D2D/SSH tests executed successfully (if applicable)
  • HTML reports generated correctly
• All existing tests pass (pytest / pre-commit run -a)

Test Commands Used

# Pre-commit (ruff, mypy, bandit all pass)
pre-commit run -a

# Integration test: no devices tagged → no API calls
nac-test -d ./data -d ./defaults -t ./tests/templates --device-tag migration --pyats -o ./tests/results/sdwan

# Integration test: one device tagged → only that device tested
# (added tags: ["migration"] to one router in data model)
nac-test -d ./data -d ./defaults -t ./tests/templates --device-tag migration --pyats -o ./tests/results/sdwan

# Integration test: no flag → all devices tested (regression-free)
nac-test -d ./data -d ./defaults -t ./tests/templates --pyats -o ./tests/results/sdwan

Checklist

• Code follows project style guidelines (pre-commit run -a passes)
• Self-review of code completed
• Code is commented where necessary (especially complex logic)
• Documentation updated (if applicable)
• No new warnings introduced
• Changes work on both macOS and Linux
• CHANGELOG.md updated (if applicable)

Additional Notes

• The value is threaded via environment variables because pyATS tests run in separate subprocesses — Python objects cannot be passed across the process boundary
• The filtering itself happens in nac-test-pyats-common (companion PR Bump robotframework-pabot from 2.10.0 to 2.11.0 #39) — this PR only handles the CLI argument and env var plumbing
• CI usage pattern:

  # Migration stage (only tagged devices)
  nac-test -d ./data -t ./tests/templates --device-tag migration
  
  # Fabric-wide operational stage (all devices)
  nac-test -d ./data -t ./tests/templates

• Test scripts can also declare groups = ["migration"] to be excluded from fabric-wide runs via --exclude migration, enabling both device-level AND test-level scoping

[oboehmer]

Thanks for the PR. I feel this won’t be the last device filter we‘ll need to limit the d2d tests to specific devices, I wonder if we can come up with something more generic which avoids adding multiple options we will need to wire through nac-test.
Maybe we could implement a device-filter option which would take a key,value parameter to limit on tag, name, Maybe device type, and possibly more, similar to radkit‘s inventory.filter()? What do you think?