SG-38126 Review and Fixups CI Tests and Code Coverage

[julien-lang]

Improve Test Results and Code Coverage reports. Make sure they are properly passed to Azure Pipelines and Codecov.io

Align with other repositories:

• SG-38126 Review and Fixups CI Tests and Code Coverage python-api#439
• SG-38126 Review and Fixups CI Tests and Code Coverage sg-jira-bridge#103

Implemented by:

• SG-38126 Create some basic test coverage shotgunEvents#101
• SG-38126 Update CI - Test results and code coverage tk-core#1082
• SG-38126 Update CI - Test results and code coverage tk-framework-desktopserver#308

Azure Pipeline CI Improvements

• Define job parameter in the "formal" form without a default value to make sure CI fails if the parameters are not provided
• Attempt to make the playbook easier to read
• Simplify the code coverage upload mechanism to codecov.io
• Drop pytest-azurepipeline
  • This package was prviously used to automatically upload test report and code coverage to Azure Pipelines which is very convenient.
  • Unfortunately, the package does not seem to be maintained anymore. And it is using outdated versions the PublishTestResults and PublishCodeCoverageResults Azure Pipeline tasks.
• Ensure all test results and code coverage are combine and send
  • tk-core and tk-framework-desktopserver have additional tests runnings which produce result and coverage files.

[Copilot]

Pull request overview

This PR enhances the CI/CD pipeline's code coverage reporting and test execution infrastructure. It modernizes the testing workflow by replacing pytest-azurepipelines with explicit pytest plugins, implements OS-specific codecov uploader URLs, and consolidates coverage upload logic across all platforms.

Changes:

• Replaced pytest-azurepipelines with individual pytest plugins (pytest, pytest-cov, pytest-nunit)
• Added OS-specific codecov download URLs to centralize platform handling
• Unified code coverage upload workflow using Azure Pipeline tasks instead of OS-specific bash/powershell scripts

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
internal/run-tests.yml	Added codecov download URLs for each OS platform with macOS pinned to v0.7.3 due to architecture constraints
internal/run-tests-with.yml	Refactored test execution and coverage reporting to use unified Azure Pipeline tasks and explicit pytest plugins
README.md	Added codecov.io coverage status badge

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[carlos-villavicencio-adsk]

LGTM. Although I shared some questions here and there.

[carlos-villavicencio-adsk]

Nice ⭐

[carlos-villavicencio-adsk]

Is this going to affect any other repos where coverage is not updated yet?

[julien-lang]

All repos using tk-ci-tools will generate coverage because the command is made in this same file.
If the repos does not have a tests/ folder, then, this playbook is not called.

[carlos-villavicencio-adsk]

It's also worth considering mentioning SG-36012 as well.