Open
Conversation
Signed-off-by: Sergei Volkov <sevolkov@redhat.com>
Contributor
📝 WalkthroughWalkthroughA new test module for EVPN connectivity is introduced, containing a test class with seven test methods. Each method includes docstrings with preconditions, steps, and checks for various EVPN scenarios. No executable test logic or assertions are implemented. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs). Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Report bugs in Issues Welcome! 🎉This pull request will be automatically processed with the following features: 🔄 Automatic Actions
📋 Available CommandsPR Status Management
Review & Approval
Testing & Validation
Container Operations
Cherry-pick Operations
Label Management
✅ Merge RequirementsThis PR will be automatically approved when the following conditions are met:
📊 Review ProcessApprovers and ReviewersApprovers:
Reviewers:
Available Labels
💡 Tips
For more information, please refer to the project documentation or contact the maintainers. |
openshift-virtualization-qe-bot
added
needs-rebase
tox:verify-tc-requirement-polarion:running
|
D/S test |
openshift-virtualization-qe-bot
added
the
tox:verify-tc-requirement-polarion:failed
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/network/bgp/evpn/test_evpn_connectivity.py`:
- Line 5: Replace the STP reference placeholder string "<will be added once it's
merged>" in the STP Reference comment at the top of test_evpn_connectivity.py
with the actual design doc URL once that PR is merged; ensure the comment reads
"STP Reference: <actual-link>" (keeping the same comment formatting) so the test
has a proper traceability link.
- Line 30: The Polarion ID placeholder "CNV-00000" used in the
pytest.mark.polarion decorators must be replaced with the real Polarion test
case IDs (or annotated with a clear TODO) so reporting/tracking works when tests
are enabled; update each occurrence of pytest.mark.polarion("CNV-00000") in this
file (all seven decorators) to the actual Polarion IDs or add a TODO comment
next to each decorator indicating to insert the proper ID, and ensure the change
is coordinated with the toggle that controls test activation (__test__ = False)
so tests only run once valid IDs are set.
- Around line 30-31: The test methods in this class (e.g.,
test_connectivity_between_udn_vms) are missing the required tier marker; add the
pytest tier3 marker decorator (`@pytest.mark.tier3`) to this method and the other
six test methods in the same class so each test method is annotated with
`@pytest.mark.tier3` (apply the same change to all 7 test methods referenced in
the file); ensure you keep any existing markers (e.g., `@pytest.mark.polarion`)
and place `@pytest.mark.tier3` alongside them.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 054ca31d-268e-40cf-8516-76840bee584d
📒 Files selected for processing (2)
tests/network/bgp/evpn/__init__.pytests/network/bgp/evpn/test_evpn_connectivity.py
|
|
||
| Tests are aimed to cover the EVPN integration for OpenShift Virtualization VMs. | ||
|
|
||
| STP Reference: <will be added once it's merged> |
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
LOW: Update STP reference placeholder.
The STP reference contains placeholder text <will be added once it's merged>. Once the design doc PR is merged, update this with the actual link for traceability.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/network/bgp/evpn/test_evpn_connectivity.py` at line 5, Replace the STP
reference placeholder string "<will be added once it's merged>" in the STP
Reference comment at the top of test_evpn_connectivity.py with the actual design
doc URL once that PR is merged; ensure the comment reads "STP Reference:
<actual-link>" (keeping the same comment formatting) so the test has a proper
traceability link.
Contributor
Author
There was a problem hiding this comment.
will be updated once the STP is merged (soon).
Contributor
There was a problem hiding this comment.
@servolkov, understood! Looks like the design doc PR (#35 in the design-docs repo) is nearly there. Just don't forget to swap in the actual URL before this PR lands in main. 👍
🧠 Learnings used
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-09-29T19:05:24.987Z
Learning: For PR `#1904` test execution, the critical validation point is test_connectivity_over_migration_between_localnet_vms which should fail gracefully on cloud clusters but pass on bare-metal/PSI clusters, representing the core nmstate conditional logic functionality.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 1776
File: tests/network/bgp/conftest.py:35-54
Timestamp: 2025-08-28T12:30:40.692Z
Learning: The BGP test suite in tests/network/bgp/ relies on session-scoped validation in tests/network/conftest.py via the network_sanity fixture's _verify_bgp() function, which validates required environment variables (VLAN_TAG, EXTERNAL_FRR_STATIC_IPV4, BGP_CLUSTER_DOMAIN_GROUP) before any BGP tests run, making individual fixture-level validation redundant.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3598
File: tests/network/sriov/conftest.py:62-71
Timestamp: 2026-01-28T17:36:14.188Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, sanity checks validate that the cluster supports at least one IP family (IPv4 or IPv6) before SR-IOV tests run. Therefore, defensive guards checking for empty IP addresses in SR-IOV VM fixtures (like `sriov_vm` in tests/network/sriov/conftest.py) are not necessary—if neither `ipv4_supported_cluster` nor `ipv6_supported_cluster` is True, the cluster is considered broken and will fail sanity validation.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 1776
File: tests/network/bgp/conftest.py:62-87
Timestamp: 2025-08-28T12:34:56.341Z
Learning: The BGP test infrastructure in tests/network/bgp/ currently uses static IP assignment via EXTERNAL_FRR_STATIC_IPV4, but the long-term goal is to transition to DHCP for IP allocation. Additional validation of EXTERNAL_FRR_STATIC_IPV4 in individual fixtures is considered redundant since validation already occurs at the session level.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 2920
File: utilities/network.py:31-31
Timestamp: 2025-12-07T13:52:56.070Z
Learning: In the openshift-virtualization-tests repository, it is acceptable for utilities/ modules to import from tests/network/libs/ (e.g., utilities/network.py importing from tests/network/libs/sriovnetworknode). The libs modules under tests/network/libs/ are library/tool modules, not test code, and this import direction does not violate architectural principles in this codebase.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 1904
File: tests/network/conftest.py:348-362
Timestamp: 2025-10-27T15:30:06.412Z
Learning: In tests/network/conftest.py, the _verify_nmstate_running_pods function currently runs unconditionally in network_sanity, but rnetser plans to implement marker-based conditional checking (following the pattern of _verify_dpdk, _verify_sriov, etc.) in a future PR after adding nmstate markers to the relevant tests.
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:48-57
Timestamp: 2026-01-19T07:54:03.630Z
Learning: The pytest marker requirement ("all new tests MUST have markers") applies only to integration/functional tests in the `tests/` directory (e.g., `tests/virt/`, `tests/storage/`, `tests/network/`), not to unit tests in `utilities/unittests/`. Unit test files in `utilities/unittests/` (such as test_oadp.py, test_sanity.py, test_operator.py, etc.) do not use pytest markers and this is the established repository convention.
</learning>
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3062
File: conftest.py:333-333
Timestamp: 2025-12-16T20:11:03.645Z
Learning: In the openshift-virtualization-tests repository, when determining smoke test impact for changes affecting py_config["os_login_param"], follow this verification methodology: (1) Find all smoke tests: `rg "pytest.mark.smoke" --type=py -B2 | grep "def test_"`, (2) For each smoke test file, search for VM creation patterns: `rg "VirtualMachineForTests|running_vm|VirtualMachineForTestsFromTemplate|wait_for_ssh|check_ssh_connectivity"`, (3) Trace the dependency chain: smoke test → VirtualMachineForTests/running_vm() → wait_for_ssh_connectivity() (default enabled) → vm.login_params property → py_config["os_login_param"][vm.os_flavor], (4) Check utilities/virt.py for login_params usage: `rg "os_login_param|login_params" utilities/virt.py -C3`. Any smoke test creating VMs with default SSH connectivity checks (running_vm with check_ssh_connectivity=True) depends on os_login_param, even if the test doesn't directly reference it.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/test_ip_persistence.py:23-34
Timestamp: 2026-01-12T11:09:08.889Z
Learning: In tests/network/provider_migration/test_ip_persistence.py, user servolkov prefers compound assertions when they provide comprehensive error messages showing all relevant diagnostic information (e.g., both MAC and IP expected/actual values), rather than splitting them into multiple assertions that would repeat similar patterns, prioritizing the DRY principle over Ruff PT018 recommendations.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3598
File: tests/network/sriov/libsriov.py:10-72
Timestamp: 2026-02-10T16:55:49.690Z
Learning: In tests/network/sriov/libsriov.py, user azhivovk considers the SR-IOV test helper functions (vm_sriov_mac, sriov_vm, sriov_cloud_init_data) sufficiently clear from their names and flow, and does not require docstrings. The module is a test-specific library rather than a large common library, making docstrings redundant.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3062
File: conftest.py:333-333
Timestamp: 2025-12-16T15:09:49.597Z
Learning: In the openshift-virtualization-tests repository, when conftest.py or utilities/bitwarden.py changes affect py_config["os_login_param"], smoke test impact must be determined by: (1) finding all smoke tests using `rg "pytest.mark.smoke"`, (2) checking each for VM creation patterns (VirtualMachineForTests, running_vm, VirtualMachineForTestsFromTemplate), (3) tracing whether running_vm is called with default check_ssh_connectivity=True, which accesses vm.login_params property that reads py_config["os_login_param"][vm.os_flavor]. The dependency chain is: smoke test → VM creation → running_vm → wait_for_ssh_connectivity → vm.login_params → os_login_param. Any smoke test creating VMs with SSH connectivity (the default) depends on os_login_param.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 2469
File: utilities/sanity.py:139-142
Timestamp: 2025-11-08T07:36:57.616Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers to keep refactoring PRs (like PR `#2469`) strictly focused on moving/organizing code into more granular modules without adding new functionality, error handling, or behavioral changes. Such improvements should be handled in separate PRs.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3873
File: tests/network/localnet/test_non_udn_localnet.py:7-9
Timestamp: 2026-02-25T10:52:23.378Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, networking infrastructure requirements such as nmstate, localnet bridge mappings, and NIC availability are NOT considered special/platform-specific infrastructure that would warrant a tier3 marker. These are standard test environment capabilities in this repository's test environment, and tests requiring them do not automatically qualify as tier3 unless they involve other truly platform-specific/complex/time-consuming characteristics beyond standard network test infrastructure.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 1776
File: libs/net/node_network.py:25-31
Timestamp: 2025-08-20T23:43:28.117Z
Learning: In the RedHatQE/openshift-virtualization-tests project, servolkov's team always uses bare metal (BM) clusters with IPv4 setup in their testing environment, making defensive checks for IPv4 data presence potentially redundant in their networking code.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3024
File: tests/network/connectivity/utils.py:17-17
Timestamp: 2025-12-15T12:33:06.686Z
Learning: In the test suite, ensure the ipv6_network_data fixture returns a factory function (Callable) and that all call sites invoke it to obtain the actual data dict, i.e., use ipv6_network_data() at call sites. This enables future extensibility for configuring secondary interfaces' IP addresses without changing call sites. Apply this pattern to all Python test files under tests.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3196
File: tests/network/upgrade/test_upgrade_network.py:4-4
Timestamp: 2025-12-22T16:27:40.244Z
Learning: For PRs that remove tests, rely on pytest --collect-only to verify the test discovery results (which tests are selected/deselected) and ensure the removal is clean and the test module remains functional. Full test execution is not required for test deletion PRs. This guideline applies to test files anywhere under the tests/ directory (e.g., tests/network/upgrade/test_upgrade_network.py) and should be used for similar test-deletion scenarios across the repository.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3358
File: tests/network/sriov/test_sriov.py:21-21
Timestamp: 2026-01-07T09:52:12.342Z
Learning: When a PR only removes or modifies pytest markers in tests (e.g., removing pytest.mark.post_upgrade) and the test logic remains unchanged, prefer verifying with pytest --collect-only instead of running the full test suite. This validates that marker usage and test selection behavior are preserved. If the test logic changes, or markers affect behavior beyond collection, run the full test suite to confirm.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:50-52
Timestamp: 2026-01-12T11:24:13.825Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when catching exceptions in Python, use LOGGER.error before re-raising and do not replace it with LOGGER.exception in except blocks. This follows the established pattern across the codebase.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3366
File: tests/storage/cdi_clone/test_clone.py:5-9
Timestamp: 2026-01-12T14:25:05.723Z
Learning: In Python tests and utility code across the repository, bitmath.parse_string_unsafe correctly parses Kubernetes quantities (e.g., '4Gi', '512Mi', PVC storage requests) without supplying system=bitmath.NIST. There are 30+ usages indicating this is the standard behavior. Reviewers should verify that code that builds or compares quantity strings does not pass the NIST parameter, and if a new test relies on quantity parsing, assume no NIST parameter is required unless explicitly documented.
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 3404
File: tests/virt/upgrade/conftest.py:291-301
Timestamp: 2026-01-14T04:08:23.032Z
Learning: In the openshift-virtualization-tests repository, when using VirtualMachine objects from ocp-resources in tests, if vm.ready is True, vm.vmi is guaranteed to exist. Therefore, you can access vm.vmi.instance.status or vm.vmi attributes without additional defensive checks (e.g., if vm.vmi: ...). Do not rely on vm.vmi being present when vm.ready may be False; guard those code paths accordingly. This guideline applies to tests under tests/ (notably in virt/upgrade/conftest.py and related test modules) and should be followed for any code paths that assume vm.vmi exists only when vm.ready is True.
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 3376
File: tests/network/general/test_ip_family_services.py:96-96
Timestamp: 2026-01-18T09:44:17.044Z
Learning: In the openshift-virtualization-tests repository, function-scoped fixtures must use pytest.fixture() with empty parentheses (not pytest.fixture without parentheses). This repo follows this convention despite Ruff PT001. Apply this consistently to all Python test files under tests/ (not just this one) to maintain repository-wide consistency.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3273
File: tests/network/connectivity/test_ovs_linux_bridge.py:5-9
Timestamp: 2026-01-18T13:18:48.808Z
Learning: In tests/network/connectivity/test_ovs_linux_bridge.py and similar test files, prefer importing ipaddress as a module and using qualified calls like ipaddress.ip_interface(...) rather than from ipaddress import ip_interface. This preserves module context for readability, especially when chaining properties (e.g., ipaddress.ip_interface(...).ip). This is an intentional exception to the general rule favoring specific imports, and should apply to test files under the tests directory where module context aids understanding.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3495
File: tests/network/third_part_ip_request/test_third_party_ip_request.py:4-12
Timestamp: 2026-01-18T14:51:50.846Z
Learning: In the openshift-virtualization-tests repository, tests consistently import pytest as a module (import pytest) and avoid from pytest import ...; this is the established pattern across 398+ test files. Do not flag or refactor imports to use specific pytest names in tests under tests/**. If a file already follows this pattern, leave it as is; this guideline applies broadly to Python test files under the tests directory.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3495
File: tests/network/third_part_ip_request/test_third_party_ip_request.py:4-12
Timestamp: 2026-01-18T14:51:50.846Z
Learning: In tests/network/ modules in the openshift-virtualization-tests repository, import network library modules from tests/network/libs using aliases with the as lib* convention (e.g., from tests.network.libs import cluster_user_defined_network as libcudn, from libs.net import netattachdef as libnad). This pattern improves readability and is consistently used across multiple network test files.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:1-8
Timestamp: 2026-01-20T01:03:13.139Z
Learning: In the openshift-virtualization-tests repository, Python imports should consistently use module-level imports for the logging module (i.e., import logging) rather than from logging import ... The established pattern spans 270+ files and should not be flagged for refactoring. Apply this guideline to Python files across the repo (e.g., tests/network/provider_migration/libprovider.py).
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 3559
File: utilities/infra.py:251-254
Timestamp: 2026-01-21T21:26:41.805Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when reviewing Python code, recognize that with Python 3.14 the syntax 'except ValueError, TypeError:' is valid if there is no 'as' clause, and should not be flagged as Python 2 syntax. If you use an 'as' binding (e.g., 'except (ValueError, TypeError) as e:'), parentheses are required. Ensure this pattern is version-consistent and not flagged as Python 2 syntax when 'as' is absent.
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 3571
File: tests/storage/storage_migration/utils.py:158-167
Timestamp: 2026-01-25T13:18:21.675Z
Learning: In reviews of the openshift-virtualization-tests repo (and similar Python code), avoid suggesting minor stylistic changes that require extra verification (e.g., removing dict.keys() checks for membership) unless the change has clear correctness or maintainability impact. Focus on fixes with observable behavior, security, performance, or maintainability benefits; defer low-impact style tweaks that are costly to verify.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3619
File: tests/network/user_defined_network/test_user_defined_network.py:97-97
Timestamp: 2026-01-27T17:18:49.973Z
Learning: In tests that exercise lookup_iface_status_ip from libs.net.vmspec, rely on the function's built-in descriptive error messages for failures. Do not add extra assertion messages for IP presence checks using this function; instead, assert on the function behavior or catch its exceptions as appropriate. This reduces duplication and clarifies failures.
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:48-54
Timestamp: 2026-01-28T16:08:33.862Z
Learning: Adhere to the repository convention for pytest parameterization: use positional arguments (tuples) in pytest.mark.parametrize with the first element being the argnames tuple, e.g. pytest.mark.parametrize(("kind", "name"), [("Backup", "test-backup")]), rather than using keyword arguments via argnames= and argvalues=. This applies to all test files in the repository; ensure any existing or new tests follow the positional pattern for parameterized tests to maintain consistency and readability.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3649
File: tests/network/user_defined_network/ip_specification/libipspec.py:1-4
Timestamp: 2026-01-29T05:30:13.982Z
Learning: In the openshift-virtualization-tests repository, Python imports should use module import style for the standard library 'json' (import json) rather than 'from json import ...'. This improves readability by making calls like json.loads and json.dumps explicit, and aligns with patterns used for modules like logging, ipaddress, cloudinit, and pytest. Apply this to all Python test files under tests/ (and similar test directories).
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3690
File: tests/after_cluster_deploy_sanity/test_after_cluster_deploy_sanity.py:65-65
Timestamp: 2026-02-02T17:41:12.759Z
Learning: In test files, keep test_* functions with simple one-line docstrings. For helper functions, utilities, and library code with non-obvious return values or side effects, use Google-style docstrings with Args, Returns, and Side effects sections. Do not require Google-style docstrings for pytest test functions themselves.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3708
File: tests/network/user_defined_network/ip_specification/test_ip_specification.py:141-145
Timestamp: 2026-02-02T19:37:50.989Z
Learning: In tests under tests/network that use traffic_generator helpers (e.g., is_tcp_connection), you can rely on pytest’s assertion introspection, descriptive test names, and internal logging from the helpers to diagnose failures. Do not add custom assertion failure messages for these helpers; instead ensure logs and clear test names convey the failure context.
Learnt from: RoniKishner
Repo: RedHatQE/openshift-virtualization-tests PR: 3697
File: tests/infrastructure/instance_types/test_common_vm_instancetype.py:53-98
Timestamp: 2026-02-03T07:34:34.184Z
Learning: In test files (Python, pytest), prefer using tier3 markers for categorization since tier2, tier1, and tier4 are not used in this repository. Do not rely on non-official markers; formalize and document the allowed markers in pytest.ini (or equivalent) to ensure consistent usage. When reviewing new tests, ensure markers align with this convention (only tier3, unless a project-wide decision defines additional tiers) and remove any unnecessary or undocumented markers.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3634
File: tests/network/migration/test_masquerade_connectivity_after_migration.py:116-116
Timestamp: 2026-02-09T17:03:18.643Z
Learning: In tests under tests/network, when locating the primary masquerade interface in tests such as test_masquerade_connectivity_after_migration.py, prefer using the hardcoded interface name 'default' (e.g., lookup_iface_status('default', ...)) rather than dynamically retrieving vm.vmi.interfaces[0].name. This is a common and accepted pattern across network tests. Only consider dynamic lookup if there is a documented exception; ensure tests remain stable and readable.
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3577
File: tests/virt/conftest.py:251-267
Timestamp: 2026-02-10T15:04:14.799Z
Learning: In Python tests, remove all bare time.sleep() calls. Replace with a waiting mechanism such as TimeoutSampler from the timeout_sampler package or a function decorated with retry (with appropriate timeout/conditions) to ensure determinism and avoid flakiness. This applies to all Python tests under the tests directory (not just this file) to maintain consistent waiting behavior across the suite.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3847
File: utilities/virt.py:2449-2453
Timestamp: 2026-02-18T06:35:39.536Z
Learning: In Python code, a function named clearly and self-descriptively can be deemed not to require a docstring. However, treat this as a context-specific guideline and not a universal rule. For public APIs or functions with side effects, prefer concise docstrings explaining behavior, inputs, outputs, and side effects. This guidance is based on the example in utilities/virt.py from RedHatQE/openshift-virtualization-tests where validate_libvirt_persistent_domain(vm, admin_client) was considered self-documenting.
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/unittests/test_os_utils.py:333-425
Timestamp: 2026-02-23T16:31:34.505Z
Learning: In integration/functional tests located under the tests/ directory, require assertion failure messages using the pattern: assert condition, "descriptive message". For unit tests under utilities/unittests/, rely on pytest's assertion introspection and descriptive test names; explicit failure messages are not required. This guidance helps maintain clear diagnostics for integration tests while keeping unit tests concise and leveraging pytest's built-in introspection.
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/pytest_utils.py:441-463
Timestamp: 2026-02-23T16:33:22.070Z
Learning: In Python code reviews, the guideline to always use named arguments for multi-argument calls does not apply to built-ins or methods that have positional-only parameters (those defined with a / in their signature). Do not flag or require named arguments for calls like dict.get(key, default=None, /), list.pop(), str.split(sep, maxsplit) and similar built-ins that cannot accept keyword arguments. Apply the named-argument rule only to functions/methods that explicitly accept keyword arguments.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3930
File: tests/network/vmi/conftest.py:0-0
Timestamp: 2026-02-24T13:49:39.277Z
Learning: In network test fixtures, allow the use of '# type: ignore' for assignments to spec.template.spec.domain.devices.interfaces immediately after calling base_vmspec(), i.e., bases this suppression pattern applies to files like tests/network/libs/vm_factory.py, tests/network/kubemacpool/explicit_range/conftest.py, tests/network/non_functional/ip_persistence/libippersistence.py, tests/network/vmi/conftest.py, etc. Do not flag or discourage type: ignore in these specific fixtures; apply this convention only within the network test suite and when the ignore is used for the described base_vmspec() context.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3873
File: tests/network/localnet/test_non_udn_localnet.py:7-9
Timestamp: 2026-02-25T10:52:09.679Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, networking infrastructure requirements (nmstate, localnet bridge mappings, NIC availability) are not automatically tier3; they are considered standard test environment capabilities. Only tests with truly platform-specific, time-consuming, or bare-metal requirements should be marked as tier3. Apply this guidance to all Python tests under tests/, including tests/network/localnet/test_non_udn_localnet.py, ensuring tier3 designation is reserved for genuine platform-specific or complex scenarios rather than general networking infra necessities.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3873
File: tests/network/localnet/test_non_udn_localnet.py:19-27
Timestamp: 2026-02-25T11:00:02.013Z
Learning: In the test codebase, do not import from conftest.py files. This avoids import ambiguity in pytest. Do not import constants or helpers defined in conftest.py into tests. If a value is needed in both conftest.py and test files, duplicate it in both places or place it in a separate utility module that is importable by tests.
| - 2 running VMs within the same EVPN CUDN. | ||
| """ | ||
|
|
||
| @pytest.mark.polarion("CNV-00000") |
Contributor
There was a problem hiding this comment.
MEDIUM: Placeholder Polarion IDs need real values.
All 7 test methods use CNV-00000 as a placeholder Polarion ID. These need to be replaced with actual Polarion test case IDs before the tests are enabled (when __test__ = False is removed).
WHY this matters: Polarion IDs link test results to requirements tracking. Placeholder IDs will cause test result reporting to fail or produce incorrect traceability.
Consider creating the Polarion test cases now and updating the IDs, or add a TODO comment tracking this dependency.
Also applies to: 43-43, 57-57, 72-72, 86-86, 101-101, 117-117
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/network/bgp/evpn/test_evpn_connectivity.py` at line 30, The Polarion ID
placeholder "CNV-00000" used in the pytest.mark.polarion decorators must be
replaced with the real Polarion test case IDs (or annotated with a clear TODO)
so reporting/tracking works when tests are enabled; update each occurrence of
pytest.mark.polarion("CNV-00000") in this file (all seven decorators) to the
actual Polarion IDs or add a TODO comment next to each decorator indicating to
insert the proper ID, and ensure the change is coordinated with the toggle that
controls test activation (__test__ = False) so tests only run once valid IDs are
set.
Contributor
Author
There was a problem hiding this comment.
will be updated before the merge once the PR is approved
Contributor
There was a problem hiding this comment.
@servolkov, understood! Since __test__ = False acts as a safety gate preventing the tests from running in the meantime, that's a reasonable approach. Just make sure the Polarion IDs are in place before flipping __test__ to True (or removing the attribute) to ensure test results are properly linked to the requirement tracking system.
🧠 Learnings used
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 1776
File: tests/network/bgp/conftest.py:35-54
Timestamp: 2025-08-28T12:30:40.692Z
Learning: The BGP test suite in tests/network/bgp/ relies on session-scoped validation in tests/network/conftest.py via the network_sanity fixture's _verify_bgp() function, which validates required environment variables (VLAN_TAG, EXTERNAL_FRR_STATIC_IPV4, BGP_CLUSTER_DOMAIN_GROUP) before any BGP tests run, making individual fixture-level validation redundant.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-09-29T19:05:24.987Z
Learning: For PR `#1904` test execution, the critical validation point is test_connectivity_over_migration_between_localnet_vms which should fail gracefully on cloud clusters but pass on bare-metal/PSI clusters, representing the core nmstate conditional logic functionality.
Learnt from: rlobillo
Repo: RedHatQE/openshift-virtualization-tests PR: 1984
File: tests/install_upgrade_operators/network_policy/test_network_policy_components.py:42-42
Timestamp: 2025-09-09T11:51:37.860Z
Learning: For SNO (Single Node OpenShift) deployments, connectivity tests that require multiple pods should be excluded at the pytest mark level rather than using runtime pytest.skip(), as SNO typically runs single replicas of components.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3598
File: tests/network/sriov/conftest.py:62-71
Timestamp: 2026-01-28T17:36:14.188Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, sanity checks validate that the cluster supports at least one IP family (IPv4 or IPv6) before SR-IOV tests run. Therefore, defensive guards checking for empty IP addresses in SR-IOV VM fixtures (like `sriov_vm` in tests/network/sriov/conftest.py) are not necessary—if neither `ipv4_supported_cluster` nor `ipv6_supported_cluster` is True, the cluster is considered broken and will fail sanity validation.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 1904
File: tests/network/conftest.py:348-362
Timestamp: 2025-10-27T15:30:06.412Z
Learning: In tests/network/conftest.py, the _verify_nmstate_running_pods function currently runs unconditionally in network_sanity, but rnetser plans to implement marker-based conditional checking (following the pattern of _verify_dpdk, _verify_sriov, etc.) in a future PR after adding nmstate markers to the relevant tests.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3072
File: tests/conftest.py:50-55
Timestamp: 2025-12-17T12:33:03.550Z
Learning: In the RedHatQE/openshift-virtualization-tests repository test environment, there is only one SR-IOV Node Network Policy (SNNP) deployed per cluster after deployment. The sriov_node_policy fixture implementation that retrieves the first policy from the namespace is intentional and correct for this single-policy environment. The team will adapt the test setup if the deployment strategy changes in the future.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 2119
File: tests/network/localnet/conftest.py:352-366
Timestamp: 2025-10-31T13:05:24.570Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, tests run sequentially by default on the same cluster (with module-scoped fixtures properly cleaned up between modules), and parallel test runs use different machines/clusters. This means cluster-scoped resources (like ClusterUserDefinedNetwork or NodeNetworkConfigurationPolicy) can safely use the same names across different test modules without risk of collision.
Learnt from: rlobillo
Repo: RedHatQE/openshift-virtualization-tests PR: 1984
File: tests/install_upgrade_operators/network_policy/test_network_policy_components.py:16-16
Timestamp: 2025-09-12T08:10:48.874Z
Learning: Network policy tests that create different types of pods (client pod, server pod, existing component pods) for connectivity testing can run on SNO, as they don't require multiple replicas of the same component to function properly.
Learnt from: rlobillo
Repo: RedHatQE/openshift-virtualization-tests PR: 1984
File: tests/install_upgrade_operators/network_policy/test_network_policy_components.py:182-187
Timestamp: 2025-09-12T14:14:28.329Z
Learning: Network policy tests in the RedHatQE/openshift-virtualization-tests repository don't require unique resource names as there are no parallel runs on the same cluster in their test environment.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 1776
File: tests/network/bgp/conftest.py:62-87
Timestamp: 2025-08-28T12:34:56.341Z
Learning: The BGP test infrastructure in tests/network/bgp/ currently uses static IP assignment via EXTERNAL_FRR_STATIC_IPV4, but the long-term goal is to transition to DHCP for IP allocation. Additional validation of EXTERNAL_FRR_STATIC_IPV4 in individual fixtures is considered redundant since validation already occurs at the session level.
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-22T15:56:00.157Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, do NOT use "REQUEST_CHANGES" review type if the PR author has already marked the PR as verified (e.g., with `/verified` command). Test execution plans are informational guides, not blocking requirements. Use COMMENT event for informational test plans, or only REQUEST_CHANGES if there are actual code issues that need to be addressed before merging.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3228
File: .coderabbit.yaml:30-41
Timestamp: 2026-01-05T10:33:55.037Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers minimal pre-merge checks in CodeRabbit configuration: only docstrings enforcement (80% threshold) is needed, not title or description checks.
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 1932
File: tests/virt/node/migration_and_maintenance/conftest.py:57-64
Timestamp: 2025-09-08T21:34:28.924Z
Learning: In OpenShift Virtualization tests, MigrationPolicy fixtures should use static names rather than unique suffixes to enable collision detection. If parallel test runs collide on cluster-scoped resource names like MigrationPolicy, it's better to know about the collision rather than hide it with unique naming, as confirmed by maintainer dshchedr.
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:5-5
Timestamp: 2026-01-07T11:52:04.988Z
Learning: In utilities/unittests/test_oadp.py (and similar test files in openshift-virtualization-tests), the `# flake8: noqa: E402` directive is required by pre-commit hooks, even when Ruff reports it as unused. This is needed because mock setup must precede imports to prevent circular dependencies, triggering E402 warnings that must be suppressed.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 2469
File: utilities/sanity.py:139-142
Timestamp: 2025-11-08T07:36:57.616Z
Learning: In the openshift-virtualization-tests repository, user rnetser prefers to keep refactoring PRs (like PR `#2469`) strictly focused on moving/organizing code into more granular modules without adding new functionality, error handling, or behavioral changes. Such improvements should be handled in separate PRs.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 0
File: :0-0
Timestamp: 2025-12-16T14:00:59.076Z
Learning: In the openshift-virtualization-tests repository, when responding to test execution plan requests from openshift-virtualization-qe-bot-3, CodeRabbit must post ONLY an inline review comment on the Files Changed tab and then stop immediately without generating any follow-up comments in the PR discussion thread. No acknowledgment messages, no confirmation of posting, no explanation - silence after posting the inline review equals success. Additional comments create empty/meaningless reviews that clutter the PR.
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:48-57
Timestamp: 2026-01-19T07:54:03.630Z
Learning: The pytest marker requirement ("all new tests MUST have markers") applies only to integration/functional tests in the `tests/` directory (e.g., `tests/virt/`, `tests/storage/`, `tests/network/`), not to unit tests in `utilities/unittests/`. Unit test files in `utilities/unittests/` (such as test_oadp.py, test_sanity.py, test_operator.py, etc.) do not use pytest markers and this is the established repository convention.
</learning>
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3062
File: conftest.py:333-333
Timestamp: 2025-12-16T20:11:03.645Z
Learning: In the openshift-virtualization-tests repository, when determining smoke test impact for changes affecting py_config["os_login_param"], follow this verification methodology: (1) Find all smoke tests: `rg "pytest.mark.smoke" --type=py -B2 | grep "def test_"`, (2) For each smoke test file, search for VM creation patterns: `rg "VirtualMachineForTests|running_vm|VirtualMachineForTestsFromTemplate|wait_for_ssh|check_ssh_connectivity"`, (3) Trace the dependency chain: smoke test → VirtualMachineForTests/running_vm() → wait_for_ssh_connectivity() (default enabled) → vm.login_params property → py_config["os_login_param"][vm.os_flavor], (4) Check utilities/virt.py for login_params usage: `rg "os_login_param|login_params" utilities/virt.py -C3`. Any smoke test creating VMs with default SSH connectivity checks (running_vm with check_ssh_connectivity=True) depends on os_login_param, even if the test doesn't directly reference it.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/test_ip_persistence.py:23-34
Timestamp: 2026-01-12T11:09:08.889Z
Learning: In tests/network/provider_migration/test_ip_persistence.py, user servolkov prefers compound assertions when they provide comprehensive error messages showing all relevant diagnostic information (e.g., both MAC and IP expected/actual values), rather than splitting them into multiple assertions that would repeat similar patterns, prioritizing the DRY principle over Ruff PT018 recommendations.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3598
File: tests/network/sriov/libsriov.py:10-72
Timestamp: 2026-02-10T16:55:49.690Z
Learning: In tests/network/sriov/libsriov.py, user azhivovk considers the SR-IOV test helper functions (vm_sriov_mac, sriov_vm, sriov_cloud_init_data) sufficiently clear from their names and flow, and does not require docstrings. The module is a test-specific library rather than a large common library, making docstrings redundant.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3062
File: conftest.py:333-333
Timestamp: 2025-12-16T15:09:49.597Z
Learning: In the openshift-virtualization-tests repository, when conftest.py or utilities/bitwarden.py changes affect py_config["os_login_param"], smoke test impact must be determined by: (1) finding all smoke tests using `rg "pytest.mark.smoke"`, (2) checking each for VM creation patterns (VirtualMachineForTests, running_vm, VirtualMachineForTestsFromTemplate), (3) tracing whether running_vm is called with default check_ssh_connectivity=True, which accesses vm.login_params property that reads py_config["os_login_param"][vm.os_flavor]. The dependency chain is: smoke test → VM creation → running_vm → wait_for_ssh_connectivity → vm.login_params → os_login_param. Any smoke test creating VMs with SSH connectivity (the default) depends on os_login_param.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3873
File: tests/network/localnet/test_non_udn_localnet.py:7-9
Timestamp: 2026-02-25T10:52:23.378Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, networking infrastructure requirements such as nmstate, localnet bridge mappings, and NIC availability are NOT considered special/platform-specific infrastructure that would warrant a tier3 marker. These are standard test environment capabilities in this repository's test environment, and tests requiring them do not automatically qualify as tier3 unless they involve other truly platform-specific/complex/time-consuming characteristics beyond standard network test infrastructure.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 1776
File: libs/net/node_network.py:25-31
Timestamp: 2025-08-20T23:43:28.117Z
Learning: In the RedHatQE/openshift-virtualization-tests project, servolkov's team always uses bare metal (BM) clusters with IPv4 setup in their testing environment, making defensive checks for IPv4 data presence potentially redundant in their networking code.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3024
File: tests/network/connectivity/utils.py:17-17
Timestamp: 2025-12-15T12:33:06.686Z
Learning: In the test suite, ensure the ipv6_network_data fixture returns a factory function (Callable) and that all call sites invoke it to obtain the actual data dict, i.e., use ipv6_network_data() at call sites. This enables future extensibility for configuring secondary interfaces' IP addresses without changing call sites. Apply this pattern to all Python test files under tests.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3196
File: tests/network/upgrade/test_upgrade_network.py:4-4
Timestamp: 2025-12-22T16:27:40.244Z
Learning: For PRs that remove tests, rely on pytest --collect-only to verify the test discovery results (which tests are selected/deselected) and ensure the removal is clean and the test module remains functional. Full test execution is not required for test deletion PRs. This guideline applies to test files anywhere under the tests/ directory (e.g., tests/network/upgrade/test_upgrade_network.py) and should be used for similar test-deletion scenarios across the repository.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3358
File: tests/network/sriov/test_sriov.py:21-21
Timestamp: 2026-01-07T09:52:12.342Z
Learning: When a PR only removes or modifies pytest markers in tests (e.g., removing pytest.mark.post_upgrade) and the test logic remains unchanged, prefer verifying with pytest --collect-only instead of running the full test suite. This validates that marker usage and test selection behavior are preserved. If the test logic changes, or markers affect behavior beyond collection, run the full test suite to confirm.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:50-52
Timestamp: 2026-01-12T11:24:13.825Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when catching exceptions in Python, use LOGGER.error before re-raising and do not replace it with LOGGER.exception in except blocks. This follows the established pattern across the codebase.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3366
File: tests/storage/cdi_clone/test_clone.py:5-9
Timestamp: 2026-01-12T14:25:05.723Z
Learning: In Python tests and utility code across the repository, bitmath.parse_string_unsafe correctly parses Kubernetes quantities (e.g., '4Gi', '512Mi', PVC storage requests) without supplying system=bitmath.NIST. There are 30+ usages indicating this is the standard behavior. Reviewers should verify that code that builds or compares quantity strings does not pass the NIST parameter, and if a new test relies on quantity parsing, assume no NIST parameter is required unless explicitly documented.
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 3404
File: tests/virt/upgrade/conftest.py:291-301
Timestamp: 2026-01-14T04:08:23.032Z
Learning: In the openshift-virtualization-tests repository, when using VirtualMachine objects from ocp-resources in tests, if vm.ready is True, vm.vmi is guaranteed to exist. Therefore, you can access vm.vmi.instance.status or vm.vmi attributes without additional defensive checks (e.g., if vm.vmi: ...). Do not rely on vm.vmi being present when vm.ready may be False; guard those code paths accordingly. This guideline applies to tests under tests/ (notably in virt/upgrade/conftest.py and related test modules) and should be followed for any code paths that assume vm.vmi exists only when vm.ready is True.
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 3376
File: tests/network/general/test_ip_family_services.py:96-96
Timestamp: 2026-01-18T09:44:17.044Z
Learning: In the openshift-virtualization-tests repository, function-scoped fixtures must use pytest.fixture() with empty parentheses (not pytest.fixture without parentheses). This repo follows this convention despite Ruff PT001. Apply this consistently to all Python test files under tests/ (not just this one) to maintain repository-wide consistency.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3273
File: tests/network/connectivity/test_ovs_linux_bridge.py:5-9
Timestamp: 2026-01-18T13:18:48.808Z
Learning: In tests/network/connectivity/test_ovs_linux_bridge.py and similar test files, prefer importing ipaddress as a module and using qualified calls like ipaddress.ip_interface(...) rather than from ipaddress import ip_interface. This preserves module context for readability, especially when chaining properties (e.g., ipaddress.ip_interface(...).ip). This is an intentional exception to the general rule favoring specific imports, and should apply to test files under the tests directory where module context aids understanding.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3495
File: tests/network/third_part_ip_request/test_third_party_ip_request.py:4-12
Timestamp: 2026-01-18T14:51:50.846Z
Learning: In the openshift-virtualization-tests repository, tests consistently import pytest as a module (import pytest) and avoid from pytest import ...; this is the established pattern across 398+ test files. Do not flag or refactor imports to use specific pytest names in tests under tests/**. If a file already follows this pattern, leave it as is; this guideline applies broadly to Python test files under the tests directory.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3495
File: tests/network/third_part_ip_request/test_third_party_ip_request.py:4-12
Timestamp: 2026-01-18T14:51:50.846Z
Learning: In tests/network/ modules in the openshift-virtualization-tests repository, import network library modules from tests/network/libs using aliases with the as lib* convention (e.g., from tests.network.libs import cluster_user_defined_network as libcudn, from libs.net import netattachdef as libnad). This pattern improves readability and is consistently used across multiple network test files.
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:1-8
Timestamp: 2026-01-20T01:03:13.139Z
Learning: In the openshift-virtualization-tests repository, Python imports should consistently use module-level imports for the logging module (i.e., import logging) rather than from logging import ... The established pattern spans 270+ files and should not be flagged for refactoring. Apply this guideline to Python files across the repo (e.g., tests/network/provider_migration/libprovider.py).
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 3559
File: utilities/infra.py:251-254
Timestamp: 2026-01-21T21:26:41.805Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when reviewing Python code, recognize that with Python 3.14 the syntax 'except ValueError, TypeError:' is valid if there is no 'as' clause, and should not be flagged as Python 2 syntax. If you use an 'as' binding (e.g., 'except (ValueError, TypeError) as e:'), parentheses are required. Ensure this pattern is version-consistent and not flagged as Python 2 syntax when 'as' is absent.
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 3571
File: tests/storage/storage_migration/utils.py:158-167
Timestamp: 2026-01-25T13:18:21.675Z
Learning: In reviews of the openshift-virtualization-tests repo (and similar Python code), avoid suggesting minor stylistic changes that require extra verification (e.g., removing dict.keys() checks for membership) unless the change has clear correctness or maintainability impact. Focus on fixes with observable behavior, security, performance, or maintainability benefits; defer low-impact style tweaks that are costly to verify.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3619
File: tests/network/user_defined_network/test_user_defined_network.py:97-97
Timestamp: 2026-01-27T17:18:49.973Z
Learning: In tests that exercise lookup_iface_status_ip from libs.net.vmspec, rely on the function's built-in descriptive error messages for failures. Do not add extra assertion messages for IP presence checks using this function; instead, assert on the function behavior or catch its exceptions as appropriate. This reduces duplication and clarifies failures.
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:48-54
Timestamp: 2026-01-28T16:08:33.862Z
Learning: Adhere to the repository convention for pytest parameterization: use positional arguments (tuples) in pytest.mark.parametrize with the first element being the argnames tuple, e.g. pytest.mark.parametrize(("kind", "name"), [("Backup", "test-backup")]), rather than using keyword arguments via argnames= and argvalues=. This applies to all test files in the repository; ensure any existing or new tests follow the positional pattern for parameterized tests to maintain consistency and readability.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3649
File: tests/network/user_defined_network/ip_specification/libipspec.py:1-4
Timestamp: 2026-01-29T05:30:13.982Z
Learning: In the openshift-virtualization-tests repository, Python imports should use module import style for the standard library 'json' (import json) rather than 'from json import ...'. This improves readability by making calls like json.loads and json.dumps explicit, and aligns with patterns used for modules like logging, ipaddress, cloudinit, and pytest. Apply this to all Python test files under tests/ (and similar test directories).
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3690
File: tests/after_cluster_deploy_sanity/test_after_cluster_deploy_sanity.py:65-65
Timestamp: 2026-02-02T17:41:12.759Z
Learning: In test files, keep test_* functions with simple one-line docstrings. For helper functions, utilities, and library code with non-obvious return values or side effects, use Google-style docstrings with Args, Returns, and Side effects sections. Do not require Google-style docstrings for pytest test functions themselves.
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3708
File: tests/network/user_defined_network/ip_specification/test_ip_specification.py:141-145
Timestamp: 2026-02-02T19:37:50.989Z
Learning: In tests under tests/network that use traffic_generator helpers (e.g., is_tcp_connection), you can rely on pytest’s assertion introspection, descriptive test names, and internal logging from the helpers to diagnose failures. Do not add custom assertion failure messages for these helpers; instead ensure logs and clear test names convey the failure context.
Learnt from: RoniKishner
Repo: RedHatQE/openshift-virtualization-tests PR: 3697
File: tests/infrastructure/instance_types/test_common_vm_instancetype.py:53-98
Timestamp: 2026-02-03T07:34:34.184Z
Learning: In test files (Python, pytest), prefer using tier3 markers for categorization since tier2, tier1, and tier4 are not used in this repository. Do not rely on non-official markers; formalize and document the allowed markers in pytest.ini (or equivalent) to ensure consistent usage. When reviewing new tests, ensure markers align with this convention (only tier3, unless a project-wide decision defines additional tiers) and remove any unnecessary or undocumented markers.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3634
File: tests/network/migration/test_masquerade_connectivity_after_migration.py:116-116
Timestamp: 2026-02-09T17:03:18.643Z
Learning: In tests under tests/network, when locating the primary masquerade interface in tests such as test_masquerade_connectivity_after_migration.py, prefer using the hardcoded interface name 'default' (e.g., lookup_iface_status('default', ...)) rather than dynamically retrieving vm.vmi.interfaces[0].name. This is a common and accepted pattern across network tests. Only consider dynamic lookup if there is a documented exception; ensure tests remain stable and readable.
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3577
File: tests/virt/conftest.py:251-267
Timestamp: 2026-02-10T15:04:14.799Z
Learning: In Python tests, remove all bare time.sleep() calls. Replace with a waiting mechanism such as TimeoutSampler from the timeout_sampler package or a function decorated with retry (with appropriate timeout/conditions) to ensure determinism and avoid flakiness. This applies to all Python tests under the tests directory (not just this file) to maintain consistent waiting behavior across the suite.
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3847
File: utilities/virt.py:2449-2453
Timestamp: 2026-02-18T06:35:39.536Z
Learning: In Python code, a function named clearly and self-descriptively can be deemed not to require a docstring. However, treat this as a context-specific guideline and not a universal rule. For public APIs or functions with side effects, prefer concise docstrings explaining behavior, inputs, outputs, and side effects. This guidance is based on the example in utilities/virt.py from RedHatQE/openshift-virtualization-tests where validate_libvirt_persistent_domain(vm, admin_client) was considered self-documenting.
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/unittests/test_os_utils.py:333-425
Timestamp: 2026-02-23T16:31:34.505Z
Learning: In integration/functional tests located under the tests/ directory, require assertion failure messages using the pattern: assert condition, "descriptive message". For unit tests under utilities/unittests/, rely on pytest's assertion introspection and descriptive test names; explicit failure messages are not required. This guidance helps maintain clear diagnostics for integration tests while keeping unit tests concise and leveraging pytest's built-in introspection.
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/pytest_utils.py:441-463
Timestamp: 2026-02-23T16:33:22.070Z
Learning: In Python code reviews, the guideline to always use named arguments for multi-argument calls does not apply to built-ins or methods that have positional-only parameters (those defined with a / in their signature). Do not flag or require named arguments for calls like dict.get(key, default=None, /), list.pop(), str.split(sep, maxsplit) and similar built-ins that cannot accept keyword arguments. Apply the named-argument rule only to functions/methods that explicitly accept keyword arguments.
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 3930
File: tests/network/vmi/conftest.py:0-0
Timestamp: 2026-02-24T13:49:39.277Z
Learning: In network test fixtures, allow the use of '# type: ignore' for assignments to spec.template.spec.domain.devices.interfaces immediately after calling base_vmspec(), i.e., bases this suppression pattern applies to files like tests/network/libs/vm_factory.py, tests/network/kubemacpool/explicit_range/conftest.py, tests/network/non_functional/ip_persistence/libippersistence.py, tests/network/vmi/conftest.py, etc. Do not flag or discourage type: ignore in these specific fixtures; apply this convention only within the network test suite and when the ignore is used for the described base_vmspec() context.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3873
File: tests/network/localnet/test_non_udn_localnet.py:7-9
Timestamp: 2026-02-25T10:52:09.679Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, networking infrastructure requirements (nmstate, localnet bridge mappings, NIC availability) are not automatically tier3; they are considered standard test environment capabilities. Only tests with truly platform-specific, time-consuming, or bare-metal requirements should be marked as tier3. Apply this guidance to all Python tests under tests/, including tests/network/localnet/test_non_udn_localnet.py, ensuring tier3 designation is reserved for genuine platform-specific or complex scenarios rather than general networking infra necessities.
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 3873
File: tests/network/localnet/test_non_udn_localnet.py:19-27
Timestamp: 2026-02-25T11:00:02.013Z
Learning: In the test codebase, do not import from conftest.py files. This avoids import ambiguity in pytest. Do not import constants or helpers defined in conftest.py into tests. If a value is needed in both conftest.py and test files, duplicate it in both places or place it in a separate utility module that is importable by tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
EVPN Connectivity Tests.
Tests are aimed to cover the EVPN integration for OpenShift Virtualization VMs.
STP: RedHatQE/openshift-virtualization-tests-design-docs#35
jira-ticket: https://issues.redhat.com/browse/CNV-80609
Summary by CodeRabbit
Release Notes