-
Notifications
You must be signed in to change notification settings - Fork 30
[IUO] Add IUO child STP for dual-stream #108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
13d89e3
58fcf00
e55a57e
417c817
f837d2c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,233 @@ | ||||||||
| # Openshift-virtualization-tests Test plan | ||||||||
|
|
||||||||
| ## **[Dual-Stream RHCOS 9.8 + RHCOS 10.2 — IUO Scope] - Quality Engineering Plan** | ||||||||
|
|
||||||||
| ### **Metadata & Tracking** | ||||||||
|
|
||||||||
| - **Feature Tracking:** [VIRTSTRAT-83](https://redhat.atlassian.net/browse/VIRTSTRAT-83) | ||||||||
| - **Epic Tracking:** [CNV-85268](https://redhat.atlassian.net/browse/CNV-85268) | ||||||||
| - **IUO Story:** [CNV-85504](https://redhat.atlassian.net/browse/CNV-85504) | ||||||||
| - **Parent STP:** [stp.md](stp.md) | ||||||||
|
OhadRevah marked this conversation as resolved.
|
||||||||
| - **QE Owner(s):** Ohad Revah (@OhadRevah) | ||||||||
| - **SIG:** sig-iuo (Install, Upgrade, Operators) | ||||||||
|
|
||||||||
| ### **Feature Overview** | ||||||||
|
|
||||||||
| This STP covers the IUO-specific aspects of dual-stream RHCOS support: validating that | ||||||||
| OpenShift Virtualization deploys and functions correctly on RHCOS 10.2, diagnostic data collection works on RHCOS 10.2 nodes, | ||||||||
| node placement policies are honored in mixed-version clusters, observability metrics work | ||||||||
| as expected, and migration metrics are accurately reported during cross-version live migration. | ||||||||
|
|
||||||||
| This STP covers testing for OCP 5.0. Automation is required for migration metrics validation on dual-stream clusters. | ||||||||
|
|
||||||||
| --- | ||||||||
|
|
||||||||
| ### **I. Motivation and Requirements Review (QE Review Guidelines)** | ||||||||
|
|
||||||||
| #### **1. Requirement & User Story Review Checklist** | ||||||||
|
|
||||||||
| - [x] **Review Requirements** | ||||||||
| - *SIG-specific requirements:* | ||||||||
| - OpenShift Virtualization components deploy and report ready on RHCOS 10.2 worker nodes | ||||||||
| - Must-gather collects complete and valid data from RHCOS 10.2 nodes, with no gaps compared to RHCOS 9.8 | ||||||||
|
coderabbitai[bot] marked this conversation as resolved.
OhadRevah marked this conversation as resolved.
|
||||||||
| - OpenShift Virtualization migration metrics (duration, data processed, bandwidth) are reported accurately during cross-version live migration | ||||||||
| - Node placement policies are respected when scheduling and migrating VMs on dual-stream clusters | ||||||||
|
|
||||||||
| - [x] **Acceptance Criteria** | ||||||||
|
OhadRevah marked this conversation as resolved.
|
||||||||
| - OpenShift Virtualization components deploy and report ready on RHCOS 10.2 worker nodes | ||||||||
| - Must-gather collects complete and valid data from RHCOS 10.2 and dual-stream cluster nodes | ||||||||
| - Migration metrics (duration, data processed, bandwidth) are reported accurately during cross-version live migration on dual-stream clusters | ||||||||
| - Node placement policies are respected when scheduling and migrating VMs on dual-stream clusters | ||||||||
|
|
||||||||
| - [x] **Testability** | ||||||||
| - *Note any SIG-specific requirements that are unclear or untestable:* All requirements are testable | ||||||||
|
OhadRevah marked this conversation as resolved.
|
||||||||
| through existing IUO test suites and new migration metrics automation on RHCOS 10.2 and dual-stream clusters. | ||||||||
|
|
||||||||
| - [x] **Non-Functional Requirements (NFRs)** | ||||||||
| - *SIG-specific NFRs:* | ||||||||
| - Monitoring: CNV metrics (including migration metrics) must report correctly on RHCOS 10.2 nodes | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: "CNV" is used in several user-facing sections (lines 48, 63, 139, 172). Consider using "OpenShift Virtualization" for consistency with the rest of the STP. Assisted-by: PI (claude-opus-4-6-1m) |
||||||||
| - *NFRs not covered and why:* | ||||||||
| - Performance: N/A — no new IUO-specific performance requirements; covered by parent STP | ||||||||
| - Security: N/A — no new auth or RBAC changes; FIPS requirement covered by parent STP | ||||||||
| - Scalability: N/A — no new scale requirements for IUO components | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Scalability NFR says "no new scale requirements" but P1 testing depends on cross-version live migration which has existing cluster-level parallelism limits (acknowledged in the parent STP). Please add: "existing cluster-level live migration parallelism limits apply (see parent STP)." Assisted-by: PI (claude-opus-4-6-1m) |
||||||||
| - UI: N/A — dual-stream RHCOS support introduces no new user journeys or UI elements for IUO; existing console functionality is unchanged | ||||||||
| - Documentation: N/A — no IUO-specific documentation changes; release notes covered by parent STP | ||||||||
|
|
||||||||
|
coderabbitai[bot] marked this conversation as resolved.
|
||||||||
| #### **2. Known Limitations** | ||||||||
|
|
||||||||
| None — reviewed and confirmed that no IUO-specific feature limitations apply for this release. | ||||||||
|
OhadRevah marked this conversation as resolved.
OhadRevah marked this conversation as resolved.
|
||||||||
|
|
||||||||
| #### **3. Technology and Design Review** | ||||||||
|
|
||||||||
| - [x] **Developer Handoff/QE Kickoff** | ||||||||
| - *Key takeaways and concerns:* CNV operators run the same el9.8 userland on both RHCOS 9.8 and | ||||||||
| RHCOS 10.2 kernels. No operator code changes are expected, but kernel differences could surface | ||||||||
| unexpected behavior in must-gather log collection or metrics. | ||||||||
|
|
||||||||
| - [x] **Technology Challenges** | ||||||||
| - *List identified challenges:* | ||||||||
| - Must-gather may encounter differences in log paths or system service names between RHCOS 9.8 | ||||||||
| and RHCOS 10.2 nodes, potentially causing incomplete data collection. | ||||||||
| - *Impact on testing approach:* Must-gather output must be compared between RHCOS 9.8 and | ||||||||
| RHCOS 10.2 to identify any gaps. | ||||||||
|
|
||||||||
| - [x] **API Extensions** | ||||||||
| - *List new or modified user-facing APIs:* N/A — see parent STP | ||||||||
| - *Testing impact:* N/A | ||||||||
|
|
||||||||
| - [x] **Test Environment Needs** | ||||||||
| - *See environment requirements in Section II.3 and testing tools in Section II.3.1* | ||||||||
|
|
||||||||
| - [x] **Topology Considerations** | ||||||||
| - *Describe topology requirements:* Same as parent STP. Dual-stream cluster required for | ||||||||
| migration and node placement scenarios; RHCOS 10.2-only cluster required for IUO regression. | ||||||||
| - *Impact on test design:* Node placement tests require labeling nodes by RHCOS version | ||||||||
| and using node affinity to control VM scheduling and migration targets. | ||||||||
|
|
||||||||
| ### **II. Software Test Plan (STP)** | ||||||||
|
|
||||||||
| #### **1. Scope of Testing** | ||||||||
|
|
||||||||
| **Testing Goals** | ||||||||
|
|
||||||||
| - **[P0]** OpenShift Virtualization components deploy, report ready, and function correctly on RHCOS 10.2-only and dual-stream clusters. | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [CRITICAL] P0 Testing Goal has no matching Section III scenario The P0 goal ("OpenShift Virtualization components deploy, report ready, and function correctly on RHCOS 10.2-only and dual-stream clusters") has no corresponding test scenario in Section III. Section III only contains P1 migration-metrics scenarios. Per AGENTS.md: "Every Testing Goal from Section II.1 has a matching scenario." Please either:
Assisted-by: PI (claude-opus-4-6-1m) |
||||||||
| - **[P1]** Verify migration metrics (duration, data processed, bandwidth) are reported accurately during cross-version live migration on dual-stream clusters (RHCOS 9.8 ↔ RHCOS 10.2). | ||||||||
|
|
||||||||
| **Out of Scope (Testing Scope Exclusions)** | ||||||||
|
|
||||||||
| - **Upgrade testing** | ||||||||
| - *Rationale:* Relevant only from 5.1.0. | ||||||||
| - *PM/Lead Agreement:* Martin Tessun / 2026-05-13 | ||||||||
|
|
||||||||
| **Test Limitations** | ||||||||
|
|
||||||||
| - None — dual-stream cluster provisioning tooling is available and operational. | ||||||||
|
|
||||||||
| #### **2. Test Strategy** | ||||||||
|
|
||||||||
| **Functional** | ||||||||
|
|
||||||||
| - [x] **Functional Testing** — Validates IUO-specific features on RHCOS 10.2 and dual-stream clusters | ||||||||
| - *Details:* Run existing IUO Tier 1 and Tier 2 suites on RHCOS 10.2-only and dual-stream clusters. | ||||||||
|
|
||||||||
| - [x] **Automation Testing** — Migration metrics validation on dual-stream clusters | ||||||||
| - *Details:* Existing IUO Tier 1/2 suites run as-is on RHCOS 10.2 cluster. New automation | ||||||||
| for migration metrics validation during cross-version live migration (RHCOS 9.8 ↔ RHCOS 10.2). | ||||||||
|
|
||||||||
| - [x] **Regression Testing** — IUO regression on RHCOS 10.2 and dual-stream clusters | ||||||||
| - *Details:* Existing IUO Tier 1 and Tier 2 regression suites run on RHCOS 10.2-only and dual-stream clusters. | ||||||||
| Failures triaged and bugs filed with RHCOS-version attribution. | ||||||||
|
|
||||||||
| - [ ] **Self-Validation Testing** | ||||||||
| - *Details:* N/A — migration metrics tests are Tier 2 scenarios requiring dual-stream clusters; | ||||||||
| not suitable for the self-validation health check package. | ||||||||
|
|
||||||||
| **Non-Functional** | ||||||||
|
|
||||||||
| - [ ] **Performance Testing** | ||||||||
| - *Details:* Covered by parent STP. | ||||||||
|
|
||||||||
| - [ ] **Scale Testing** | ||||||||
| - *Details:* Covered by parent STP. | ||||||||
|
|
||||||||
| - [ ] **Security Testing** | ||||||||
| - *Details:* Covered by parent STP. FIPS requirement applies to all testing. | ||||||||
|
|
||||||||
| - [ ] **Usability Testing** | ||||||||
| - *Details:* N/A — dual-stream RHCOS support introduces no new user journeys or UI elements for IUO; existing console functionality is unchanged. | ||||||||
|
|
||||||||
| - [x] **Monitoring** — Verify CNV metrics on RHCOS 10.2 | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Monitoring details say OpenShift Virtualization metrics on RHCOS 10.2 nodes must be verified, but Testing Goals and Section III only cover migration metrics. Either narrow the Monitoring claim to match the actual goal/scenario scope, or add a goal + scenario for baseline metrics verification on RHCOS 10.2. Assisted-by: PI (claude-opus-4-6-1m) |
||||||||
| - *Details:* Verify CNV metrics (including migration metrics: duration, data processed, | ||||||||
| bandwidth) are reported correctly on RHCOS 10.2 nodes and during cross-version live migration. | ||||||||
|
|
||||||||
| **Integration & Compatibility** | ||||||||
|
|
||||||||
| - [ ] **Compatibility Testing** | ||||||||
| - *Details:* Not applicable for this STP. | ||||||||
|
|
||||||||
| - [ ] **Upgrade Testing** | ||||||||
|
OhadRevah marked this conversation as resolved.
|
||||||||
| - *Details:* Relevant only from 5.1.0. | ||||||||
|
|
||||||||
| - [x] **Dependencies** — Dual-stream cluster provisioning | ||||||||
| - *Details:* Dual-stream cluster provisioning tooling is available via QE DevOps. Dedicated CI lanes | ||||||||
| are operational for both RHCOS 10.2-only and dual-stream clusters. | ||||||||
|
|
||||||||
| - [ ] **Cross Integrations** | ||||||||
| - *Details:* Covered by parent STP. | ||||||||
|
|
||||||||
| **Infrastructure** | ||||||||
|
|
||||||||
| - [ ] **Cloud Testing** | ||||||||
| - *Details:* Covered by parent STP. | ||||||||
|
coderabbitai[bot] marked this conversation as resolved.
|
||||||||
|
|
||||||||
| #### **3. Test Environment** | ||||||||
|
|
||||||||
| Covered by the parent STP. IUO-specific requirements: | ||||||||
|
|
||||||||
| - **Cluster Topology:** | ||||||||
| - RHCOS 10.2-only cluster: for IUO Tier 1/2 regression | ||||||||
| - Dual-stream cluster (RHCOS 9.8 + RHCOS 10.2 workers): for migration metrics, | ||||||||
| node placement, and must-gather dual-node scenarios | ||||||||
|
|
||||||||
| - **OCP & OpenShift Virtualization Version(s):** OCP 5.0 with CNV 5.0 | ||||||||
|
|
||||||||
| - **Storage:** ocs-storagecluster-ceph-rbd-virtualization | ||||||||
|
|
||||||||
| - **Platform:** Bare metal | ||||||||
|
|
||||||||
| - **Special Configurations:** None — RHCOS 9.8 worker nodes are identified by the `worker-rhcos9` | ||||||||
| node role label, applied automatically during cluster provisioning. | ||||||||
|
|
||||||||
| #### **3.1. Testing Tools & Frameworks** | ||||||||
|
|
||||||||
| - **Test Framework:** Standard. Tests require logic to identify nodes by RHCOS version | ||||||||
| for node placement and migration validation. | ||||||||
|
|
||||||||
| - **CI/CD:** Dedicated CI lanes for RHCOS 10.2 and dual-stream clusters: | ||||||||
| - RHCOS 10.2: IUO testing and observability lanes | ||||||||
| - Dual-stream: IUO testing and observability lanes | ||||||||
|
|
||||||||
| - **Other Tools:** N/A | ||||||||
|
|
||||||||
| #### **4. Entry Criteria** | ||||||||
|
|
||||||||
| Covered by the parent STP. IUO-specific entry criteria: | ||||||||
|
|
||||||||
| - [x] Requirements and design documents are **approved and merged** | ||||||||
| - [x] RHCOS 10.2-only cluster available with IUO CI lanes provisioned | ||||||||
| - [x] Dual-stream cluster available via QE DevOps tooling (required for migration/node-placement scenarios) | ||||||||
|
OhadRevah marked this conversation as resolved.
|
||||||||
|
|
||||||||
| #### **5. Risks** | ||||||||
|
|
||||||||
| No IUO-specific risks identified. Feature-wide risks are covered by the parent STP. | ||||||||
|
|
||||||||
|
coderabbitai[bot] marked this conversation as resolved.
|
||||||||
| --- | ||||||||
|
|
||||||||
| ### **III. Test Scenarios & Traceability** | ||||||||
|
|
||||||||
| IUO coverage for dual-stream RHCOS is primarily provided through regression testing | ||||||||
| (existing Tier 1/2 suites on RHCOS 10.2 clusters). The following new test scenarios | ||||||||
| are required for migration metrics validation on dual-stream clusters: | ||||||||
|
|
||||||||
| - **[CNV-85504]** — As a VM operator, I want migration metrics to be reported accurately when migrating from an RHCOS 9.8 node to an RHCOS 10.2 node. | ||||||||
| - *Test Scenario:* [Tier 2] Live migrate a VM from an RHCOS 9.8 node to an RHCOS 10.2 node and verify that migration metrics (duration, data processed, bandwidth) are reported correctly. | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Migration metrics scenarios say "reported correctly" — could you define measurable pass criteria? For example: duration > 0 and consistent with observed migration time; data processed > 0; bandwidth non-zero during active transfer. Same applies to line 217. Assisted-by: PI (claude-opus-4-6-1m) |
||||||||
| - *Priority:* P1 | ||||||||
|
|
||||||||
| - **[CNV-85504]** — As a VM operator, I want migration metrics to be reported accurately when migrating from an RHCOS 10.2 node to an RHCOS 9.8 node. | ||||||||
| - *Test Scenario:* [Tier 2] Live migrate a VM from an RHCOS 10.2 node to an RHCOS 9.8 node and verify that migration metrics (duration, data processed, bandwidth) are reported correctly. | ||||||||
| - *Priority:* P1 | ||||||||
|
|
||||||||
| --- | ||||||||
|
|
||||||||
| ### **IV. Sign-off and Approval** | ||||||||
|
|
||||||||
| This Software Test Plan requires approval from the following stakeholders: | ||||||||
|
|
||||||||
| * **Reviewers:** | ||||||||
| - QE Architect: Ruth Netser (@rnetser) | ||||||||
| - sig-iuo representatives: @orenc1 @hmeir @rlobillo | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reviewer list uses bare GitHub handles without full names. Each reviewer should include role, name, and handle. For example:
Suggested change
(Please verify the correct full names.) Assisted-by: PI (claude-opus-4-6-1m) |
||||||||
| - sig-virt representative: Akriti Gupta (parent STP owner) | ||||||||
| * **Approvers:** | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [CRITICAL] Approvers list is missing a Dev Lead The parent STP lists "Principal Developer: Luboslav Pivarc @xpivarc" and the network child lists "Principal Developer: Edward Haas @EdDev" — but the IUO child only has QE Architect, sig-iuo Lead, and PM. Please add the IUO or feature Dev Lead as an approver with name and GitHub handle. Assisted-by: PI (claude-opus-4-6-1m)
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is still unresolved from my previous review — please add a Dev Lead as an approver with name and GitHub handle (e.g., Luboslav Pivarc @xpivarc or the IUO-specific dev lead). Assisted-by: PI (claude-opus-4-6-1m) |
||||||||
| - QE Architect: Ruth Netser (@rnetser) | ||||||||
| - sig-iuo Lead: @hmeir | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Approver entry lacks a display name. Approvers require role, name, and GitHub handle.
Suggested change
(Please verify the correct full name.) Assisted-by: PI (claude-opus-4-6-1m) |
||||||||
| - Product Manager: Martin Tessun (@mtessun) | ||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No linked GitHub issue for this PR. Consider linking a tracking issue or noting Jira-only tracking (e.g., CNV-85504) in the PR description.
Assisted-by: PI (claude-opus-4-6-1m)