Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
233 changes: 233 additions & 0 deletions stps/sig-virt/dual-stream-cluster-rhcos9-rhcos10/iuo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,233 @@
# Openshift-virtualization-tests Test plan

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No linked GitHub issue for this PR. Consider linking a tracking issue or noting Jira-only tracking (e.g., CNV-85504) in the PR description.


Assisted-by: PI (claude-opus-4-6-1m)


## **[Dual-Stream RHCOS 9.8 + RHCOS 10.2 — IUO Scope] - Quality Engineering Plan**

### **Metadata & Tracking**

- **Feature Tracking:** [VIRTSTRAT-83](https://redhat.atlassian.net/browse/VIRTSTRAT-83)
- **Epic Tracking:** [CNV-85268](https://redhat.atlassian.net/browse/CNV-85268)
- **IUO Story:** [CNV-85504](https://redhat.atlassian.net/browse/CNV-85504)
- **Parent STP:** [stp.md](stp.md)
Comment thread
OhadRevah marked this conversation as resolved.
- **QE Owner(s):** Ohad Revah (@OhadRevah)
- **SIG:** sig-iuo (Install, Upgrade, Operators)

### **Feature Overview**

This STP covers the IUO-specific aspects of dual-stream RHCOS support: validating that
OpenShift Virtualization deploys and functions correctly on RHCOS 10.2, diagnostic data collection works on RHCOS 10.2 nodes,
node placement policies are honored in mixed-version clusters, observability metrics work
as expected, and migration metrics are accurately reported during cross-version live migration.

This STP covers testing for OCP 5.0. Automation is required for migration metrics validation on dual-stream clusters.

---

### **I. Motivation and Requirements Review (QE Review Guidelines)**

#### **1. Requirement & User Story Review Checklist**

- [x] **Review Requirements**
- *SIG-specific requirements:*
- OpenShift Virtualization components deploy and report ready on RHCOS 10.2 worker nodes
- Must-gather collects complete and valid data from RHCOS 10.2 nodes, with no gaps compared to RHCOS 9.8
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Comment thread
OhadRevah marked this conversation as resolved.
- OpenShift Virtualization migration metrics (duration, data processed, bandwidth) are reported accurately during cross-version live migration
- Node placement policies are respected when scheduling and migrating VMs on dual-stream clusters

- [x] **Acceptance Criteria**
Comment thread
OhadRevah marked this conversation as resolved.
- OpenShift Virtualization components deploy and report ready on RHCOS 10.2 worker nodes
- Must-gather collects complete and valid data from RHCOS 10.2 and dual-stream cluster nodes
- Migration metrics (duration, data processed, bandwidth) are reported accurately during cross-version live migration on dual-stream clusters
- Node placement policies are respected when scheduling and migrating VMs on dual-stream clusters

- [x] **Testability**
- *Note any SIG-specific requirements that are unclear or untestable:* All requirements are testable
Comment thread
OhadRevah marked this conversation as resolved.
through existing IUO test suites and new migration metrics automation on RHCOS 10.2 and dual-stream clusters.

- [x] **Non-Functional Requirements (NFRs)**
- *SIG-specific NFRs:*
- Monitoring: CNV metrics (including migration metrics) must report correctly on RHCOS 10.2 nodes

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "CNV" is used in several user-facing sections (lines 48, 63, 139, 172). Consider using "OpenShift Virtualization" for consistency with the rest of the STP.


Assisted-by: PI (claude-opus-4-6-1m)

- *NFRs not covered and why:*
- Performance: N/A — no new IUO-specific performance requirements; covered by parent STP
- Security: N/A — no new auth or RBAC changes; FIPS requirement covered by parent STP
- Scalability: N/A — no new scale requirements for IUO components

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scalability NFR says "no new scale requirements" but P1 testing depends on cross-version live migration which has existing cluster-level parallelism limits (acknowledged in the parent STP). Please add: "existing cluster-level live migration parallelism limits apply (see parent STP)."


Assisted-by: PI (claude-opus-4-6-1m)

- UI: N/A — dual-stream RHCOS support introduces no new user journeys or UI elements for IUO; existing console functionality is unchanged
- Documentation: N/A — no IUO-specific documentation changes; release notes covered by parent STP

Comment thread
coderabbitai[bot] marked this conversation as resolved.
#### **2. Known Limitations**

None — reviewed and confirmed that no IUO-specific feature limitations apply for this release.
Comment thread
OhadRevah marked this conversation as resolved.
Comment thread
OhadRevah marked this conversation as resolved.

#### **3. Technology and Design Review**

- [x] **Developer Handoff/QE Kickoff**
- *Key takeaways and concerns:* CNV operators run the same el9.8 userland on both RHCOS 9.8 and
RHCOS 10.2 kernels. No operator code changes are expected, but kernel differences could surface
unexpected behavior in must-gather log collection or metrics.

- [x] **Technology Challenges**
- *List identified challenges:*
- Must-gather may encounter differences in log paths or system service names between RHCOS 9.8
and RHCOS 10.2 nodes, potentially causing incomplete data collection.
- *Impact on testing approach:* Must-gather output must be compared between RHCOS 9.8 and
RHCOS 10.2 to identify any gaps.

- [x] **API Extensions**
- *List new or modified user-facing APIs:* N/A — see parent STP
- *Testing impact:* N/A

- [x] **Test Environment Needs**
- *See environment requirements in Section II.3 and testing tools in Section II.3.1*

- [x] **Topology Considerations**
- *Describe topology requirements:* Same as parent STP. Dual-stream cluster required for
migration and node placement scenarios; RHCOS 10.2-only cluster required for IUO regression.
- *Impact on test design:* Node placement tests require labeling nodes by RHCOS version
and using node affinity to control VM scheduling and migration targets.

### **II. Software Test Plan (STP)**

#### **1. Scope of Testing**

**Testing Goals**

- **[P0]** OpenShift Virtualization components deploy, report ready, and function correctly on RHCOS 10.2-only and dual-stream clusters.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[CRITICAL] P0 Testing Goal has no matching Section III scenario

The P0 goal ("OpenShift Virtualization components deploy, report ready, and function correctly on RHCOS 10.2-only and dual-stream clusters") has no corresponding test scenario in Section III. Section III only contains P1 migration-metrics scenarios.

Per AGENTS.md: "Every Testing Goal from Section II.1 has a matching scenario."

Please either:

  1. Add a Tier 1/2 scenario for the P0 goal (e.g., "Verify OpenShift Virtualization operators deploy and report ready on RHCOS 10.2-only and dual-stream clusters, and a VM can be created and started successfully"), or
  2. Remove the P0 Testing Goal and keep deployment/readiness coverage exclusively under II.2 Regression Testing — regression tests are not listed in Section III.

Assisted-by: PI (claude-opus-4-6-1m)

- **[P1]** Verify migration metrics (duration, data processed, bandwidth) are reported accurately during cross-version live migration on dual-stream clusters (RHCOS 9.8 ↔ RHCOS 10.2).

**Out of Scope (Testing Scope Exclusions)**

- **Upgrade testing**
- *Rationale:* Relevant only from 5.1.0.
- *PM/Lead Agreement:* Martin Tessun / 2026-05-13

**Test Limitations**

- None — dual-stream cluster provisioning tooling is available and operational.

#### **2. Test Strategy**

**Functional**

- [x] **Functional Testing** — Validates IUO-specific features on RHCOS 10.2 and dual-stream clusters
- *Details:* Run existing IUO Tier 1 and Tier 2 suites on RHCOS 10.2-only and dual-stream clusters.

- [x] **Automation Testing** — Migration metrics validation on dual-stream clusters
- *Details:* Existing IUO Tier 1/2 suites run as-is on RHCOS 10.2 cluster. New automation
for migration metrics validation during cross-version live migration (RHCOS 9.8 ↔ RHCOS 10.2).

- [x] **Regression Testing** — IUO regression on RHCOS 10.2 and dual-stream clusters
- *Details:* Existing IUO Tier 1 and Tier 2 regression suites run on RHCOS 10.2-only and dual-stream clusters.
Failures triaged and bugs filed with RHCOS-version attribution.

- [ ] **Self-Validation Testing**
- *Details:* N/A — migration metrics tests are Tier 2 scenarios requiring dual-stream clusters;
not suitable for the self-validation health check package.

**Non-Functional**

- [ ] **Performance Testing**
- *Details:* Covered by parent STP.

- [ ] **Scale Testing**
- *Details:* Covered by parent STP.

- [ ] **Security Testing**
- *Details:* Covered by parent STP. FIPS requirement applies to all testing.

- [ ] **Usability Testing**
- *Details:* N/A — dual-stream RHCOS support introduces no new user journeys or UI elements for IUO; existing console functionality is unchanged.

- [x] **Monitoring** — Verify CNV metrics on RHCOS 10.2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring details say OpenShift Virtualization metrics on RHCOS 10.2 nodes must be verified, but Testing Goals and Section III only cover migration metrics. Either narrow the Monitoring claim to match the actual goal/scenario scope, or add a goal + scenario for baseline metrics verification on RHCOS 10.2.


Assisted-by: PI (claude-opus-4-6-1m)

- *Details:* Verify CNV metrics (including migration metrics: duration, data processed,
bandwidth) are reported correctly on RHCOS 10.2 nodes and during cross-version live migration.

**Integration & Compatibility**

- [ ] **Compatibility Testing**
- *Details:* Not applicable for this STP.

- [ ] **Upgrade Testing**
Comment thread
OhadRevah marked this conversation as resolved.
- *Details:* Relevant only from 5.1.0.

- [x] **Dependencies** — Dual-stream cluster provisioning
- *Details:* Dual-stream cluster provisioning tooling is available via QE DevOps. Dedicated CI lanes
are operational for both RHCOS 10.2-only and dual-stream clusters.

- [ ] **Cross Integrations**
- *Details:* Covered by parent STP.

**Infrastructure**

- [ ] **Cloud Testing**
- *Details:* Covered by parent STP.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

#### **3. Test Environment**

Covered by the parent STP. IUO-specific requirements:

- **Cluster Topology:**
- RHCOS 10.2-only cluster: for IUO Tier 1/2 regression
- Dual-stream cluster (RHCOS 9.8 + RHCOS 10.2 workers): for migration metrics,
node placement, and must-gather dual-node scenarios

- **OCP & OpenShift Virtualization Version(s):** OCP 5.0 with CNV 5.0

- **Storage:** ocs-storagecluster-ceph-rbd-virtualization

- **Platform:** Bare metal

- **Special Configurations:** None — RHCOS 9.8 worker nodes are identified by the `worker-rhcos9`
node role label, applied automatically during cluster provisioning.

#### **3.1. Testing Tools & Frameworks**

- **Test Framework:** Standard. Tests require logic to identify nodes by RHCOS version
for node placement and migration validation.

- **CI/CD:** Dedicated CI lanes for RHCOS 10.2 and dual-stream clusters:
- RHCOS 10.2: IUO testing and observability lanes
- Dual-stream: IUO testing and observability lanes

- **Other Tools:** N/A

#### **4. Entry Criteria**

Covered by the parent STP. IUO-specific entry criteria:

- [x] Requirements and design documents are **approved and merged**
- [x] RHCOS 10.2-only cluster available with IUO CI lanes provisioned
- [x] Dual-stream cluster available via QE DevOps tooling (required for migration/node-placement scenarios)
Comment thread
OhadRevah marked this conversation as resolved.

#### **5. Risks**

No IUO-specific risks identified. Feature-wide risks are covered by the parent STP.

Comment thread
coderabbitai[bot] marked this conversation as resolved.
---

### **III. Test Scenarios & Traceability**

IUO coverage for dual-stream RHCOS is primarily provided through regression testing
(existing Tier 1/2 suites on RHCOS 10.2 clusters). The following new test scenarios
are required for migration metrics validation on dual-stream clusters:

- **[CNV-85504]** — As a VM operator, I want migration metrics to be reported accurately when migrating from an RHCOS 9.8 node to an RHCOS 10.2 node.
- *Test Scenario:* [Tier 2] Live migrate a VM from an RHCOS 9.8 node to an RHCOS 10.2 node and verify that migration metrics (duration, data processed, bandwidth) are reported correctly.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Migration metrics scenarios say "reported correctly" — could you define measurable pass criteria? For example: duration > 0 and consistent with observed migration time; data processed > 0; bandwidth non-zero during active transfer. Same applies to line 217.


Assisted-by: PI (claude-opus-4-6-1m)

- *Priority:* P1

- **[CNV-85504]** — As a VM operator, I want migration metrics to be reported accurately when migrating from an RHCOS 10.2 node to an RHCOS 9.8 node.
- *Test Scenario:* [Tier 2] Live migrate a VM from an RHCOS 10.2 node to an RHCOS 9.8 node and verify that migration metrics (duration, data processed, bandwidth) are reported correctly.
- *Priority:* P1

---

### **IV. Sign-off and Approval**

This Software Test Plan requires approval from the following stakeholders:

* **Reviewers:**
- QE Architect: Ruth Netser (@rnetser)
- sig-iuo representatives: @orenc1 @hmeir @rlobillo

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer list uses bare GitHub handles without full names. Each reviewer should include role, name, and handle.

For example:

Suggested change
- sig-iuo representatives: @orenc1 @hmeir @rlobillo
- sig-iuo representatives: Oren Cohen (@orenc1), Hila Meir (@hmeir), Roberto Lobillo (@rlobillo)
- sig-virt representative: Akriti Gupta (@akri3i) (parent STP owner)

(Please verify the correct full names.)


Assisted-by: PI (claude-opus-4-6-1m)

- sig-virt representative: Akriti Gupta (parent STP owner)
* **Approvers:**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[CRITICAL] Approvers list is missing a Dev Lead

The parent STP lists "Principal Developer: Luboslav Pivarc @xpivarc" and the network child lists "Principal Developer: Edward Haas @EdDev" — but the IUO child only has QE Architect, sig-iuo Lead, and PM.

Please add the IUO or feature Dev Lead as an approver with name and GitHub handle.


Assisted-by: PI (claude-opus-4-6-1m)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still unresolved from my previous review — please add a Dev Lead as an approver with name and GitHub handle (e.g., Luboslav Pivarc @xpivarc or the IUO-specific dev lead).


Assisted-by: PI (claude-opus-4-6-1m)

- QE Architect: Ruth Netser (@rnetser)
- sig-iuo Lead: @hmeir

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approver entry lacks a display name. Approvers require role, name, and GitHub handle.

Suggested change
- sig-iuo Lead: @hmeir
- sig-iuo Lead: Hila Meir (@hmeir)

(Please verify the correct full name.)


Assisted-by: PI (claude-opus-4-6-1m)

- Product Manager: Martin Tessun (@mtessun)
Loading