Skip to content

OCPBUGS-95615: fix(kubevirt): use only first MachineInternalIP per family in EndpointSlice endpoints#8892

Open
dpateriya wants to merge 1 commit into
openshift:mainfrom
dpateriya:fix/hcco-endpointslice-extra-internal-ips
Open

OCPBUGS-95615: fix(kubevirt): use only first MachineInternalIP per family in EndpointSlice endpoints#8892
dpateriya wants to merge 1 commit into
openshift:mainfrom
dpateriya:fix/hcco-endpointslice-extra-internal-ips

Conversation

@dpateriya

@dpateriya dpateriya commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Since CAPK PR kubernetes-sigs/cluster-api-provider-kubevirt#366 began exposing all VMI interface IPs for dual-stack CSR approval, Machine.Status.Addresses includes ovn-k8s-mp0 management port IPs from the guest cluster's OVN overlay alongside the VM's routable pod network IP.

The HCCO machine controller was putting all MachineInternalIP addresses into EndpointSlice.Endpoint.Addresses, causing MetalLB/kube-proxy to round-robin traffic to non-routable ovn-k8s-mp0 IPs and producing ~50% connection timeouts on LoadBalancer services exposed via MetalLB.

The existing link-local filter (OCPBUGS-83711, PR #8264) does not catch these IPs because they are regular RFC1918 addresses (e.g. 10.8.x.2), not link-local.

This PR uses only the first MachineInternalIP per IP family, which is the VM's primary pod network IP on the management cluster. This matches the convention already used in e2e tests (firstMachineAddress in nodepool_kv_advanced_multinet_test.go).

Which issue(s) this PR fixes:

Related to OCPBUGS-83711 (link-local filter) and OCPBUGS-84269 / PR #8230 (CIDR conflict false positive) — same root cause (CAPK PR #366), different symptom.

Customer cases: 04485238, 04485252

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Made with Cursor

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of machines with multiple internal IPs so endpoint generation now uses only the first IPv4 and first IPv6 address per machine.
    • Prevents extra same-family addresses from being included when building KubeVirt-related service endpoints.
    • Added test coverage for machines reporting multiple IPv4 and IPv6 internal addresses.

…tSlice endpoints

Since CAPK PR openshift#366 exposes all VMI interface IPs for dual-stack CSR
approval, Machine.Status.Addresses includes ovn-k8s-mp0 management
port IPs from the guest cluster's OVN overlay alongside the VM's
routable pod network IP. The HCCO machine controller was putting all
MachineInternalIP addresses into EndpointSlice Endpoint.Addresses,
causing MetalLB/kube-proxy to round-robin traffic to non-routable
IPs and producing ~50% connection timeouts on LoadBalancer services.

The existing link-local filter (OCPBUGS-83711) does not catch these
IPs because they are regular RFC1918 addresses (e.g. 10.8.x.2), not
link-local.

Use only the first MachineInternalIP per IP family, which is the VM's
primary pod network IP on the management cluster. This matches the
convention already used in e2e tests (firstMachineAddress).

Signed-off-by: Divyam Pateriya <dpateriy@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This change modifies the address selection logic in the KubeVirt passthrough service reconciler to select only the first IPv4 address and first IPv6 address per machine when multiple addresses of the same family exist, rather than appending all addresses per family. Test data and a test case are added to verify machines with multiple same-family internal IP addresses correctly produce endpoint slices referencing only the first address per family.

Changes

Area Change
Address filtering logic Guards IPv4/IPv6 address append with emptiness check on respective slice, replacing unconditional append
Test coverage Adds machinesWithExtraInternalIPs test data and a test case validating filtering behavior

Sequence Diagram(s)

Not applicable — this is a targeted logic fix within a single function plus associated test additions, not a multi-component feature or workflow change.

Estimated code review effort: Medium

Suggested labels: bug, needs-tests-review

Suggested reviewers: Not enough information available to determine relevant reviewers from the provided summary.

🐰 Poem

A rabbit hopped through IPs galore,
Found doubles where there once were four,
"Just one per family," it declared with glee,
First address kept, the rest set free,
Now endpoint slices, clean once more.

🚥 Pre-merge checks | ✅ 11
✅ Passed checks (11 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All added test titles in machine_test.go are static strings; no timestamps, IPs, names, or generated values appear in test names.
Test Structure And Quality ✅ Passed PASS: The added test is a focused table-driven unit case using fake clients; no cluster resources, waits, or cleanup gaps, and it matches repo patterns.
Topology-Aware Scheduling Compatibility ✅ Passed Only EndpointSlice address filtering/test data changed; no replicas, affinity, nodeSelector, PDB, or topology-aware scheduling logic was added.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No Ginkgo e2e tests were added; the changes are controller unit tests, so the IPv4/disconnected-network e2e check is not applicable.
No-Weak-Crypto ✅ Passed Changed code only filters MachineInternalIP addresses; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret comparisons found.
Container-Privileges ✅ Passed PASS: PR only updates Go controller/test code for EndpointSlice IP selection; no container/K8s manifests or privilege-related fields were added in changed files.
No-Sensitive-Data-In-Logs ✅ Passed The diff only changes IP selection logic and test fixtures; it adds no new log statements or sensitive fields to logs.
Title check ✅ Passed The title clearly and accurately summarizes the main change to EndpointSlice address selection.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci openshift-ci Bot requested review from csrwng and sjenning July 1, 2026 17:59
@openshift-ci

openshift-ci Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dpateriya
Once this PR has been reviewed and has the lgtm label, please assign cblecker for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release and removed do-not-merge/needs-area labels Jul 1, 2026
@dpateriya dpateriya changed the title fix(kubevirt): use only first MachineInternalIP per family in EndpointSlice endpoints fix\OCPBUGS-95615 :( kubevirt) use only first MachineInternalIP per family in EndpointSlice endpoints Jul 1, 2026
@dpateriya

Copy link
Copy Markdown
Contributor Author

/jira refresh

@openshift-ci-robot

Copy link
Copy Markdown

@dpateriya: No Jira issue is referenced in the title of this pull request.
To reference a jira issue, add 'XYZ-NNN:' to the title of this pull request and request another refresh with /jira refresh.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@dpateriya dpateriya changed the title fix\OCPBUGS-95615 :( kubevirt) use only first MachineInternalIP per family in EndpointSlice endpoints fix: OCPBUGS-95615 : ( kubevirt) use only first MachineInternalIP per family in EndpointSlice endpoints Jul 1, 2026
@dpateriya

Copy link
Copy Markdown
Contributor Author

/jira refresh

@openshift-ci-robot

Copy link
Copy Markdown

@dpateriya: No Jira issue is referenced in the title of this pull request.
To reference a jira issue, add 'XYZ-NNN:' to the title of this pull request and request another refresh with /jira refresh.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@dpateriya dpateriya changed the title fix: OCPBUGS-95615 : ( kubevirt) use only first MachineInternalIP per family in EndpointSlice endpoints OCPBUGS-95615 : ( kubevirt) use only first MachineInternalIP per family in EndpointSlice endpoints Jul 1, 2026
@dpateriya

Copy link
Copy Markdown
Contributor Author

/jira refresh

@openshift-ci-robot

Copy link
Copy Markdown

@dpateriya: No Jira issue is referenced in the title of this pull request.
To reference a jira issue, add 'XYZ-NNN:' to the title of this pull request and request another refresh with /jira refresh.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@dpateriya dpateriya changed the title OCPBUGS-95615 : ( kubevirt) use only first MachineInternalIP per family in EndpointSlice endpoints OCPBUGS-95615: fix(kubevirt): use only first MachineInternalIP per family in EndpointSlice endpoints Jul 1, 2026
@openshift-ci-robot openshift-ci-robot added jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 1, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@dpateriya: This pull request references Jira Issue OCPBUGS-95615, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

What this PR does / why we need it:

Since CAPK PR kubernetes-sigs/cluster-api-provider-kubevirt#366 began exposing all VMI interface IPs for dual-stack CSR approval, Machine.Status.Addresses includes ovn-k8s-mp0 management port IPs from the guest cluster's OVN overlay alongside the VM's routable pod network IP.

The HCCO machine controller was putting all MachineInternalIP addresses into EndpointSlice.Endpoint.Addresses, causing MetalLB/kube-proxy to round-robin traffic to non-routable ovn-k8s-mp0 IPs and producing ~50% connection timeouts on LoadBalancer services exposed via MetalLB.

The existing link-local filter (OCPBUGS-83711, PR #8264) does not catch these IPs because they are regular RFC1918 addresses (e.g. 10.8.x.2), not link-local.

This PR uses only the first MachineInternalIP per IP family, which is the VM's primary pod network IP on the management cluster. This matches the convention already used in e2e tests (firstMachineAddress in nodepool_kv_advanced_multinet_test.go).

Which issue(s) this PR fixes:

Related to OCPBUGS-83711 (link-local filter) and OCPBUGS-84269 / PR #8230 (CIDR conflict false positive) — same root cause (CAPK PR #366), different symptom.

Customer cases: 04485238, 04485252

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Made with Cursor

Summary by CodeRabbit

  • Bug Fixes
  • Improved handling of machines with multiple internal IPs so endpoint generation now uses only the first IPv4 and first IPv6 address per machine.
  • Prevents extra same-family addresses from being included when building KubeVirt-related service endpoints.
  • Added test coverage for machines reporting multiple IPv4 and IPv6 internal addresses.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.28%. Comparing base (ce9dd2c) to head (9b87317).
⚠️ Report is 27 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8892   +/-   ##
=======================================
  Coverage   43.28%   43.28%           
=======================================
  Files         771      771           
  Lines       95503    95503           
=======================================
  Hits        41335    41335           
  Misses      51284    51284           
  Partials     2884     2884           
Files with missing lines Coverage Δ
...usterconfigoperator/controllers/machine/machine.go 67.00% <100.00%> (ø)
Flag Coverage Δ
cmd-support 36.67% <ø> (ø)
cpo-hostedcontrolplane 45.31% <ø> (ø)
cpo-other 45.10% <100.00%> (ø)
hypershift-operator 53.59% <ø> (ø)
other 31.69% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@openshift-ci

openshift-ci Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

@dpateriya: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@dpateriya

Copy link
Copy Markdown
Contributor Author

/Verified by me

Env: OCP 4.21.21 Management Cluster
Patched Image: quay.io/rhn_support_dpateriy/hypershift:OCPBUGS-95615-cpo

Machine addresses (before fix)

Machine kubevirt-test-vh8dr-4t9bh reports two non-link-local IPv4 InternalIPs:

InternalIP: 10.130.2.8     ← routable VM pod IP
InternalIP: 10.132.0.2     ← ovn-k8s-mp0 management port IP (non-routable)

BEFORE (unpatched HCCO)

EndpointSlice contains both addresses — traffic round-robins to the non-routable IP:

# oc get endpointslice a74e87e10acbc4a4cabb0d67d79efe20-kubevirt-test-vh8dr-4t9bh-ipv4 -n clusters-kubevirt-test -o yaml
endpoints:
- addresses:
  - 10.130.2.8
  - 10.132.0.2    # ← non-routable, causes ~50% connection timeouts
  conditions:
    ready: true
    serving: true
    terminating: false

AFTER (patched HCCO)

Patched the HCCO deployment with the fix image, deleted EndpointSlices, and they were recreated with only the first address per family:

# oc get endpointslice -n clusters-kubevirt-test -l endpointslice.kubernetes.io/managed-by=control-plane-operator.hypershift.openshift.io
NAME                                                                            ADDRESSTYPE   PORTS         ENDPOINTS    AGE
a74e87e10acbc4a4cabb0d67d79efe20-kubevirt-test-vh8dr-4t9bh-ipv4                 IPv4          31797,32158   10.130.2.8   4s
default-ingress-passthrough-service-p4j29k9h2m-kubevirt-test-vh8dr-4t9bh-ipv4   IPv4          32707         10.130.2.8   4s

Only 10.130.2.8 (the routable VM pod IP) is present. The 10.132.0.2 (ovn-k8s-mp0) is correctly filtered.

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jul 4, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@dpateriya: This PR has been marked as verified by me.

Details

In response to this:

/Verified by me

Env: OCP 4.21.21 Management Cluster
Patched Image: quay.io/rhn_support_dpateriy/hypershift:OCPBUGS-95615-cpo

Machine addresses (before fix)

Machine kubevirt-test-vh8dr-4t9bh reports two non-link-local IPv4 InternalIPs:

InternalIP: 10.130.2.8     ← routable VM pod IP
InternalIP: 10.132.0.2     ← ovn-k8s-mp0 management port IP (non-routable)

BEFORE (unpatched HCCO)

EndpointSlice contains both addresses — traffic round-robins to the non-routable IP:

# oc get endpointslice a74e87e10acbc4a4cabb0d67d79efe20-kubevirt-test-vh8dr-4t9bh-ipv4 -n clusters-kubevirt-test -o yaml
endpoints:
- addresses:
 - 10.130.2.8
 - 10.132.0.2    # ← non-routable, causes ~50% connection timeouts
 conditions:
   ready: true
   serving: true
   terminating: false

AFTER (patched HCCO)

Patched the HCCO deployment with the fix image, deleted EndpointSlices, and they were recreated with only the first address per family:

# oc get endpointslice -n clusters-kubevirt-test -l endpointslice.kubernetes.io/managed-by=control-plane-operator.hypershift.openshift.io
NAME                                                                            ADDRESSTYPE   PORTS         ENDPOINTS    AGE
a74e87e10acbc4a4cabb0d67d79efe20-kubevirt-test-vh8dr-4t9bh-ipv4                 IPv4          31797,32158   10.130.2.8   4s
default-ingress-passthrough-service-p4j29k9h2m-kubevirt-test-vh8dr-4t9bh-ipv4   IPv4          32707         10.130.2.8   4s

Only 10.130.2.8 (the routable VM pod IP) is present. The 10.132.0.2 (ovn-k8s-mp0) is correctly filtered.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants