SPLAT-2503: Fixed "null" value for vSphere api and ingress VIP in UPI scenario #2795

vr4manta · 2025-09-11T17:54:21Z

Changes

Fixed scenario for vSphere UPI when an update to machineCIDR is causing the status field update to fail due to "null" in the ingress and api VIP fields instead of empty string[]

Notes

This fix is needed for vSphere hybrid support. During CI e2e testing for the feature, we have hit the issue this PR is attempting to address. Not sure how this is not hit in the field if someone attempts to adjust infrastructure/cluster at Day 2, but it is very recreateable for this feature.

CI Job with the error: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/69114/rehearse-69114-periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env/1965813825533906944

Error in question:

2025-09-10T17:32:09.424252252Z I0910 17:32:09.424176       1 log.go:245] Reconciling Infrastructure.config.openshift.io cluster
2025-09-10T17:32:09.429111595Z I0910 17:32:09.429073       1 log.go:245] reconciling (config.openshift.io/v1, Kind=Infrastructure) /cluster
2025-09-10T17:32:09.436011516Z I0910 17:32:09.435947       1 log.go:245] Error while updating status of infrastructures.config.openshift.io/cluster: failed to apply / update (config.openshift.io/v1, Kind=Infrastructure) /cluster: Infrastructure.config.openshift.io "cluster" is invalid: [status.platformStatus.vsphere.apiServerInternalIPs: Invalid value: "null": platformStatus.vsphere.apiServerInternalIPs in body must be of type array: "null", status.platformStatus.vsphere.ingressIPs: Invalid value: "null": platformStatus.vsphere.ingressIPs in body must be of type array: "null", <nil>: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation]
2025-09-10T17:32:09.453184214Z I0910 17:32:09.453136       1 log.go:245] Network operator config updated with conditions:
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T17:21:09Z"
2025-09-10T17:32:09.453184214Z   message: 'Error while updating status of infrastructures.config.openshift.io/cluster:
2025-09-10T17:32:09.453184214Z     failed to apply / update (config.openshift.io/v1, Kind=Infrastructure) /cluster:
2025-09-10T17:32:09.453184214Z     Infrastructure.config.openshift.io "cluster" is invalid: [status.platformStatus.vsphere.apiServerInternalIPs:
2025-09-10T17:32:09.453184214Z     Invalid value: "null": platformStatus.vsphere.apiServerInternalIPs in body must
2025-09-10T17:32:09.453184214Z     be of type array: "null", status.platformStatus.vsphere.ingressIPs: Invalid value:
2025-09-10T17:32:09.453184214Z     "null": platformStatus.vsphere.ingressIPs in body must be of type array: "null",
2025-09-10T17:32:09.453184214Z     <nil>: Invalid value: "null": some validation rules were not checked because the
2025-09-10T17:32:09.453184214Z     object was invalid; correct the existing errors to complete validation]'
2025-09-10T17:32:09.453184214Z   reason: UpdateInfrastructureStatus
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Degraded
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:36:49Z"
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Upgradeable
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:36:49Z"
2025-09-10T17:32:09.453184214Z   status: "False"
2025-09-10T17:32:09.453184214Z   type: ManagementStateDegraded
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T17:26:28Z"
2025-09-10T17:32:09.453184214Z   status: "False"
2025-09-10T17:32:09.453184214Z   type: Progressing
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:38:33Z"
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Available

openshift-ci-robot · 2025-09-11T17:54:25Z

@vr4manta: This pull request references SPLAT-2503 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.21.0" version, but no target version was set.

In response to this:

SPLAT-2503

Changes

Fixed scenario for vSphere UPI when an update to machineCIDR is causing the status field update to fail due to "null" in the ingress and api VIP fields instead of empty string[]

Notes

CI Job with the error: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/69114/rehearse-69114-periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env/1965813825533906944

Error in question:

2025-09-10T17:32:09.424252252Z I0910 17:32:09.424176       1 log.go:245] Reconciling Infrastructure.config.openshift.io cluster
2025-09-10T17:32:09.429111595Z I0910 17:32:09.429073       1 log.go:245] reconciling (config.openshift.io/v1, Kind=Infrastructure) /cluster
2025-09-10T17:32:09.436011516Z I0910 17:32:09.435947       1 log.go:245] Error while updating status of infrastructures.config.openshift.io/cluster: failed to apply / update (config.openshift.io/v1, Kind=Infrastructure) /cluster: Infrastructure.config.openshift.io "cluster" is invalid: [status.platformStatus.vsphere.apiServerInternalIPs: Invalid value: "null": platformStatus.vsphere.apiServerInternalIPs in body must be of type array: "null", status.platformStatus.vsphere.ingressIPs: Invalid value: "null": platformStatus.vsphere.ingressIPs in body must be of type array: "null", <nil>: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation]
2025-09-10T17:32:09.453184214Z I0910 17:32:09.453136       1 log.go:245] Network operator config updated with conditions:
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T17:21:09Z"
2025-09-10T17:32:09.453184214Z   message: 'Error while updating status of infrastructures.config.openshift.io/cluster:
2025-09-10T17:32:09.453184214Z     failed to apply / update (config.openshift.io/v1, Kind=Infrastructure) /cluster:
2025-09-10T17:32:09.453184214Z     Infrastructure.config.openshift.io "cluster" is invalid: [status.platformStatus.vsphere.apiServerInternalIPs:
2025-09-10T17:32:09.453184214Z     Invalid value: "null": platformStatus.vsphere.apiServerInternalIPs in body must
2025-09-10T17:32:09.453184214Z     be of type array: "null", status.platformStatus.vsphere.ingressIPs: Invalid value:
2025-09-10T17:32:09.453184214Z     "null": platformStatus.vsphere.ingressIPs in body must be of type array: "null",
2025-09-10T17:32:09.453184214Z     <nil>: Invalid value: "null": some validation rules were not checked because the
2025-09-10T17:32:09.453184214Z     object was invalid; correct the existing errors to complete validation]'
2025-09-10T17:32:09.453184214Z   reason: UpdateInfrastructureStatus
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Degraded
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:36:49Z"
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Upgradeable
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:36:49Z"
2025-09-10T17:32:09.453184214Z   status: "False"
2025-09-10T17:32:09.453184214Z   type: ManagementStateDegraded
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T17:26:28Z"
2025-09-10T17:32:09.453184214Z   status: "False"
2025-09-10T17:32:09.453184214Z   type: Progressing
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:38:33Z"
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Available

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

vr4manta · 2025-09-11T17:56:46Z

/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env

openshift-ci · 2025-09-11T17:56:52Z

@vr4manta: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/afa28690-8f38-11f0-817a-67a706c6d1f6-0

vr4manta · 2025-09-12T11:55:56Z

/retest
/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env

openshift-ci · 2025-09-12T11:56:00Z

@vr4manta: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/717dfdd0-8fcf-11f0-8f3d-d290c9d84519-0

rvanderp3 · 2025-09-12T13:04:02Z

/lgtm

vr4manta · 2025-09-12T13:05:56Z

the /payload-job finally passed the virt step after adjusting the machineCIDR. The fix looks good for this scenario.

vr4manta · 2025-09-12T13:25:25Z

/assign @tssurya

openshift-ci-robot · 2025-09-12T14:16:21Z

@vr4manta: This pull request references SPLAT-2503 which is a valid jira issue.

In response to this:

SPLAT-2503

Changes

Fixed scenario for vSphere UPI when an update to machineCIDR is causing the status field update to fail due to "null" in the ingress and api VIP fields instead of empty string[]

Notes

This fix is needed for vSphere hybrid support. During CI e2e testing for the feature, we have hit the issue this PR is attempting to address. Not sure how this is not hit in the field if someone attempts to adjust infrastructure/cluster at Day 2, but it is very recreateable for this feature.

CI Job with the error: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/69114/rehearse-69114-periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env/1965813825533906944

Error in question:

2025-09-10T17:32:09.424252252Z I0910 17:32:09.424176       1 log.go:245] Reconciling Infrastructure.config.openshift.io cluster
2025-09-10T17:32:09.429111595Z I0910 17:32:09.429073       1 log.go:245] reconciling (config.openshift.io/v1, Kind=Infrastructure) /cluster
2025-09-10T17:32:09.436011516Z I0910 17:32:09.435947       1 log.go:245] Error while updating status of infrastructures.config.openshift.io/cluster: failed to apply / update (config.openshift.io/v1, Kind=Infrastructure) /cluster: Infrastructure.config.openshift.io "cluster" is invalid: [status.platformStatus.vsphere.apiServerInternalIPs: Invalid value: "null": platformStatus.vsphere.apiServerInternalIPs in body must be of type array: "null", status.platformStatus.vsphere.ingressIPs: Invalid value: "null": platformStatus.vsphere.ingressIPs in body must be of type array: "null", <nil>: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation]
2025-09-10T17:32:09.453184214Z I0910 17:32:09.453136       1 log.go:245] Network operator config updated with conditions:
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T17:21:09Z"
2025-09-10T17:32:09.453184214Z   message: 'Error while updating status of infrastructures.config.openshift.io/cluster:
2025-09-10T17:32:09.453184214Z     failed to apply / update (config.openshift.io/v1, Kind=Infrastructure) /cluster:
2025-09-10T17:32:09.453184214Z     Infrastructure.config.openshift.io "cluster" is invalid: [status.platformStatus.vsphere.apiServerInternalIPs:
2025-09-10T17:32:09.453184214Z     Invalid value: "null": platformStatus.vsphere.apiServerInternalIPs in body must
2025-09-10T17:32:09.453184214Z     be of type array: "null", status.platformStatus.vsphere.ingressIPs: Invalid value:
2025-09-10T17:32:09.453184214Z     "null": platformStatus.vsphere.ingressIPs in body must be of type array: "null",
2025-09-10T17:32:09.453184214Z     <nil>: Invalid value: "null": some validation rules were not checked because the
2025-09-10T17:32:09.453184214Z     object was invalid; correct the existing errors to complete validation]'
2025-09-10T17:32:09.453184214Z   reason: UpdateInfrastructureStatus
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Degraded
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:36:49Z"
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Upgradeable
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:36:49Z"
2025-09-10T17:32:09.453184214Z   status: "False"
2025-09-10T17:32:09.453184214Z   type: ManagementStateDegraded
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T17:26:28Z"
2025-09-10T17:32:09.453184214Z   status: "False"
2025-09-10T17:32:09.453184214Z   type: Progressing
2025-09-10T17:32:09.453184214Z - lastTransitionTime: "2025-09-10T16:38:33Z"
2025-09-10T17:32:09.453184214Z   status: "True"
2025-09-10T17:32:09.453184214Z   type: Available

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

vr4manta · 2025-09-17T14:33:30Z

/assign @danwinship

vr4manta · 2025-09-29T11:28:52Z

/retest

danwinship · 2025-09-29T13:58:36Z

/approve

openshift-ci · 2025-09-29T14:03:35Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danwinship, rvanderp3, vr4manta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [danwinship]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vr4manta · 2025-09-29T16:24:57Z

/verified by "periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env"
Verified fix using /payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env

openshift-ci-robot · 2025-09-29T16:25:08Z

@vr4manta: This PR has been marked as verified by "periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env".

In response to this:

/verified by "periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env"
Verified fix using /payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-vsphere-ovn-upi-hybrid-env

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2025-09-29T17:16:36Z

/retest-required

Remaining retests: 0 against base HEAD 6f6d0ee and 2 for PR HEAD f8a4b09 in total

vr4manta · 2025-10-02T11:51:18Z

/retest

openshift-ci-robot · 2025-10-02T15:27:30Z

/retest-required

Remaining retests: 0 against base HEAD 339bfc9 and 1 for PR HEAD f8a4b09 in total

openshift-ci-robot · 2025-10-08T02:39:36Z

/retest-required

Remaining retests: 0 against base HEAD 5ce4b3c and 0 for PR HEAD f8a4b09 in total

openshift-ci-robot · 2025-10-08T06:39:45Z

/hold

Revision f8a4b09 was retested 3 times: holding

vr4manta · 2025-10-08T11:03:41Z

/unhold
/retest-required

openshift-ci-robot · 2025-10-08T15:05:58Z

/retest-required

Remaining retests: 0 against base HEAD 5ce4b3c and 2 for PR HEAD f8a4b09 in total

openshift-ci · 2025-10-09T17:49:54Z

@vr4manta: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-serial	`f8a4b09`	link	false	`/test e2e-aws-ovn-serial`
ci/prow/security	`f8a4b09`	link	false	`/test security`
ci/prow/4.20-upgrade-from-stable-4.19-e2e-aws-ovn-upgrade	`f8a4b09`	link	false	`/test 4.20-upgrade-from-stable-4.19-e2e-aws-ovn-upgrade`
ci/prow/4.20-upgrade-from-stable-4.19-e2e-azure-ovn-upgrade	`f8a4b09`	link	false	`/test 4.20-upgrade-from-stable-4.19-e2e-azure-ovn-upgrade`
ci/prow/e2e-aws-hypershift-ovn-kubevirt	`f8a4b09`	link	false	`/test e2e-aws-hypershift-ovn-kubevirt`
ci/prow/e2e-aws-ovn-serial-ipsec-2of2	`f8a4b09`	link	false	`/test e2e-aws-ovn-serial-ipsec-2of2`
ci/prow/e2e-ovn-ipsec-step-registry	`f8a4b09`	link	unknown	`/test e2e-ovn-ipsec-step-registry`
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp-local-gw	`f8a4b09`	link	unknown	`/test e2e-metal-ipi-ovn-dualstack-bgp-local-gw`
ci/prow/hypershift-e2e-aks	`f8a4b09`	link	unknown	`/test hypershift-e2e-aks`
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp	`f8a4b09`	link	unknown	`/test e2e-metal-ipi-ovn-dualstack-bgp`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 11, 2025

openshift-ci bot requested review from creydr and cybertron September 11, 2025 17:55

Fixed "null" value for vSphere api and ingress VIP in UPI scenario

f8a4b09

vr4manta force-pushed the SPLAT-2503 branch from a081825 to f8a4b09 Compare September 11, 2025 18:46

openshift-ci bot assigned rvanderp3 Sep 12, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 12, 2025

openshift-ci bot assigned tssurya Sep 12, 2025

openshift-ci bot assigned danwinship Sep 17, 2025

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 29, 2025

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 29, 2025

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 8, 2025

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 8, 2025

SPLAT-2503: Fixed "null" value for vSphere api and ingress VIP in UPI scenario #2795

Are you sure you want to change the base?

SPLAT-2503: Fixed "null" value for vSphere api and ingress VIP in UPI scenario #2795

Conversation

vr4manta commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Notes

Uh oh!

openshift-ci-robot commented Sep 11, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Notes

Uh oh!

vr4manta commented Sep 11, 2025

Uh oh!

openshift-ci bot commented Sep 11, 2025

Uh oh!

vr4manta commented Sep 12, 2025

Uh oh!

openshift-ci bot commented Sep 12, 2025

Uh oh!

rvanderp3 commented Sep 12, 2025

Uh oh!

vr4manta commented Sep 12, 2025

Uh oh!

vr4manta commented Sep 12, 2025

Uh oh!

openshift-ci-robot commented Sep 12, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Notes

Uh oh!

vr4manta commented Sep 17, 2025

Uh oh!

vr4manta commented Sep 29, 2025

Uh oh!

danwinship commented Sep 29, 2025

Uh oh!

openshift-ci bot commented Sep 29, 2025

Uh oh!

vr4manta commented Sep 29, 2025

Uh oh!

openshift-ci-robot commented Sep 29, 2025

Uh oh!

openshift-ci-robot commented Sep 29, 2025

Uh oh!

vr4manta commented Oct 2, 2025

Uh oh!

openshift-ci-robot commented Oct 2, 2025

Uh oh!

openshift-ci-robot commented Oct 8, 2025

Uh oh!

openshift-ci-robot commented Oct 8, 2025

Uh oh!

vr4manta commented Oct 8, 2025

Uh oh!

openshift-ci-robot commented Oct 8, 2025

Uh oh!

openshift-ci bot commented Oct 9, 2025

Uh oh!

Uh oh!

vr4manta commented Sep 11, 2025 •

edited

Loading

openshift-ci-robot commented Sep 11, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Sep 12, 2025 •

edited by openshift-ci bot

Loading