Skip to content

WIP: CNTRLPLANE-371: Clean up gates for 4.20 #2259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

WIP: CNTRLPLANE-371: Clean up gates for 4.20 #2259

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
48911fc
Clean up gates that are enabled by default in Kubernetes
bertinatto Apr 2, 2025
dcb647a
make update
bertinatto Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions features.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,22 +44,17 @@
| MetricsCollectionProfiles| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| MinimumKubeletVersion| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| MixedCPUsAllocation| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| NodeSwap| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to keep this as we only want to turn on swap in certain cases

| NutanixMultiSubnets| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| OVNObservability| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| OnClusterBuild| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| PinnedImages| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| PlatformOperators| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| ProcMountType| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| RouteAdvertisements| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| RouteExternalCertificate| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| ServiceAccountTokenNodeBinding| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| SignatureStores| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| SigstoreImageVerification| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| TranslateStreamCloseWebsocketRequests| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| UpgradeStatus| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| UserNamespacesPodSecurityStandards| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| UserNamespacesSupport| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| VSphereConfigurableMaxAllowedBlockVolumesPerNode| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| VSphereHostVMGroupZonal| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| VSphereMultiDisk| | | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
Expand All @@ -75,8 +70,6 @@
| BareMetalLoadBalancer| <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| BuildCSIVolumes| <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| ChunkSizeMiB| <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| CloudDualStackNodeIPs| <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| DisableKubeletCloudCredentialProviders| <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| GCPLabelsTags| <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| HardwareSpeed| <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
| IngressControllerLBSubnetsAWS| <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> | <span style="background-color: #519450">Enabled</span> |
Expand Down
56 changes: 0 additions & 56 deletions features/features.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,6 @@ var (
enhancementPR("https://github.com/openshift/enhancements/pull/1706").
mustRegister()

FeatureGateServiceAccountTokenNodeBinding = newFeatureGate("ServiceAccountTokenNodeBinding").
reportProblemsToJiraComponent("apiserver-auth").
contactPerson("stlaz").
productScope(kubernetes).
enhancementPR("https://github.com/kubernetes/enhancements/issues/4193").
enableIn(configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()
Copy link
Member Author

@bertinatto bertinatto Apr 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This gate has been locked on by default in Kubernetes 1.33, so we need to drop it from here.
CC @ibihim

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop it? you mean we remove it completely, instead of adding configv1.Default?
Hm, means we would need to drop the feature gate, wherever it is being used?


FeatureGateValidatingAdmissionPolicy = newFeatureGate("ValidatingAdmissionPolicy").
reportProblemsToJiraComponent("kube-apiserver").
contactPerson("benluddy").
Expand Down Expand Up @@ -92,14 +84,6 @@ var (
enableIn(configv1.Default, configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

FeatureGateNodeSwap = newFeatureGate("NodeSwap").
reportProblemsToJiraComponent("node").
contactPerson("ehashman").
productScope(kubernetes).
enhancementPR("https://github.com/kubernetes/enhancements/issues/2400").
enableIn(configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is enabled by default since Kubernetes 1.30: https://github.com/kubernetes/kubernetes/blob/7cd7fea30d054efe4f2cc60073a29d2c618258ec/pkg/features/kube_features.go#L1546-L1550

Should this be dropped?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it locked on yet? Or does this gate existing force it down to TPNU clusters only?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the latter, we want to keep this TPNU only

FeatureGateMachineAPIProviderOpenStack = newFeatureGate("MachineAPIProviderOpenStack").
reportProblemsToJiraComponent("openstack").
contactPerson("egarcia").
Expand Down Expand Up @@ -195,14 +179,6 @@ var (
enableIn(configv1.Default, configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

FeatureGateCloudDualStackNodeIPs = newFeatureGate("CloudDualStackNodeIPs").
reportProblemsToJiraComponent("machine-config-operator/platform-baremetal").
contactPerson("mkowalsk").
productScope(kubernetes).
enhancementPR("https://github.com/kubernetes/enhancements/issues/3705").
enableIn(configv1.Default, configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was removed in 1.32: https://github.com/kubernetes/kubernetes/blob/7cd7fea30d054efe4f2cc60073a29d2c618258ec/CHANGELOG/CHANGELOG-1.32.md?plain=1#L962

Should we drop this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already have a PR open to drop this, but yes, whoever of us gets there first

FeatureGateVSphereHostVMGroupZonal = newFeatureGate("VSphereHostVMGroupZonal").
reportProblemsToJiraComponent("splat").
contactPerson("jcpowermac").
Expand Down Expand Up @@ -409,14 +385,6 @@ var (
enableIn(configv1.Default, configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

FeatureGateDisableKubeletCloudCredentialProviders = newFeatureGate("DisableKubeletCloudCredentialProviders").
reportProblemsToJiraComponent("cloud-provider").
contactPerson("jspeed").
productScope(kubernetes).
enhancementPR("https://github.com/kubernetes/enhancements/issues/2395").
enableIn(configv1.Default, configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in 1.33: https://github.com/kubernetes/kubernetes/blob/7cd7fea30d054efe4f2cc60073a29d2c618258ec/CHANGELOG/CHANGELOG-1.33.md?plain=1#L366

@JoelSpeed should we drop it in 4.20?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already have a PR open to drop this, but yes, whoever of us gets there first

FeatureGateOnClusterBuild = newFeatureGate("OnClusterBuild").
reportProblemsToJiraComponent("MachineConfigOperator").
contactPerson("dkhater").
Expand Down Expand Up @@ -465,14 +433,6 @@ var (
enableIn(configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

FeatureGateTranslateStreamCloseWebsocketRequests = newFeatureGate("TranslateStreamCloseWebsocketRequests").
reportProblemsToJiraComponent("kube-apiserver").
contactPerson("akashem").
productScope(kubernetes).
enhancementPR("https://github.com/kubernetes/enhancements/issues/4006").
enableIn(configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enabled by default in 1.30: https://github.com/kubernetes/kubernetes/blob/7cd7fea30d054efe4f2cc60073a29d2c618258ec/pkg/features/kube_features.go#L1823-L1826

@tkashem do you know if we should drop it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll keep it for now until these features are GA'ed in upstream.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless it's locked on by default upstream, we should keep this until we GA the feature

FeatureGateVolumeAttributesClass = newFeatureGate("VolumeAttributesClass").
reportProblemsToJiraComponent("Storage / Kubernetes External Components").
contactPerson("dfajmon").
Expand Down Expand Up @@ -671,14 +631,6 @@ var (
enableIn(configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

FeatureGateUserNamespacesSupport = newFeatureGate("UserNamespacesSupport").
reportProblemsToJiraComponent("Node").
contactPerson("haircommander").
productScope(kubernetes).
enhancementPR("https://github.com/kubernetes/enhancements/issues/127").
enableIn(configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

Copy link
Member Author

@bertinatto bertinatto Apr 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is beta and on by default in 1.33.
@haircommander confirmed on Slack that this will be on by default in 4.20

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it locked on upstream, or just on by default?

We may instead want to promote the feature to GA here and retain the ability to disable it if we find issues, for at least 1 more release

FeatureGateUserNamespacesPodSecurityStandards = newFeatureGate("UserNamespacesPodSecurityStandards").
reportProblemsToJiraComponent("Node").
contactPerson("haircommander").
Expand All @@ -687,14 +639,6 @@ var (
enableIn(configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

FeatureGateProcMountType = newFeatureGate("ProcMountType").
reportProblemsToJiraComponent("Node").
contactPerson("haircommander").
productScope(kubernetes).
enhancementPR("https://github.com/kubernetes/enhancements/issues/4265").
enableIn(configv1.DevPreviewNoUpgrade, configv1.TechPreviewNoUpgrade).
mustRegister()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enabled by default in 1.33: https://github.com/kubernetes/kubernetes/blob/7cd7fea30d054efe4f2cc60073a29d2c618258ec/pkg/features/kube_features.go#L1619-L1623

@haircommander should we drop this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes we should!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it locked on upstream, or just on by default?

We may instead want to promote the feature to GA here and retain the ability to disable it if we find issues, for at least 1 more release

FeatureGateVSphereMultiNetworks = newFeatureGate("VSphereMultiNetworks").
reportProblemsToJiraComponent("SPLAT").
contactPerson("rvanderp").
Expand Down
21 changes: 0 additions & 21 deletions payload-manifests/featuregates/featureGate-Hypershift-Default.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,9 +133,6 @@
{
"name": "NewOLMCatalogdAPIV1Metas"
},
{
"name": "NodeSwap"
},
{
"name": "NutanixMultiSubnets"
},
Expand All @@ -151,9 +148,6 @@
{
"name": "PlatformOperators"
},
{
"name": "ProcMountType"
},
{
"name": "RouteAdvertisements"
},
Expand All @@ -166,9 +160,6 @@
{
"name": "SELinuxMount"
},
{
"name": "ServiceAccountTokenNodeBinding"
},
{
"name": "ShortCertRotation"
},
Expand All @@ -181,18 +172,12 @@
{
"name": "SigstoreImageVerificationPKI"
},
{
"name": "TranslateStreamCloseWebsocketRequests"
},
{
"name": "UpgradeStatus"
},
{
"name": "UserNamespacesPodSecurityStandards"
},
{
"name": "UserNamespacesSupport"
},
{
"name": "VSphereConfigurableMaxAllowedBlockVolumesPerNode"
},
Expand Down Expand Up @@ -237,12 +222,6 @@
{
"name": "ChunkSizeMiB"
},
{
"name": "CloudDualStackNodeIPs"
},
{
"name": "DisableKubeletCloudCredentialProviders"
},
{
"name": "ExternalOIDC"
},
Expand Down
21 changes: 0 additions & 21 deletions payload-manifests/featuregates/featureGate-Hypershift-DevPreviewNoUpgrade.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,6 @@
{
"name": "ChunkSizeMiB"
},
{
"name": "CloudDualStackNodeIPs"
},
{
"name": "ClusterMonitoringConfig"
},
Expand All @@ -92,9 +89,6 @@
{
"name": "DNSNameResolver"
},
{
"name": "DisableKubeletCloudCredentialProviders"
},
{
"name": "DualReplica"
},
Expand Down Expand Up @@ -206,9 +200,6 @@
{
"name": "NodeDisruptionPolicy"
},
{
"name": "NodeSwap"
},
{
"name": "NutanixMultiSubnets"
},
Expand All @@ -233,9 +224,6 @@
{
"name": "PrivateHostedZoneAWS"
},
{
"name": "ProcMountType"
},
{
"name": "RouteAdvertisements"
},
Expand All @@ -248,9 +236,6 @@
{
"name": "SELinuxMount"
},
{
"name": "ServiceAccountTokenNodeBinding"
},
{
"name": "SetEIPForNLBIngressController"
},
Expand All @@ -266,18 +251,12 @@
{
"name": "SigstoreImageVerificationPKI"
},
{
"name": "TranslateStreamCloseWebsocketRequests"
},
{
"name": "UpgradeStatus"
},
{
"name": "UserNamespacesPodSecurityStandards"
},
{
"name": "UserNamespacesSupport"
},
{
"name": "VSphereConfigurableMaxAllowedBlockVolumesPerNode"
},
Expand Down
21 changes: 0 additions & 21 deletions payload-manifests/featuregates/featureGate-Hypershift-TechPreviewNoUpgrade.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,6 @@
{
"name": "ChunkSizeMiB"
},
{
"name": "CloudDualStackNodeIPs"
},
{
"name": "ClusterMonitoringConfig"
},
Expand All @@ -107,9 +104,6 @@
{
"name": "DNSNameResolver"
},
{
"name": "DisableKubeletCloudCredentialProviders"
},
{
"name": "DualReplica"
},
Expand Down Expand Up @@ -218,9 +212,6 @@
{
"name": "NodeDisruptionPolicy"
},
{
"name": "NodeSwap"
},
{
"name": "NutanixMultiSubnets"
},
Expand All @@ -245,18 +236,12 @@
{
"name": "PrivateHostedZoneAWS"
},
{
"name": "ProcMountType"
},
{
"name": "RouteAdvertisements"
},
{
"name": "RouteExternalCertificate"
},
{
"name": "ServiceAccountTokenNodeBinding"
},
{
"name": "SetEIPForNLBIngressController"
},
Expand All @@ -266,18 +251,12 @@
{
"name": "SigstoreImageVerification"
},
{
"name": "TranslateStreamCloseWebsocketRequests"
},
{
"name": "UpgradeStatus"
},
{
"name": "UserNamespacesPodSecurityStandards"
},
{
"name": "UserNamespacesSupport"
},
{
"name": "VSphereConfigurableMaxAllowedBlockVolumesPerNode"
},
Expand Down
Loading