Skip to content

Releases: kubernetes/kops

v1.24.0-beta.3

27 Jun 06:08
@olemarkus olemarkus
v1.24.0-beta.3
9a0ea1d
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.0-beta.3 Pre-release
Pre-release

Release Notes

https://kops.sigs.k8s.io/releases/1.24-notes/

What's Changed

Full Changelog: v1.24.0-beta.2...v1.24.0-beta.3

Contributors

olemarkus, hakman, and zetaab
Assets 24
Loading

v1.24.0-beta.2

12 Jun 16:48
@olemarkus olemarkus
v1.24.0-beta.2
0997f9e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.0-beta.2 Pre-release
Pre-release

Release Notes

https://kops.sigs.k8s.io/releases/1.24-notes/

What's Changed

Full Changelog: v1.24.0-beta.1...v1.24.0-beta.2

Contributors

olemarkus and hakman
Assets 24
Loading

v1.24.0-beta.1

05 Jun 05:41
@hakman hakman
v1.24.0-beta.1
f95ef56
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.0-beta.1 Pre-release
Pre-release

Release Notes

https://kops.sigs.k8s.io/releases/1.24-notes/

What's Changed

New Contributors

Full Changelog: v1.24.0-alpha.5...v1.24.0-beta.1

Contributors

olemarkus, diversario, and 7 other contributors
Assets 24
Loading

v1.24.0-alpha.5

22 May 06:11
@olemarkus olemarkus
v1.24.0-alpha.5
97b5795
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.0-alpha.5 Pre-release
Pre-release

Release Notes

https://kops.sigs.k8s.io/releases/1.24-notes/

What's Changed

New Contributors

Full Changelog: v1.24.0-alpha.4...v1.24.0-alpha.5

Contributors

olemarkus, justinsb, and 10 other contributors
Assets 24
Loading

v1.24.0-alpha.4

11 May 12:27
@hakman hakman
v1.24.0-alpha.4
6846960
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.0-alpha.4 Pre-release
Pre-release

Release Notes

https://kops.sigs.k8s.io/releases/1.24-notes/

What's Changed

Read more

Contributors

glebiller, olemarkus, and 26 other contributors
Assets 24
Loading

v1.23.2

11 May 16:55
@olemarkus olemarkus
v1.23.2
4125bbb
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.23.2

Release notes for kOps 1.23 series

Significant changes

  • If the Kubernetes version is 1.23 or later and the external AWS Cloud Controller Manager is
    being used, then Kubernetes Node resources will be named after their AWS instance ID instead of their domain name and
    managed subnets will be configured to launch instances with Resource Based Names.

  • Support for ShutdownGracePeriod and ShutdownGracePeriodCriticalPods. By default, kOps will set ShutdownGracePeriod to 30 seconds and ShutdownGracePeriodCriticalPods to 10 seconds if the Kubernetes version is above 1.21.

  • By enabling the pod identity webhook, you no longer need to modify your Pod specs to assume IAM roles.

Breaking changes

  • Support for Kubernetes version 1.17 has been removed.

  • Support for the Lyft CNI has been removed.

  • The Weave CNI is not supported for Kubernetes 1.23 or later.

  • Support for CentOS 7 has been removed.

  • Support for CentOS 8 has been removed (replaced by Rocky Linux 8).

  • Support for Debian 9 has been removed.

  • Support for RHEL 7 is has been removed.

  • Support for Ubuntu 16.04 (Xenial) has been removed.

  • Cilium now has disable-cnp-status-updates: true by default. Set this to false if you rely on the CiliumNetworkPolicy status fields.

Required actions

Deprecations

  • Support for Kubernetes version 1.18 is deprecated and will be removed in kOps 1.24.

  • Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

  • All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

  • The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in future versions of kOps.

  • Due to lack of maintainers, the Aliyun/Alibaba Cloud support has been deprecated and will be removed in kOps 1.24.

  • Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

Other changes of note

  • The kops create cluster command has a new --discovery-store flag for specifying a public store for the OIDC-compatible discovery documents.
    If this flag is used in AWS, it will enable IRSA.

  • If externalDns.provider is external-dns, then externalDns.watchIngress will now default to true.

  • This release introduces a v1alpha3 API version. This API version is a work in progress and is likely to be replaced in kOps 1.24.
    It is recommended to keep using the v1alpha2 API version.

  • IPv6 pod subnets is in a working state using public IPv6 addresses for the Pod network. This works with both Cilium and Calico. IPv6 is still behind a feature flag until service controllers and addons implement support for IPv6. See the IPv6 documentation.

  • The kops rolling-update cluster command has a new --drain-timeout flag for specifying the maximum amount of time to wait when attempting to drain a node. Previously, rolling-updates would attempt to drain a node for an indefinite amount of time. If --drain-timeout is not specified, a default of 15 minutes is applied.

  • Fix inconsistent output of kops get clusters -ojson. This will now always return a list (irrespective of a single or multiple clusters) to keep the format consistent. However, note that kops get cluster dev.example.com -ojson will continue to work as previously, and will return a single object.

  • Digital Ocean kops now has vpc support. You can specify a network-cidr range while creating the kops cluster. kops resources will be created in the new vpc range. Also supports shared vpc; you can specify the vpc uuid while creating kops cluster.

1.23.1 to 1.23.2

Assets 26
Loading

v1.23.1

20 Apr 12:04
@olemarkus olemarkus
v1.23.1
83ccae8
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.23.1

Release notes for kOps 1.23 series

Significant changes

  • If the Kubernetes version is 1.23 or later and the external AWS Cloud Controller Manager is
    being used, then Kubernetes Node resources will be named after their AWS instance ID instead of their domain name and
    managed subnets will be configured to launch instances with Resource Based Names.

  • Support for ShutdownGracePeriod and ShutdownGracePeriodCriticalPods. By default, kOps will set ShutdownGracePeriod to 30 seconds and ShutdownGracePeriodCriticalPods to 10 seconds if the Kubernetes version is above 1.21.

  • By enabling the pod identity webhook, you no longer need to modify your Pod specs to assume IAM roles.

Breaking changes

  • Support for Kubernetes version 1.17 has been removed.

  • Support for the Lyft CNI has been removed.

  • The Weave CNI is not supported for Kubernetes 1.23 or later.

  • Support for CentOS 7 has been removed.

  • Support for CentOS 8 has been removed.

  • Support for Debian 9 has been removed.

  • Support for RHEL 7 is has been removed.

  • Support for Ubuntu 16.04 (Xenial) has been removed.

  • Support for Ubuntu 18.04 (Bionic) has been removed.

  • Cilium now has disable-cnp-status-updates: true by default. Set this to false if you rely on the CiliumNetworkPolicy status fields.

Required actions

Deprecations

  • Support for Kubernetes version 1.18 is deprecated and will be removed in kOps 1.24.

  • Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

  • All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

  • The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in future versions of kOps.

  • Due to lack of maintainers, the Aliyun/Alibaba Cloud support has been deprecated and will be removed in kOps 1.24.

  • Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

Other changes of note

  • The kops create cluster command has a new --discovery-store flag for specifying a public store for the OIDC-compatible discovery documents.
    If this flag is used in AWS, it will enable IRSA.

  • If externalDns.provider is external-dns, then externalDns.watchIngress will now default to true.

  • This release introduces a v1alpha3 API version. This API version is a work in progress and is likely to be replaced in kOps 1.24.
    It is recommended to keep using the v1alpha2 API version.

  • IPv6 pod subnets is in a working state using public IPv6 addresses for the Pod network. This works with both Cilium and Calico. IPv6 is still behind a feature flag until service controllers and addons implement support for IPv6. See the IPv6 documentation.

  • The kops rolling-update cluster command has a new --drain-timeout flag for specifying the maximum amount of time to wait when attempting to drain a node. Previously, rolling-updates would attempt to drain a node for an indefinite amount of time. If --drain-timeout is not specified, a default of 15 minutes is applied.

  • Fix inconsistent output of kops get clusters -ojson. This will now always return a list (irrespective of a single or multiple clusters) to keep the format consistent. However, note that kops get cluster dev.example.com -ojson will continue to work as previously, and will return a single object.

  • Digital Ocean kops now has vpc support. You can specify a network-cidr range while creating the kops cluster. kops resources will be created in the new vpc range. Also supports shared vpc; you can specify the vpc uuid while creating kops cluster.

1.23.0 to 1.23.1

Assets 26
Loading

v1.22.5

20 Apr 12:00
@olemarkus olemarkus
v1.22.5
bd2ae03
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.22.5

Release notes for kOps 1.22 series

Significant changes

Instance metadata service version 2

On AWS, kOps will enable Instance Metadata Service Version 2 and require tokens on new clusters with Kubernetes 1.22. In addition, the following max hop limits will be set by default:

  • worker and API server Nodes, and bastions, will have a limit of 1 hop.
  • control plane nodes will have a limit of 3 hops to accommodate for controller Pods without host networking that need to assume roles.

This will increase security by default, but may break some types of workloads. In order to revert to old behavior, add the following to the InstanceGroup:

spec:
  instanceMetadata:
    httpTokens: optional

External ServiceAccountPermissions

Many of kOps addons can now make direct use of external permissions.
This can be enabled by adding the following to the Cluster spec:

spec:
  iam:
    useServiceAccountExternalPermissions: true

Currently this is only available using the AWS cloud provider.

Managed nvidia instances

kOps can now provision instances with nvidia GPUs and configure it for container workloads without the need of hooks and operators. See GPU support

Breacking change in NodeLocalDNS

Since 1.22.0 Cluster spec.kubeDNS.nodeLocalDNS.forwardToKubeDNS default behaviour changes from true to false.

Other significant changes

  • New clusters on AWS will no longer provision an SSH public key by default. To provision
    an SSH public key on a new cluster, use the --ssh-public-key flag to kops create cluster.

  • The kOps Terraform support now renders managed files through the Terraform configuration instead
    of writing them to S3 directly. This defers changes to these files until the time of terraform apply.
    This feature may be temporarily disabled by turning off the TerraformManagedFiles feature flag
    using export KOPS_FEATURE_FLAGS="-TerraformManagedFiles".

  • kOps now implements graceful rotation of its Certificate Authorities and the service
    account signing key. See the documentation on How to rotate all secrets / credentials

  • New clusters running Kubernetes 1.22 will have AWS EBS CSI driver enabled by default.

  • kOps now supports Debian 11 (Bullseye).

  • kOps can now use external-dns as a drop-in replacement for dns-controller.

Breaking changes

Control plane pods no longer mount /srv/kubernetes

For security reasons, /srv/kubernetes is no longer mounted in the kube-apiserver and kube-controller-manager Pods. This also means the files in the default file assets path will be unavailable. If you have file assets or other files needed by kube-apiserver, you must put these into /srv/kubernetes/kube-apiserver/ or /srv/kubernetes/kube-controller-manager, respectively.

For file assets, it means adding an explicit path as shown below:

  fileAssets:
  - name: audit-policy-config
    path: /srv/kubernetes/kube-apiserver/audit-policy-config.yaml # make sure you add the path
    roles:
    - Master
    content: |
      apiVersion: audit.k8s.io/v1
      kind: Policy
      rules:
      - level: Metadata

Other breaking changes

  • Support for Kubernetes versions 1.15 and 1.16 has been removed.

  • The legacy addons from https://github.com/kubernetes/kops/tree/master/addons have been deprecated and will not be available in Kubernetes 1.23+. Use managed addons instead.

  • The legacy location for downloads s3://https://kubeupv2.s3.amazonaws.com/kops/ has been deprecated and will not be used for new releases. The new canonical downloads location is https://artifacts.k8s.io/binaries/kops/.

  • The assets phase of kops update cluster has been removed. It is replaced by the new kops get assets --copy command.

  • Support for importing and converting kubeup clusters has been removed.

  • Support for Cilium and RHEL 8 has been removed. Cilium users will need to migrate to a distribution with a newer Linux kernel.

Required actions

  • Amazon Linux 2 users are encouraged to use the AMIs based on the 5.10 Linux kernel. See the documentation for more information.

  • Terraform support now requires Terraform >=0.15.0.
    Users on older versions must follow Terraform's recommended upgrade path of applying one minor version at a time prior to running kops update cluster --target terraform.

  • The kOps Terraform support now renders managed files through the Terraform configuration instead
    of writing them to S3 directly. If, after upgrading kOps and applying a new Terraform plan,
    you subsequently downgrade to an earlier version of kOps, the generated plan will delete these
    files, breaking the cluster. Prior to applying the plan, you will need to orphan all the
    aws_s3_bucket_object objects the plan wants to destroy. Use terraform state rm on each of them.
    Then re-run terraform plan until there are no such objects in the plan.

    If you applied the plan without first orphaning all of these objects, fix the cluster by re-running
    kops update cluster --target terraform.

  • Terraform users of clusters with names beginning with digits will need to move resources prior to upgrading to kOps 1.22. Some of the following commands will need to be run depending on the particular cluster configuration. Confirm the Terraform plan doesn't destroy any of these resources before running terraform apply.

    # View the existing terraform resource names for the exact value to use
HYPHENATED_CLUSTER_NAME=123-cluster-example-com
terraform state mv "aws_iam_openid_connect_provider.${HYPHENATED_CLUSTER_NAME}" "aws_iam_openid_connect_provider.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_internet_gateway.${HYPHENATED_CLUSTER_NAME}" "aws_internet_gateway.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_route_table.${HYPHENATED_CLUSTER_NAME}" "aws_route_table.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_vpc.${HYPHENATED_CLUSTER_NAME}" "aws_vpc.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_vpc_dhcp_options.${HYPHENATED_CLUSTER_NAME}" "aws_vpc_dhcp_options.prefix_${HYPHENATED_CLUSTER_NAME}"
terraform state mv "aws_vpc_dhcp_options_association.${HYPHENATED_CLUSTER_NAME}" "aws_vpc_dhcp_options_association.prefix_${HYPHENATED_CLUSTER_NAME}"

Deprecations

  • Support for Kubernetes version 1.17 is deprecated and will be removed in kOps 1.23.

  • Support for Kubernetes version 1.18 is deprecated and will be removed in kOps 1.24.

  • Support for the Lyft CNI is deprecated and will be removed in kOps 1.23.

  • Support for CentOS 7 is deprecated and will be removed in future versions of kOps.

  • Support for CentOS 8 is deprecated and will be removed in future versions of kOps.

  • Support for Debian 9 (Stretch) is deprecated and will be removed in future versions of kOps.

  • Support for RHEL 7 is deprecated and will be removed in future versions of kOps.

  • Support for Ubuntu 18.04 (Bionic) is deprecated and will be removed in future versions of kOps.

  • All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

  • The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in kOps 1.23.

  • The TerraformJSON feature flag is deprecated and will be removed in kOps 1.23. Only native HCL2 Terraform output will be supported.

  • Due to lack of maintainers, the Aliyun/Alibaba Cloud support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this cloud provider.

  • Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

Other changes of note

  • Support for shell completion has been substantially improved. kOps has added support for shell completion in fish and PowerShell.

  • It is no longer necessary to set AWS_SDK_LOAD_CONFIG=1 in the environment when using AWS assumed roles with the kops CLI.

  • There is a new command kops get assets for listing image and file assets used by a cluster.
    It also includes a --copy flag to copy the assets to local repositories.
    See the documentation on Using local asset repositories for more information.

  • kOps now provisions TLS server certificates signed by the Kubernetes general CA to kube-controller-manager and kube-scheduler.
    The previous behavior of using self-signed certs may be restored by setting kubeControllerManager.tlsCertFile and/or
    kubeScheduler.tlsCertFile to "" in the cluster spec.

  • Cilium now supports the wireguard protocol for transparent encryption.

1.22.4 to 1.22.5

Read more
Assets 26
Loading

v1.24.0-alpha.3

19 Mar 05:11
@hakman hakman
v1.24.0-alpha.3
e112d81
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.24.0-alpha.3 Pre-release
Pre-release

Release notes for kOps 1.24 series

⚠ kOps 1.24 has not been released yet! ⚠

This is a document to gather the release notes prior to the release.

Significant changes

Karpenter support

By enabling the Karpenter feature flag, users can now create InstanceGroups managed by (https://karpenter.sh)[Karpenter]:

spec:
  manager: Karpenter

You can also start a Karpenter-only cluster with kops create cluster --instance-manager=karpenter ...

kOps will directly manage the Karpenter Provisioner resources. Read more about how Karpenter works on kOps in the Karpenter docs.

Other significant changes

Breaking changes

Required actions

Deprecations

  • Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

  • Support for Kubernetes version 1.20 is deprecated and will be removed in kOps 1.26.

  • All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

  • The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in future versions of kOps.

  • Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

  • Support for Docker has been removed for Kubernetes 1.24+. See https://kubernetes.io/blog/2020/12/02/dockershim-faq

Other changes of note

Full change list since 1.24.0-alpha.2 release

Read more
Assets 26
Loading

v1.23.0

06 Mar 11:15
@hakman hakman
v1.23.0
a067cd7
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.23.0

Release notes for kOps 1.23 series

Significant changes

  • If the Kubernetes version is 1.23 or later and the external AWS Cloud Controller Manager is
    being used, then Kubernetes Node resources will be named after their AWS instance ID instead of their domain name and
    managed subnets will be configured to launch instances with Resource Based Names.

  • Support for ShutdownGracePeriod and ShutdownGracePeriodCriticalPods. By default, kOps will set ShutdownGracePeriod to 30 seconds and ShutdownGracePeriodCriticalPods to 10 seconds if the Kubernetes version is above 1.21.

  • By enabling the pod identity webhook, you no longer need to modify your Pod specs to assume IAM roles.

Breaking changes

  • Support for Kubernetes version 1.17 has been removed.

  • Support for the Lyft CNI has been removed.

  • The Weave CNI is not supported for Kubernetes 1.23 or later.

  • Support for CentOS 7 has been removed.

  • Support for CentOS 8 has been removed.

  • Support for Debian 9 has been removed.

  • Support for RHEL 7 is has been removed.

  • Support for Ubuntu 16.04 (Xenial) has been removed.

  • Support for Ubuntu 18.04 (Bionic) has been removed.

  • Cilium now has disable-cnp-status-updates: true by default. Set this to false if you rely on the CiliumNetworkPolicy status fields.

Required actions

Deprecations

  • Support for Kubernetes version 1.18 is deprecated and will be removed in kOps 1.24.

  • Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

  • All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

  • The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in future versions of kOps.

  • Due to lack of maintainers, the Aliyun/Alibaba Cloud support has been deprecated and will be removed in kOps 1.24.

  • Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

Other changes of note

  • The kops create cluster command has a new --discovery-store flag for specifying a public store for the OIDC-compatible discovery documents.
    If this flag is used in AWS, it will enable IRSA.

  • If externalDns.provider is external-dns, then externalDns.watchIngress will now default to true.

  • This release introduces a v1alpha3 API version. This API version is a work in progress and is likely to be replaced in kOps 1.24.
    It is recommended to keep using the v1alpha2 API version.

  • IPv6 pod subnets is in a working state using public IPv6 addresses for the Pod network. This works with both Cilium and Calico. IPv6 is still behind a feature flag until service controllers and addons implement support for IPv6. See the IPv6 documentation.

  • The kops rolling-update cluster command has a new --drain-timeout flag for specifying the maximum amount of time to wait when attempting to drain a node. Previously, rolling-updates would attempt to drain a node for an indefinite amount of time. If --drain-timeout is not specified, a default of 15 minutes is applied.

  • Fix inconsistent output of kops get clusters -ojson. This will now always return a list (irrespective of a single or multiple clusters) to keep the format consistent. However, note that kops get cluster dev.example.com -ojson will continue to work as previously, and will return a single object.

  • Digital Ocean kops now has vpc support. You can specify a network-cidr range while creating the kops cluster. kops resources will be created in the new vpc range. Also supports shared vpc; you can specify the vpc uuid while creating kops cluster.

Full change list since 1.23.0-beta.2 release

Assets 26
Loading