v0.7.1

[gardener-extension-provider-vsphere]

🐛 Bug Fixes

• [OPERATOR] An issue causing decoding of a Shoot resource to fail because of strict decoding is now fixed. (#148, @ialidzhikov)

v0.7.0

[gardener-extension-provider-vsphere]

⚠️ Breaking Changes

• [USER] Extension resource configs (InfrastructureConfig, ControlPlaneConfigs, WorkerConfig) are now deserialized in "strict" mode, including during validation by the admission webhook. This means that resources with fields that are not allowed by the API schema will be rejected by validation. Creating new shoots containing such resources will not be possible, and reconciling existing shoots will fail with an appropriate error until you manually update the shoot to make sure any extension resource configs contained in it are valid. (#141, @stoyanr)

✨ New Features

• [USER] The vSphere extension does now support shoot clusters with Kubernetes version 1.21. You should consider the Kubernetes release notes before upgrading to 1.21. (#143, @rfranzke)

🏃 Others

• [OPERATOR] The revisionHistoryLimit of different Deployments was increased. (#145, @MartinWeindel)
• [OPERATOR] update to golang v1.16.2 (#140, @MartinWeindel)
• [OPERATOR] set systemdisk size from pool volume size (#137, @MartinWeindel)
• [OPERATOR] update to vsphere-csi-driver v2.1.1 (#136, @MartinWeindel)
• [DEVELOPER] github.com/gardener/gardener dependency is now updated to v1.19.0. For the complete list of changes, see the release notes. (#138, @MartinWeindel)

[machine-controller-manager]

🐛 Bug Fixes

• [DEVELOPER] Azure: Improved NIC creation and deletion logic to handle NIC creation and deletions more gracefully. (gardener/machine-controller-manager#594, @prashanth26)

v0.6.0

[gardener-extension-provider-vsphere]

⚠️ Breaking Changes

• [OPERATOR] The ValidatingWebhookConfiguration of the vpshere admission controller has been changed from version v1beta1 to v1. Please make sure to deploy the admission controller only to clusters with a Kubernetes version >= 1.16 (#123, @timuthy)

🏃 Others

• [OPERATOR] The cloud-controller-manager VPA does now specify minAllowed values to prevent too low resource recommendations from VPA that lead to OOM. (#132, @ialidzhikov)
• [OPERATOR] Added a command line option --version to show version (#130, @MartinWeindel)
• [OPERATOR] Cloud provider config for the cloud controller manager is now stored in a Secret instead of a ConfigMap. (#124, @MartinWeindel)
• [DEVELOPER] Vendor gardener/gardener@570ae178874b. (#121, @vpnachev)

📰 Noteworthy

• [OPERATOR] The validator/admission component's Helm chart is now deploying a VerticalPodAutoscaler resource by default. If undesired or no VPA is available in the garden cluster then it can be turned of via .Values.global.vpa.enabled=false. (#126, @rfranzke)

[machine-controller-manager]

⚠️ Breaking Changes

• [DEVELOPER] machine-controller-manager now checks for misconfigured PodDisruptionBudgets (ones that require zero voluntary evictions and make impossible the graceful Node drain) and sets better Machine .status.lastOperation.description for such Machines. This change is breaking as out-of-tree providers need new RBAC permissions - list and watch access for PodDisruptionBudgets in the target cluster. (gardener/machine-controller-manager#591, @ialidzhikov)

🏃 Others

• [OPERATOR] Avoid the deletion of the machines in CrashLoopBackoff state by the safety controller (gardener/machine-controller-manager#589, @AxiomSamarth)

[machine-controller-manager-provider-vsphere]

🏃 Others

• [OPERATOR] updated machine-controller-manager dependency to v0.37.0 (gardener/machine-controller-manager-provider-vsphere#10, @MartinWeindel)

v0.5.0

[gardener-extension-provider-vsphere]

✨ New Features

• [USER] The vSphere extension may now support shoot clusters with Kubernetes version 1.20. You should consider the Kubernetes release notes before upgrading to 1.20. (#118, @rfranzke)

🏃 Others

• [USER] New dashboards which expose logs for cloud-controller-manager and csi-driver-controller. (#113, @Kristian-ZH)
• [OPERATOR] use patched csi-driver to fix detaching volume for non-existing node (#115, @MartinWeindel)
• [OPERATOR] A bug that was preventing the deletion of machines with outdated credentials is now fixed. (#112, @vpnachev)

[machine-controller-manager]

✨ New Features

• [OPERATOR] All machine classes do now support an optional .{spec.}credentialsSecretRef field in addition to today's .{spec.}secretRef field. If .{spec.}credentialsSecretRef is non-nil then the provider credentials will be read out of this secret. The user-data for the machine bring-up is still required to be part of the secret referenced by .{spec.}secretRef. (gardener/machine-controller-manager#578, @rfranzke)
• [OPERATOR] Some machine class secrets are now supporting alternative data keys: (gardener/machine-controller-manager#578, @rfranzke)
  • The machine class secret for Alicloud machines does now also accept the data keys accessKeyID and accessKeySecret as alternatives for today's keys.
  • The machine class secret for AWS machines does now also accept the data keys accessKeyID and secretAccessKey as alternatives for today's keys.
  • The machine class secret for Azure machines does now also accept the data keys clientID, clientSecret, subscriptionID and tenantID as alternatives for today's keys.
  • The machine class secret for GCP machines does now also accept the data key serviceaccount.json as alternatives for today's key.

🏃 Others

• [OPERATOR] Bumped AWS SDK version to v1.23.13 (gardener/machine-controller-manager#580, @zjj2wry)
• [OPERATOR] An issue causing panic when the encoded machine template hash length is less than expect limit is now fixed. (gardener/machine-controller-manager#575, @ialidzhikov)
• [OPERATOR] MCM will delete Azure machines even if the underlying resource group is already deleted. (gardener/machine-controller-manager#566, @dkistner)
• [OPERATOR] Set Machine Phase to Terminating before draining. (gardener/machine-controller-manager#564, @prashanth26)
• [OPERATOR] An issue causing panic when the encoded machine template hash length is less than expect limit is now fixed. (gardener/machine-controller-manager#577, @AxiomSamarth)
• [DEVELOPER] Update docker images to use gcr copy (gardener/machine-controller-manager#574, @prashanth26)
• [DEVELOPER] Update docker image versions to golang:1.15.5 & alpine:3.12.1 (gardener/machine-controller-manager#574, @prashanth26)

📰 Noteworthy

• [OPERATOR] Machine force deletion computation is based on deletionTimestamp instead of LastUpdatedTimestamp. (gardener/machine-controller-manager#564, @prashanth26)

v0.4.1

[gardener-extension-provider-vsphere]

Improvements

• [OPERATOR] Adding missing role for machine-controller-manager for shoot (#110, @MartinWeindel)

v0.4.0

[gardener-extension-provider-vsphere]

Improvements

• [OPERATOR] updated dependencies: gardener to v1.13-dev, mcm to v0.35.0 (#106, @MartinWeindel)
• [OPERATOR] shoot default storageclass: enable allowVolumeExpansion if resizer is available (#105, @MartinWeindel)

[machine-controller-manager]

Most notable changes

• [USER] NetworkUnavailable node condition is also considered by default while considering the machine's to be unhealthy. (gardener/machine-controller-manager#543, @rewiko)
• [USER] AWS: Allows deletion of machines even on modify instance call failure (gardener/machine-controller-manager#515, @prashanth26)
• [OPERATOR] Machine force deletion computation is based on deletionTimestamp instead of LastUpdatedTimestamp. (gardener/machine-controller-manager#564, @prashanth26)
• [OPERATOR] OOT: Introduced a backoff in re-enqueuing machines on creation/deletion failures. Avoids throttling APIServer & provider calls. (gardener/machine-controller-manager#557, @prashanth26)
• [OPERATOR] Introduced a backoff in re-enqueuing machines on creation/deletion failures. Avoids throttling APIServer & provider calls. (gardener/machine-controller-manager#525, @hardikdr)

Improvements

• [USER] The default drainTimeout value has been updated from 12hours to 2hours. (gardener/machine-controller-manager#554, @prashanth26)
• [USER] OOT: Fixed regression with maxEvictRetries (gardener/machine-controller-manager#554, @prashanth26)
• [USER] Adds the ability to specify an already existing OpenStack Neutron network in the subnetID of an OpenStackMachineClass. MCM will deploy new machines into the given subnet by pre-allocating Neutron ports and pass them to the Nova server object. (gardener/machine-controller-manager#545, @MrBatschner)
• [USER] The machine-controller-manager supports now machines attached to Azure VirtualMachineScaleSet Orchestration Mode VM (VMO). (gardener/machine-controller-manager#519, @dkistner)
• [USER] Restored tag verification in the Azure driver to filter VMs/disks/NICs based on tags (gardener/machine-controller-manager#507, @zuzzas)
• [OPERATOR] Set Machine Phase to Terminating before draining. (gardener/machine-controller-manager#564, @prashanth26)
• [OPERATOR] Allow migration to continue when ProviderMachineClass is missing but MachineClass with the same name as ProviderMachineClass is found. Updates Machine object references to the MachineClass. (gardener/machine-controller-manager#559, @prashanth26)
• [OPERATOR] Use cache-based listers to GET the machine-object while reconciling. (gardener/machine-controller-manager#558, @hardikdr)
• [OPERATOR] OOT: Enqueue machine only when node conditions have changed. (gardener/machine-controller-manager#557, @prashanth26)
• [OPERATOR] Adapted integration tests to handle possibly orphaned resources. (gardener/machine-controller-manager#550, @hardikdr)
• [OPERATOR] OOT: Fixes drain timeout issues on retires (gardener/machine-controller-manager#548, @prashanth26)
• [OPERATOR] NetworkUnavailable nodeCondition added to the example, some CNI will update this condition depending on the state of the CNI or the network availability. (gardener/machine-controller-manager#543, @rewiko)
• [OPERATOR] Added a more comprehensive set of events to trigger machine class reconciliations. (gardener/machine-controller-manager#531, @prashanth26)
• [OPERATOR] Finalizers are added by default for all machine class objects. (gardener/machine-controller-manager#531, @prashanth26)
• [OPERATOR] Bootstrap token injection now works in the new OOT Machine controller (gardener/machine-controller-manager#521, @zuzzas)
• [OPERATOR] Add support for ServerGroups in the Openstack driver. VMs can now be created in the ServerGroup specified in the respective MachineClass. (gardener/machine-controller-manager#511, @kon-angelo)
• [OPERATOR] Bugfix: Consider CSI PersistentVolumes during the eviction of Pods with PersistentVolumes. (gardener/machine-controller-manager#509, @ialidzhikov)
• [DEVELOPER] Adds a new phase CrashLoopBackOff that is set due to machine creation failures. (gardener/machine-controller-manager#525, @hardikdr)
• [DEVELOPER] The field availabilitySets in the AzureMachineClass is now deprecated in favour of the field machineSet, which allow to configure AvailabilitySets and VirtualMachineScaleSet Orchestration Mode VM (VMO). The field will be removed in the future. (gardener/machine-controller-manager#519, @dkistner)

[machine-controller-manager-provider-vsphere]

Improvements

• [OPERATOR] updated machine-controller-manager dependency to v0.35.0 (gardener/machine-controller-manager-provider-vsphere#8, @MartinWeindel)

v0.3.0

[gardener-extension-provider-vsphere]

Improvements

• [OPERATOR] add optional DockerDaemonOptions to cloudprofile config for HTTP proxy and insecure registries (#95, @MartinWeindel)
• [OPERATOR] add optional DHCPOptions to cloudprofile config for specifying search domain, NTP, etc. (#95, @MartinWeindel)
• [DEVELOPER] An issue causing make test to fail on macOS is now fixed. (#99, @ialidzhikov)

[machine-controller-manager-provider-vsphere]

Improvements

• [USER] Bugfix: Consider CSI PersistentVolumes during the eviction of Pods with PersistentVolumes. (gardener/machine-controller-manager-provider-vsphere#5, @MartinWeindel)
• [OPERATOR] fixed rare nil pointer deference in GetMachineStatus (gardener/machine-controller-manager-provider-vsphere#7, @MartinWeindel)
• [OPERATOR] updated dependency machine-controller-manager to v.0.34.3 (gardener/machine-controller-manager-provider-vsphere#6, @MartinWeindel)

v0.2.0

[gardener-extension-provider-vsphere]

Improvements

• [OPERATOR] update vsphere-csi-driver from v2.0.0 to v2.0.1 (#94, @MartinWeindel)
• [OPERATOR] Adds priority class for extension pods to prevent preemption. (#93, @danielfoehrKn)
• [OPERATOR] updated dependency cloud-provider-vsphere to v1.2.1 (#90, @MartinWeindel)
• [OPERATOR] LB cleanup in infrastructure controller now also deletes an orphan NSXT-T LB service without any virtual servers. (#85, @MartinWeindel)
• [DEVELOPER] A new integration test for infrastructure creation and deletion has been added. (#92, @MartinWeindel)

[machine-controller-manager]

Most notable changes

• [OPERATOR] Introduced a backoff in re-enqueuing machines on creation/deletion failures. Avoids throttling APIServer & provider calls. (gardener/machine-controller-manager#523, @hardikdr)

Improvements

• [OPERATOR] Added a more comprehensive set of events to trigger machine class reconciliations. (gardener/machine-controller-manager#532, @prashanth26)
• [OPERATOR] Finalizers are added by default for all machine class objects. (gardener/machine-controller-manager#532, @prashanth26)
• [OPERATOR] AWS: Allow deletion of VMs even on list image or modify instance failure (gardener/machine-controller-manager#516, @prashanth26)
• [DEVELOPER] Adds a new phase CrashLoopBackOff that is set due to machine creation failures. (gardener/machine-controller-manager#523, @hardikdr)

v0.1.0

[gardener-extension-provider-vsphere]

Action Required

• [OPERATOR] ⚠️ This version is not compatible with Gardener < v1.2. (#14, @MartinWeindel)
  • The controlplanebackup webhook is removed as of gardener/gardener#1762. The controlplaneexposure webhook that was previously targeting StatefulSets is now targeting Etcd resources of the druid.
• [OPERATOR] Shoots created with old versions of this provider-vsphere cannot be handled because of incompatible changes in the NSX-T object model and must be deleted before upgrading to this version (#12, @MartinWeindel)

Most notable changes

• [OPERATOR] The logging configuration has been adapted to cater with Loki which was introduced as part of gardener/gardener#2515. If an older version of this extension is used with a Gardener version >= v1.8 and the Logging feature gate is enabled then the fluent-bit matcher won't catch the log anymore (resulting in the logs being collected, but not nicely parsed and instead displayed in nested JSON). It is recommended update to at least this extension version when Gardener >= v1.8 is used and the Logging feature gate is enabled. (#57, @Kristian-ZH)
• [OPERATOR] The gardener-extension-provider-vsphere now offers a validating webhook which checks shoot resources of type vsphere in the core.gardener.cloud group. Operators should register this webhook in the Garden cluster to further prevent invalid modifications on vsphere shoots. (#17, @MartinWeindel)
• [OPERATOR] The network infrastructure is created with the new "Simplified API" of NSX-T. (#12, @MartinWeindel)
• [OPERATOR] Initial contribution of vSphere extension controller for Gardener (originally contributed by @MartinWeindel via gardener/gardener-extensions#537) (959152f)

Improvements

• [USER] Do not mutate kube-apiserver exposure resources which Gardener marks as managed by it with core.gardener.cloud/apiserver-exposure: gardener-managed label. (#71, @MartinWeindel)
• [USER] set external Tier-1 gateway and load balancer (#62, @MartinWeindel)
• [USER] Allow to use infrastructure state file in YAML for the infra-cli (#38, @MartinWeindel)
• [USER] Allow to delete infrastructure if cluster creation was tried with invalid credentials (#27, @MartinWeindel)
• [OPERATOR] authorize shoot specific IP pools and T1 gateway by authorized-shoots tag (#76, @MartinWeindel)
• [OPERATOR] restrict clusterID used for CSI to 63 characters (#74, @MartinWeindel)
• [OPERATOR] The StorageClasses in the shoot cluster managed by Gardener are now re-created in case the update request failed due to changed immutable fields. (#73, @vpnachev)
• [OPERATOR] The default webhook server port is now changed to 10250. (#70, @stoyanr)
• [OPERATOR] Worker extension allows configuring following parameters on machine-deployment: drainTimeout, creationTimeout, healthTimeout, maxEvictRetries, nodeConditions. (#69, @hardikdr)
• [OPERATOR] downgrading vsphere-csi-driver to v1.0.2 to resolve PVC detachment issue (#64, @MartinWeindel)
• [OPERATOR] clean up possible orphan load balancer objects during deleting the infrastructure (#58, @MartinWeindel)
• [OPERATOR] allow to specify additional VM options memoryReservationLockedToMax and extraConfig for a machineType with the machineTypeOptions in the CloudProfileConfig (#54, @MartinWeindel)
• [OPERATOR] updated vsphere-csi-driver to v2.0.0 (#52, @MartinWeindel)
• [OPERATOR] support for remote authentication with VMware Identity Manager (#48, @MartinWeindel)
• [OPERATOR] support separate vSphere/NSX-T users for components (#46, @MartinWeindel)
• [OPERATOR] The pre-defined ValidatingWebhookConfiguration assets now set sideEffects=None. This enables dry-run related operations, like kubectl diff, for core.gardener.cloud resources. (#45, @timuthy)
• [OPERATOR] Fixed a bug in the healthcheck library that prevents checks after a Shoot has been woken up from hibernation. Gardener extensions require a minor change during the healthcheck registration. (#43, @MartinWeindel)
• [OPERATOR] infra-cli command 'destroy-loadbalancers': support NSX-T connection via proxy (#42, @MartinWeindel)
• [OPERATOR] kube-controller-manageer no longer has access to blocked CIDRs. (#39, @mvladev)
• [OPERATOR] Support for new DHCP server in NSX-T 3.0 (vSphere 7.0) (#36, @MartinWeindel)
• [OPERATOR] The vSphere provider now allows Gardener to restart the Cloud-Controller-Manager and the Machine-Controller-Manager (see gardener/gardener#2098 for more information). (#29, @timuthy)
• [OPERATOR] The vSphere shoot validator now checks that workers[].minimum != 0 if workers[].maximum >0 since autoscaling does not support this setup yet. (#23, @MartinWeindel)
• [OPERATOR] fix kube-apiserver svc after hibernation (#2, @MartinWeindel)
• [OPERATOR] allow to override load balancer size in controlplaneconfig (#2, @MartinWeindel)
• [OPERATOR] add random suffix to logical switch name to avoid name clash if cleanup fails in vsphere (#2, @MartinWeindel)

[gardener]

Improvements

• [USER] The generic Worker actuator does now exit its reconciliation flows early if it detects an error during the machine reconciliation. This allows to faster propagate problems to the end-user. (gardener/gardener#2348, @rfranzke)
• [USER] The extension health check library does now allow individual health checks to return the Progressing status. This allows to provide more accurate status information and less false negative health reports. (gardener/gardener#2289, @rfranzke)
• [OPERATOR] The Gardenlet sets an additional annotation to Extension CRDs during reconciliation to guarantee an update event for the watching clients. (gardener/gardener#2290, @danielfoehrKn)
• [OPERATOR] Fixes a bug in the extension libraries that could lead to duplicate reconciliation of extension resources. When respecting the operation annotation set by the Gardenlet during reconciliation, extension controllers now only watch the Extension CRD. (gardener/gardener#2290, @danielfoehrKn)
• [OPERATOR] Fixes a bug in the extension library of all extension resources that lead to not stopping the reconciliation of extension resources when the Shoot is in 'failed' state (Shoot.Status.lastOperation.state = Failed). (gardener/gardener#2279, @danielfoehrKn)
• [DEVELOPER] The ShootNotFailed predicate in the extensions library does now work as expected. (gardener/gardener#2265, @rfranzke)

[gardener-extensions]

Improvements

• [USER] An issue in the machine reconciliation has been fixed that caused the reconciliation being stuck in some cases where a previously broken worker configuration (e.g., due to the specification of machine types that are not available in certain availability zones) has been corrected. (gardener-attic/gardener-extensions#626, @rfranzke)
• [USER] Allow force deletion of machines incase of cluster hibernation (gardener-attic/gardener-extensions#609, @prashanth26)
• [OPERATOR] Add Separate sidecar controller to save worker state under Worker.Status.State (gardener-attic/gardener-extensions#600, @vlvasilev)
  • Add Worker.Status.State restoration functionality
  • Add migration functionality
• [OPERATOR] The machine.sapcloud.io CRDs applied by the ./pkg/controller/worker package do now get a protection label for accidental deletion. See also: gardener/gardener#2066 (gardener-attic/gardener-extensions#625, @rfranzke)
• [OPERATOR] Credentials used by machine-controller-manager are now updated during worker deletion. (gardener-attic/gardener-extensions#606, @EmoinLanyu)

[machine-controller-manager]

Most notable changes

• [USER] Support for Spot Instances is available in AWS driver. If the spotPrice is empty, price is automatically set to the on-demand price so that Spot instance can launch immediately. (gardener/machine-controller-manager#481, @zuzzas)
• [OPERATOR] RBAC policies have to be updated to allow updating of node/status resources. (gardener/machine-controller-manager#492, @guydaichs)
• [OPERATOR] New flag delete-migrated-machine-class is introduced. When set to true (defaulted to false), deletes any provider-specific machine class (e.g. AWSMachineClass) that has the machine.sapcloud.io/migrated annotation set on it. (gardener/machine-controller-manager#484, @prashanth26)
• [DEVELOPER] Added migration logic for moving from provider-specific machine class to generic machine classes in out of tree code path. On migration, the machine.sapcloud.io/migrated annotation set on the old machine class. (gardener/machine-controller-manager#484, @prashanth26)
• [DEVELOPER] The machine controller adds finalizer only when machine reference is present, deletes it otherwise. (gardener/machine-controller-manager#484, @prashanth26)

Improvements

• [USER] Retry when secret is referred by machineClass is missing (gardener/machine-controller-manager#495, @AxiomSamarth)
• [USER] Node condition is added to the status of terminating nodes indicating the termination start time and reason (Unhealthy|ScaleDown) (gardener/machine-controller-manager#492, @guydaichs)
• [OPERATOR] All nodes under machine deployments being rolled-out are annotated with cluster-autoscaler.kubernetes.io/scale-down-disabled: "True" during the period of rolling-update. (gardener/machine-controller-manager#496, @hardikdr)
• [OPERATOR] A new command line flag autoscaler-scaldown-annotation-during-rollout is introduced to disable annotating the nodes with cluster-autoscaler annotation cluster-autoscaler.kubernetes.io/scale-down-disabled during rollout. (gardener/machine-controller-manager#496, @hardikdr)
• [OPERATOR] Avoids race between secret finalizer creation/deletion between in-tree & OOT controllers (gardener/machine-controll...