You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CAPH reconciles each HetznerBareMetalHost by reading its credentials fresh from Kubernetes secrets on every pass. Those credentials come in two kinds, and both live in secrets that belong to the cluster or the machine, not to the host.
The Robot API user and password, in the Hetzner secret the cluster points to (HetznerCluster.spec.hetznerSecretRef), which also holds the hcloud token.
The SSH keys, a cluster-wide rescue key (HetznerCluster.spec.sshKeys.robotRescueSecretRef) and a per-machine key for the installed OS (HetznerBareMetalMachine.spec.sshSpec.secretRef, copied onto the host).
When a secret holds a wrong value, the two credential kinds fail differently:
Wrong robot credentials stop the host. The Robot API answers unauthorized, so the controller marks RobotCredentialsAvailable false and returns without asking for a retry. Missing or empty credentials end the same way. The failure is not permanent, so any later reconcile would recover the host. But once the credentials are corrected, nothing triggers that reconcile, and the host stays stuck.
A wrong SSH secret only slows the host. A missing one makes the host requeue every five minutes. A rotated key is then picked up on the next pass, late rather than never.
Note
The host controller's error path already assumes this watch exists. It returns with a comment that the host "will be triggered as soon as the secret is updated", but no such watch is registered today.
The Solution
Give the host controller a watch on secrets, so a corrected secret takes effect at once instead of leaving the host stuck or picking the fix up late. PR #2001 did this for HCloudMachines and the hcloud token in issue #1972.
When a secret changes, reconcile every host that reads it. That is every host of a HetznerCluster whose hetznerSecretRef or robotRescueSecretRef names the secret, plus any host whose own OS SSH secretRef names it.
The host controller is the only place that reads these credentials and calls the Robot API, so it is the only controller that needs the watch.
The Problem
CAPH reconciles each HetznerBareMetalHost by reading its credentials fresh from Kubernetes secrets on every pass. Those credentials come in two kinds, and both live in secrets that belong to the cluster or the machine, not to the host.
HetznerCluster.spec.hetznerSecretRef), which also holds the hcloud token.HetznerCluster.spec.sshKeys.robotRescueSecretRef) and a per-machine key for the installed OS (HetznerBareMetalMachine.spec.sshSpec.secretRef, copied onto the host).When a secret holds a wrong value, the two credential kinds fail differently:
RobotCredentialsAvailablefalse and returns without asking for a retry. Missing or empty credentials end the same way. The failure is not permanent, so any later reconcile would recover the host. But once the credentials are corrected, nothing triggers that reconcile, and the host stays stuck.Note
The host controller's error path already assumes this watch exists. It returns with a comment that the host "will be triggered as soon as the secret is updated", but no such watch is registered today.
The Solution
Give the host controller a watch on secrets, so a corrected secret takes effect at once instead of leaving the host stuck or picking the fix up late. PR #2001 did this for HCloudMachines and the hcloud token in issue #1972.
IgnoreInsignificantSecretUpdates, which ignores metadata-only updates and fires only when a secret's data changes.hetznerSecretReforrobotRescueSecretRefnames the secret, plus any host whose own OS SSHsecretRefnames it.The host controller is the only place that reads these credentials and calls the Robot API, so it is the only controller that needs the watch.