In-place update support in CAPBK

[sm4ll-3gg]

What would you like to be added (User Story)?

As a runtime hooks developer want to be able to "re-bootstrap" the node during in-place update

Detailed Description

Relates to #13497. The current issue describes the same case from the bootstrap provider pov.

Context
We use machine images for server provisioning and we strive to stick to the immutable infrastructure approach wherever possible. During the last setup phase, in cloud-init, a bootstrap token is used to generate certificates that are used by kubelet to authorize with kube-apiserver. So, in order to be able to update a server via resetup, we prefer to re-bootstrap the node. All we need is a fresh bootstrap token and, probably, updated cloud-init to be able to change most of KubeadmConfig params

Problems

• Bootstrap token. CAPBK stops refreshing it once the node appears in the cluster. So, it's most likely expired when the update starts
• Data secret. CAPBK renders KubeadmConfig into cloud-init only once, and it will never be updated without dirty manual intervention
• Kubelet handles join of the node with the same name correctly, but it updates only subset of fields, so it won't be possible to update params that affect the node's manifest (as far as I understand). Code: https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kubelet_node_status.go#L85-L142

Suggestion
I believe the following changes would be useful for CAPBK:

• Regenerate data secret if the KubeadmConfig is updated during in-place update
• Refresh the bootstrap token until in-place update is successfully completed

Regarding node re-registration by kubelet, I think that it could be left to the hooks developers to decide what fields can be updated in-place. I'm not sure this is the best solution, but I guess it won't make things worse and can be reassessed in the future

Anything else you would like to add?

No response

Label(s) to be applied

/kind feature
/area bootstrap
/area machine
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.

[k8s-ci-robot]

This issue is currently awaiting triage.

If CAPI contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[fabriziopandini]

Personally I'm -1 to this request

From what I can see, it looks like you are trying you are trying to re-use the machine infrastructure here, and this is not what in-place feature has been designed for.

Also, the community already explored more elegant ways to re-use the same infrastructure (e.g the BYOH provider)