From abe580f7de5de70f6b3d4d63cbbe5dfd58546792 Mon Sep 17 00:00:00 2001 From: Hannes Baum Date: Thu, 7 Dec 2023 14:41:19 +0100 Subject: [PATCH] fixup! K8s cluster robustness features (#414) Signed-off-by: Hannes Baum --- Standards/scs-0215-v1-robustness-features.md | 41 ++++++++------------ 1 file changed, 16 insertions(+), 25 deletions(-) diff --git a/Standards/scs-0215-v1-robustness-features.md b/Standards/scs-0215-v1-robustness-features.md index 75172d8dc..2179997b7 100644 --- a/Standards/scs-0215-v1-robustness-features.md +++ b/Standards/scs-0215-v1-robustness-features.md @@ -97,7 +97,7 @@ To mitigate this scenario, the etcd keyspace can be compacted. Additionally, an gives back disk space to the underlying file system and can help bring the cluster back into an operable state, if it ran out of space earlier. -This can be achieved by providing the necessary flags/parameters to etcd, either via the KubeadmControlPlane or in the +etcd keyspace maintenance can be achieved by providing the necessary flags/parameters to etcd, either via the KubeadmControlPlane or in the configuration file of the etcd cluster, if it is managed independent of the Kubernetes cluster. Possible flags, that can be set for this feature, are: @@ -118,7 +118,7 @@ meaning that the previous year should only have 1 backup, but the current week s Information about the backup process can be found in the etcd documentation: [Upgrade etcd](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/) -### CA expiration avoidance +### Certificate rotation In order to secure the communication of a Kubernetes cluster, (TLS) certificates signed by a controlled Certificate Authority (CA) can be used. @@ -130,8 +130,9 @@ kubelet server certificate. Another recommendation is to set `serverTLSBootstrap and rotation of the certificate for the kubelet according to the documentation. A clusters certificates can either be rotated by updating the cluster, which according to the Kubernetes documentation -automatically renews the certificates, or by running the kubeadm certs renew command for the certificates that need to -updated, which can be seen in the following inline code +automatically renews the certificates. +Another method would be the manual renewal, which can be done through multiple methods, depending on the K8s cluster +used. An example for a default K8s cluster would be to execute the command ```bash kubeadm certs renew all @@ -140,7 +141,7 @@ kubeadm certs renew all Since clusters conformant with the SCS standards would probably be updated within the time period described in the standard [SCS-0210-v2](https://github.com/SovereignCloudStack/standards/tree/main/Standards/scs-0210-v2-k8s-version-policy.md), this rotation can probably be assumed to happen. Nevertheless, the alternative can still be mentioned in the standard. -Additionally, the Certificate Signing Request (CSR) needs to be approved manually due to security reasons with the commands +Additionally, the Certificate Signing Request (CSR) need to be approved manually due to security reasons with the commands ```bash kubectl get csr @@ -152,8 +153,8 @@ this would be the [Kubelet CSR approver](https://github.com/postfinance/kubelet- a K8s cluster and requires `serverTLSBootstrap` to be set to true. Other controllers with a similar functionality might have other specific requirements, which won't be explored in this document. -Another problem is that the CA might expire. Unfortunately, kubeadm doesn't have any tooling -at the moment to renew the CA. Instead, there is documentation for manually rotating the CA, which can be found +Another problem is that the CA might expire. Unfortunately, not all K8s tools have functionality to renew these +certificates with the help of commands. Instead, there is documentation for manually rotating the CA, which can be found under [Manual rotation of ca certificate](https://kubernetes.io/docs/tasks/tls/manual-rotation-of-ca-certificates/). Further information can be found in the Kubernetes documentation: @@ -168,16 +169,16 @@ areas. ### Kube-API rate limiting -The number of requests send to the kube-api or Kubernetes API server SHOULD be limited +The number of requests send to the kube-api or Kubernetes API server MUST be limited in order to protect the server against outages, deceleration or malfunctions due to an overload of requests. -In order to do so, at least the following parameters SHOULD be set on a Kubernetes cluster: +In order to do so, at least the following parameters MUST be set on a Kubernetes cluster: * max-requests-inflight * max-mutating-requests-inflight * min-request-timeout -Values for these flags/parameters MUST be adapted to the needs of the environment and +Values for these flags/parameters SHOULD be adapted to the needs of the environment and the expected load. A cluster MUST also activate and configure a Ratelimit admission controller. @@ -219,28 +220,18 @@ These numbers can be adapted to the security setup and concerns like storage or usage. It is also RECOMMENDED to encrypt the backups in order to secure them further. How this is done is up to the operator. -### CA expiration avoidance +### Certificate rotation It should be avoided, that certificates expire either on the whole cluster or for single components. To avoid this scenario, certificates SHOULD be rotated regularly; in the case of SCS, we REQUIRE at least a yearly certificate rotation. -To achieve a complete certificate rotation, the parameters `serverTLSBootstrap` and `rotateCertificates` MUST be set. +To achieve a complete certificate rotation, the parameters `serverTLSBootstrap` and `rotateCertificates` +MUST be set in the kubelet configuration. The certificates can be rotated by either updating the Kubernetes cluster, which automatically -renews certificates, or by manually renewing them with the command +renews certificates, or by manually renewing them. How this is done is dependent on the used K8s cluster. -```bash -kubeadm certs renew all -``` - -After this, new CSRs MUST be approved manually, normally done with - -```bash -kubectl get csr -kubectl certificate approve -``` - -or be approved with a third-party controller, e.g. the [kubelet-csr-approver](https://github.com/postfinance/kubelet-csr-approver). +After this, new CSRs MUST be approved manually or with a third-party controller, e.g. the [kubelet-csr-approver](https://github.com/postfinance/kubelet-csr-approver). It is also RECOMMENDED to renew the CA regularly to avoid an expiration of the CA. This standard doesn't set a timeline for this, since it is dependent on the CA.