Skip to content

Commit 8e4cd93

Browse files
committed
kms: refine proposal
Signed-off-by: Damien Grisonnet <[email protected]>
1 parent ecf7cbb commit 8e4cd93

File tree

2 files changed

+113
-77
lines changed

2 files changed

+113
-77
lines changed
143 KB
Loading

enhancements/kube-apiserver/kms-encryption-provider.md

Lines changed: 113 additions & 77 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,12 @@ reviewers:
99
- "@rvanderp"
1010
approvers:
1111
- "@tkashem"
12+
- "@deads2k"
13+
- "@derekwaynecarr"
1214
api-approvers:
1315
- "@JoelSpeed"
1416
creation-date: 2024-08-14
15-
last-updated: 2024-08-14
17+
last-updated: 2024-11-21
1618
status: implementable
1719
see-also:
1820
- "/enhancements/kube-apiserver/encrypting-data-at-datastore-layer.md"
@@ -31,9 +33,9 @@ Provide a user-configurable interface to support encryption of data stored in et
3133

3234
## Motivation
3335

34-
Today, we support local AES encryption at the datastore layer which protects against etcd data leaks in the event of a etcd backup compromise. However, aescbc and aesgcm which are supported ecncryption technologies today available in OpenShift do not protect against online host compromise i.e. in such cases attacker can decrypt encrypted data from etcd using local keys, KMS managed keys protects against such scenarios.
36+
Today, we support local AES encryption at the datastore layer. It protects against etcd data leaks in the event of a etcd backup compromise. However, aescbc and aesgcm, which are supported encryption technologies today available in OpenShift do not protect against online host compromise i.e. in such cases, attackers can decrypt encrypted data from etcd using local keys, KMS managed keys protects against such scenarios.
3537

36-
Users of OpenShift would like the encrypt secret data in etcd using self-managed KMS backed keys.
38+
Users of OpenShift would like to encrypt secret data in etcd using self-managed KMS-backed keys.
3739
- https://issues.redhat.com/browse/OCPSTRAT-108
3840

3941
### User Stories
@@ -43,8 +45,8 @@ As an OpenShift administrator, I want to encrypt secrets in my cluster at rest u
4345
As an OpenShift administrator, I want to let the KMS provider manage the lifecycle of the encryption keys.
4446

4547
As an OpenShift user, I enable encryption by setting apiserver.spec.encryption.type to kms.
46-
- After some time passes, user makes a backup of etcd.
47-
- The user confirms that the secret values are encrypted by checking to see if they have the related kms prefix.
48+
- After some time passes, a user makes a backup of etcd.
49+
- The user confirms that the secret values are encrypted by checking to see if they have the related KMS prefix.
4850

4951
### Goals
5052

@@ -58,13 +60,48 @@ As an OpenShift user, I enable encryption by setting apiserver.spec.encryption.t
5860
### Non-Goals
5961

6062
1. Allow the user to force key rotation
61-
2. Support for the complete lifecyle of KMS managed keys directly within OpenShift control plane
63+
2. Support for the complete lifecycle of KMS-managed keys directly within OpenShift control-plane
6264
3. Support for hardware security modules
63-
4. The user has in-depth understanding of each phase of the encryption process
65+
4. The user has an in-depth understanding of each phase of the encryption process
6466
5. Completely recover the cluster in the event of the KMS instance itself going down or keys getting lost
6567
6. Allow users to configure which resources will be encrypted
6668

67-
### Proposal
69+
## Proposal
70+
71+
To support KMS encryption in OpenShift, we will be able to leverage the work that was done in [upstream Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/). However, we will need to extend and adapt the encryption workflow in OpenShift to support new constraints introduced by the externalization of encryption keys in a KMS. Because OpenShift will not own the keys from the KMS, we will also need to provide tools to the users to detect KMS-related failures and take action toward recovering their clusters whenever possible.
72+
73+
There exist two versions of the KMS API upstream today. In OpenShift, we specifically want to use KMS v2 only as it is an optimized version of the first API that is more production-ready than its predecessor, which was infamous for its impact on performance and the pressure it puts on the KMS.
74+
75+
In OpenShift, we will add a new encryption type to the list in the apiserver API as `kms`, similar to the list of possible encryption providers from the apiserver's [EncryptionConfig](https://github.com/kubernetes/apiserver/blob/cccad306d649184bf2a0e319ba830c53f65c445c/pkg/apis/apiserver/types_encryption.go#L89-L101).
76+
77+
Unlike the other providers, KMS will require additional configuration from the users. Because of that, we will add a new configuration to the API to allow users to configure the connection to their KMS. In this new API, users will be able to configure some options specific to the KMS provider they want to use. There will be dedicated APIs for each KMS supported by OpenShift.
78+
79+
From a UX perspective, these are the only changes the KMS feature will introduce. It is intentionally minimal to reduce the burden on the users and the potential for errors.
80+
81+
In practice, this feature will re-use as much of the existing encryption logic as possible. It will leverage the existing encryption and migration workflow introduced for AES-CBC and AES-GCM. However, unlike the aescbc and aesgcm providers, the encryption keys for KMS are not managed by the apiserver operators, so we have to extend the existing controllers to support that new workflow. On top of that, the operators used to be in charge of rotating the keys, but it will now be in the hands of the users to rotate the keys. We will need to introduce a new workflow to react when keys are rotated in the external KMS to make sure that the encrypted data is migrated to use the new key.
82+
83+
### KMS plugins
84+
85+
One aspect of the upstream feature that wasn't mentioned yet is that it requires a third-party application called a KMS plugin to bridge between the apiservers and the KMS. In OpenShift, these plugins will be configured and managed by the kube-apiserver-operator. There are multiple reasons behind this choice:
86+
87+
1. Reduces the complexity for the users that want to use the KMS feature
88+
2. Simplifies key rotation when users manually rotate the key because it requires creating a second instance of the plugin that would use the new key while the old plugin would still allow using the old key as a read key
89+
3. It is cheap to maintain the plugins as we can either leverage the upstream communities or the vendors
90+
4. Have more trust and guarantees towards the plugins that will be running in the platform
91+
92+
Plugins available in the open will be forked and maintained downstream. The images for these plugins will be published and distributed on the official Red Hat registry in the same way as they are today for HyperShift.
93+
Existing forks include:
94+
95+
* https://github.com/openshift/aws-encryption-provider/
96+
* https://github.com/openshift/azure-kubernetes-kms/
97+
98+
For the plugins we can't distribute, an `image` field will be available in the relevant KMS API to allow users to configure the plugin.
99+
100+
In the future, we will also be able to think about ways to qualify new plugins to be distributed and supported by OCP.
101+
102+
### API Extensions
103+
104+
During the tech-preview support of the KMS feature, it will be placed behind the `KMSEncryptionProvider` feature-gate.
68105

69106
OpenShift would need to align closer with KMS evolution upstream with respect to the different Kubernetes Encryption Providers available today.
70107

@@ -75,82 +112,88 @@ diff --git a/config/v1/types_apiserver.go b/config/v1/types_apiserver.go
75112
index d815556d2..c9098024f 100644
76113
--- a/config/v1/types_apiserver.go
77114
+++ b/config/v1/types_apiserver.go
78-
@@ -173,6 +173,9 @@ type APIServerNamedServingCert struct {
79-
ServingCertificate SecretNameReference `json:"servingCertificate"`
80-
}
81-
82-
+// APIServerEncryption is used to encrypt sensitive resources on the cluster.
83-
+// +openshift:validation:FeatureGateAwareXValidation:featureGate=KMSEncryptionProvider,rule="has(self.type) && self.type == 'KMS' ? has(self.kms) : !has(self.kms)",message="kms config is required when encryption type is KMS, and forbidden otherwise"
84-
+// +union
85-
type APIServerEncryption struct {
86-
// type defines what encryption type should be used to encrypt resources at the datastore layer.
87-
// When this field is unset (i.e. when it is set to the empty string), identity is implied.
88-
@@ -191,9 +194,23 @@ type APIServerEncryption struct {
89-
// +unionDiscriminator
90-
// +optional
91-
Type EncryptionType `json:"type,omitempty"`
92-
+
93-
+ // kms defines the configuration for the external KMS instance that manages the encryption keys,
94-
+ // when KMS encryption is enabled sensitive resources will be encrypted using keys managed by an
95-
+ // externally configured KMS instance.
96-
+ //
97-
+ // The Key Management Service (KMS) instance provides symmetric encryption and is responsible for
98-
+ // managing the lifecyle of the encryption keys outside of the control plane.
99-
+ // This allows integration with an external provider to manage the data encryption keys securely.
100-
+ //
101-
+ // +openshift:enable:FeatureGate=KMSEncryptionProvider
102-
+ // +unionMember
103-
+ // +optional
104-
+ KMS *KMSConfig `json:"kms,omitempty"`
105-
}
106-
107-
-// +kubebuilder:validation:Enum="";identity;aescbc;aesgcm
108-
+// +openshift:validation:FeatureGateAwareEnum:featureGate="",enum="";identity;aescbc;aesgcm
109-
+// +openshift:validation:FeatureGateAwareEnum:featureGate=KMSEncryptionProvider,enum="";identity;aescbc;aesgcm;KMS
110-
type EncryptionType string
111-
112-
const (
113115
@@ -208,6 +225,11 @@ const (
114-
// aesgcm refers to a type where AES-GCM with random nonce and a 32-byte key
115-
// is used to perform encryption at the datastore layer.
116-
EncryptionTypeAESGCM EncryptionType = "aesgcm"
116+
// aesgcm refers to a type where AES-GCM with random nonce and a 32-byte key
117+
// is used to perform encryption at the datastore layer.
118+
EncryptionTypeAESGCM EncryptionType = "aesgcm"
117119
+
118-
+ // kms refers to a type of encryption where the encryption keys are managed
119-
+ // outside the control plane in a Key Management Service instance,
120-
+ // encryption is still performed at the datastore layer.
121-
+ EncryptionTypeKMS EncryptionType = "KMS"
120+
+ // kms refers to a type of encryption where the encryption keys are managed
121+
+ // outside the control plane in a Key Management Service instance,
122+
+ // encryption is still performed at the datastore layer.
123+
+ EncryptionTypeKMS EncryptionType = "KMS"
122124
)
123-
124-
type APIServerStatus struct {
125+
```
126+
127+
The default value today is an empty string, which implies identity and that no encryption is used in the cluster by default. Other possible local encryption schemes include `aescbc` and `aesgcm`, which will remain as-is. Similar to how local AES encryption works, the apiserver operators will observe this config and apply the KMS EncryptionProvider to the EncryptionConfig.
128+
129+
```diff
130+
@@ -191,9 +194,23 @@ type APIServerEncryption struct {
131+
// +unionDiscriminator
132+
// +optional
133+
Type EncryptionType `json:"type,omitempty"`
134+
+
135+
+ // kms defines the configuration for the external KMS instance that manages the encryption keys,
136+
+ // when KMS encryption is enabled sensitive resources will be encrypted using keys managed by an
137+
+ // externally configured KMS instance.
138+
+ //
139+
+ // The Key Management Service (KMS) instance provides symmetric encryption and is responsible for
140+
+ // managing the lifecyle of the encryption keys outside of the control plane.
141+
+ // This allows integration with an external provider to manage the data encryption keys securely.
142+
+ //
143+
+ // +openshift:enable:FeatureGate=KMSEncryptionProvider
144+
+ // +unionMember
145+
+ // +optional
146+
+ KMS *KMSConfig `json:"kms,omitempty"`
147+
```
148+
149+
As mentioned before, the KMS encryption type will have a dedicated configuration.
150+
151+
```diff
125152
diff --git a/config/v1/types_kmsencryption.go b/config/v1/types_kmsencryption.go
126153
new file mode 100644
127-
index 000000000..575affae6
154+
index 000000000..8841cd749
128155
--- /dev/null
129156
+++ b/config/v1/types_kmsencryption.go
130-
@@ -0,0 +1,50 @@
157+
@@ -0,0 +1,49 @@
131158
+package v1
132159
+
133160
+// KMSConfig defines the configuration for the KMS instance
134161
+// that will be used with KMSEncryptionProvider encryption
135162
+// +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'AWS' ? has(self.aws) : !has(self.aws)",message="aws config is required when kms provider type is AWS, and forbidden otherwise"
136163
+// +union
137164
+type KMSConfig struct {
138-
+ // type defines the kind of platform for the KMS provider.
139-
+ // Available provider types are AWS only.
140-
+ //
141-
+ // +unionDiscriminator
142-
+ // +kubebuilder:validation:Required
143-
+ Type KMSProviderType `json:"type"`
165+
+ // type defines the kind of platform for the KMS provider
166+
+ //
167+
+ // +unionDiscriminator
168+
+ // +kubebuilder:validation:Required
169+
+ Type KMSProviderType `json:"type"`
144170
+
145-
+ // aws defines the key config for using an AWS KMS instance
146-
+ // for the encryption. The AWS KMS instance is managed
147-
+ // by the user outside the purview of the control plane.
148-
+ //
149-
+ // +unionMember
150-
+ // +optional
151-
+ AWS *AWSKMSConfig `json:"aws,omitempty"`
171+
+ // aws defines the key config for using an AWS KMS instance
172+
+ // for the encryption. The AWS KMS instance is managed
173+
+ // by the user outside the purview of the control plane.
174+
+ //
175+
+ // +unionMember
176+
+ // +optional
177+
+ AWS *AWSKMSConfig `json:"aws,omitempty"`
152178
+}
179+
180+
+// KMSProviderType is a specific supported KMS provider
181+
+// +kubebuilder:validation:Enum="";AWS
182+
+type KMSProviderType string
153183
+
184+
+const (
185+
+ // AWSKMSProvider represents a supported KMS provider for use with AWS KMS
186+
+ AWSKMSProvider KMSProviderType = "AWS"
187+
+)
188+
```
189+
190+
This configuration will also include an enum of the various KMS supported by OCP. At first, it will only have the `AWS` type, but we will add more as we progress on the feature. This enum is essential to avoid potential ambiguities that might arise with future KMS.
191+
192+
Each KMS type will have a dedicated configuration that will be reflected on the plugin when installed. It will only be a partial representation of the plugin's configuration because most fields are irrelevant to the end users.
193+
194+
At first, this configuration will only include the `AWSKMSConfig`, but more KMS-specifc configs will be added as we include new KMS.
195+
196+
```diff
154197
+// AWSKMSConfig defines the KMS config specific to AWS KMS provider
155198
+type AWSKMSConfig struct {
156199
+ // keyARN specifies the Amazon Resource Name (ARN) of the AWS KMS key used for encryption.
@@ -169,18 +212,11 @@ index 000000000..575affae6
169212
+ // +kubebuilder:validation:XValidation:rule="self.matches('^[a-z]{2}-[a-z]+-[0-9]+$') && self.size() <= 64",message="region must be a valid AWS region"
170213
+ Region string `json:"region"`
171214
+}
172-
+
173-
+// KMSProviderType is a specific supported KMS provider
174-
+// +kubebuilder:validation:Enum="";AWS
175-
+type KMSProviderType string
176-
+
177-
+const (
178-
+ // AWSKMSProvider represents a supported KMS provider for use with AWS KMS
179-
+ AWSKMSProvider KMSProviderType = "AWS"
180-
+)
181215
```
182216

183-
The default value today is an empty string which implies identity and that no encryption is used in the cluster by default. Other possible local encryption schemes include `aescbc` and `aesgcm` which will remain as-is. Similar to how local AES encryption works, the kube-apiserver operator and openshift-apiserver operator will observe this config to apply the KMS Encryption Provider config onto kube-apiserver(s) and openshift-apiserver(s) respectively.
217+
From a very high-level, the figure below shows how the new APIs will be used by the various components.
218+
219+
![KMS high-level design](./kms-design.png)
184220

185221
### Implementation Details/Notes/Constraints
186222

0 commit comments

Comments
 (0)