Skip to content

Commit 57b77d8

Browse files
adriandolekevinAlbs
adriandole
and
kevinAlbs
authored
DRIVERS-2732 CSFLE/QE KMIP support for encrypt/decrypt (#1507)
Spec additions, YAML and JSON tests for KMIP encrypt/decrypt --------- Co-authored-by: Kevin Albertson <[email protected]>
1 parent 2c3be08 commit 57b77d8

13 files changed

+964
-38
lines changed

‎source/client-side-encryption/client-side-encryption.md

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1087,7 +1087,8 @@ If the kmsProvider has KMS provider type "kmip", the masterKey is required and h
10871087
{
10881088
keyId: Optional<String>, // keyId is the KMIP Unique Identifier to a 96 byte KMIP Secret Data managed object.
10891089
// If keyId is omitted, the driver creates a random 96 byte KMIP Secret Data managed object.
1090-
endpoint: Optional<String> // Host with optional port.
1090+
endpoint: Optional<String>, // Host with optional port.
1091+
delegated: Optional<Boolean> // If true, this key should be decrypted by the KMIP server.
10911092
}
10921093
```
10931094

@@ -1727,12 +1728,13 @@ Data keys are stored in the MongoDB key vault collection with the following sche
17271728
| **Name** | **Type** | **Description** |
17281729
| provider | "local" | |
17291730

1730-
| | | |
1731-
| -------- | -------- | -------------------------------------------------------------------------------------- |
1732-
| **Name** | **Type** | **Description** |
1733-
| provider | "kmip" | |
1734-
| endpoint | String | Optional. Defaults to kmip.endpoint from KMS providers. |
1735-
| keyId | String | Required. keyId is the Unique Identifier to a 96 byte KMIP Secret Data managed object. |
1731+
| | | |
1732+
| --------- | -------- | -------------------------------------------------------------------------------------- |
1733+
| **Name** | **Type** | **Description** |
1734+
| provider | "kmip" | |
1735+
| endpoint | String | Optional. Defaults to kmip.endpoint from KMS providers. |
1736+
| delegated | Boolean | Optional. Defaults to false. |
1737+
| keyId | String | Required. keyId is the Unique Identifier to a 96 byte KMIP Secret Data managed object. |
17361738

17371739
Data keys are needed for encryption and decryption. They are identified in the intent-to-encrypt marking and ciphertext.
17381740
Data keys may be retrieved by querying the "\_id" with a UUID or by querying the "keyAltName" with a string.
@@ -2327,6 +2329,12 @@ KMS credentials would require added work in drivers inspecting the KMS providers
23272329
additional test coverage. Supporting on-demand KMS credentials for named KMS providers can be considered as future work
23282330
if needed.
23292331

2332+
### What is the KMIP `delegated` option?
2333+
2334+
By default, the KMS will retrieve the key encryption key from the KMIP server and use it to encrypt the data key. If the
2335+
`delegated` option is set to true (recommended), the KMIP server will instead perform encryption and decryption locally,
2336+
ensuring that the key encryption key never leaves the server.
2337+
23302338
## Future work
23312339

23322340
### Make libmonogocrypt cache window configurable
@@ -2381,6 +2389,8 @@ explicit session parameter as described in the [Drivers Sessions Specification](
23812389

23822390
## Changelog
23832391

2392+
- 2024-03-20: Add `delegated` option to "kmip" KMS provider
2393+
23842394
- 2024-02-27: Migrated from reStructuredText to Markdown.
23852395

23862396
- 2024-01-10: Add named KMS providers

‎source/client-side-encryption/etc/generate-test.py

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,35 @@
179179
"keyId": "1"
180180
},
181181
"keyAltNames": ["altname", "kmip_altname"]
182+
},
183+
"kmip_delegated": {
184+
"_id": {
185+
"$uuid": "7411e9af-c688-4df7-8143-5e60ae96cba6"
186+
},
187+
"keyMaterial": {
188+
"$binary": {
189+
"base64": "5TLMFWlguBWe5GUESTvOVtkdBsCrynhnV72XRyZ66/nk+EP9/1oEp1t1sg0+vwCTqULHjBiUE6DRx2mYD/Eup1+u2Jgz9/+1sV1drXeOPALNPkSgiZiDbIb67zRi+wTABEcKcegJH+FhmSGxwUoQAiHCsCbcvia5P8tN1lt98YQ=", "subType": "00"
190+
}
191+
},
192+
"creationDate": {
193+
"$date": {
194+
"$numberLong": "1634220190041"
195+
}
196+
},
197+
"updateDate": {
198+
"$date": {
199+
"$numberLong": "1634220190041"
200+
}
201+
},
202+
"status": {
203+
"$numberInt": "0"
204+
},
205+
"masterKey": {
206+
"provider": "kmip",
207+
"delegated": True,
208+
"keyId": "11"
209+
},
210+
"keyAltNames": ["delegated"]
182211
}
183212
}
184213

@@ -254,6 +283,11 @@
254283
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic"
255284
}
256285
},
286+
"encrypted_string_kmip_delegated": {
287+
"encrypt": {
288+
"keyId": [keys["kmip_delegated"]["_id"]],
289+
"bsonType": "string",
290+
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic"}},
257291
},
258292
"bsonType": "object"
259293
},
@@ -563,6 +597,17 @@
563597
}
564598
}
565599
},
600+
{
601+
"schema": "all",
602+
"field": "encrypted_string_kmip_delegated",
603+
"plaintext": "string0",
604+
"data": {
605+
"$binary": {
606+
"base64": "AXQR6a/GiE33gUNeYK6Wy6YCkB+8NVfAAjIbvLqyXIg6g1a8tXrym92DPoqmxpcdQyH0vQM3aFNMz7tZwQBimKs29ztZV/LWjM633HhO5ACl9A==",
607+
"subType": "06"
608+
}
609+
}
610+
},
566611
{
567612
"schema": "local:name2",
568613
"field": "encrypted_string",

‎source/client-side-encryption/etc/test-templates/kmipKMS.yml.template

Lines changed: 45 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ collection_name: &collection_name "default"
55

66
data: []
77
json_schema: {{schema("all")}}
8-
key_vault_data: [{{key("kmip")}}]
8+
key_vault_data: [{{key("kmip")}},{{key("kmip_delegated")}}]
99

1010
tests:
1111
- description: "Insert a document with auto encryption using KMIP KMS provider"
@@ -43,4 +43,47 @@ tests:
4343
collection:
4444
# Outcome is checked using a separate MongoClient without auto encryption.
4545
data:
46-
- *doc0_encrypted
46+
- *doc0_encrypted
47+
48+
- description: "Insert a document with auto encryption using KMIP delegated KMS provider"
49+
clientOptions:
50+
autoEncryptOpts:
51+
kmsProviders:
52+
kmip: {}
53+
operations:
54+
- name: insertOne
55+
arguments:
56+
document: &doc1 { _id: 1, encrypted_string_kmip_delegated: "string0" }
57+
expectations:
58+
- command_started_event:
59+
command:
60+
listCollections: 1
61+
filter:
62+
name: *collection_name
63+
command_name: listCollections
64+
# Auto encryption will request the collection info.
65+
- command_started_event:
66+
command:
67+
listCollections: 1
68+
filter:
69+
name: datakeys
70+
command_name: listCollections
71+
# Then key is fetched from the key vault.
72+
- command_started_event:
73+
command:
74+
find: datakeys
75+
filter: { $or: [ { _id: { $in: [ {{key("kmip_delegated")["_id"]}} ] } }, { keyAltNames: { $in: [] } } ] }
76+
$db: keyvault
77+
command_name: find
78+
- command_started_event:
79+
command:
80+
insert: *collection_name
81+
documents:
82+
- &doc1_encrypted { _id: 1, encrypted_string_kmip_delegated: {{ciphertext("string0", field="encrypted_string_kmip_delegated", schema="all")}} }
83+
ordered: true
84+
command_name: insert
85+
outcome:
86+
collection:
87+
# Outcome is checked using a separate MongoClient without auto encryption.
88+
data:
89+
- *doc1_encrypted

‎source/client-side-encryption/tests/legacy/azureKMS.json

Lines changed: 11 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎source/client-side-encryption/tests/legacy/azureKMS.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ database_name: &database_name "default"
44
collection_name: &collection_name "default"
55

66
data: []
7-
json_schema: {'properties': {'encrypted_string_aws': {'encrypt': {'keyId': [{'$binary': {'base64': 'AAAAAAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_azure': {'encrypt': {'keyId': [{'$binary': {'base64': 'AZURE+AAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_gcp': {'encrypt': {'keyId': [{'$binary': {'base64': 'GCP+AAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_local': {'encrypt': {'keyId': [{'$binary': {'base64': 'AAAAAAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_kmip': {'encrypt': {'keyId': [{'$binary': {'base64': 'dBHpr8aITfeBQ15grpbLpQ==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}}, 'bsonType': 'object'}
7+
json_schema: {'properties': {'encrypted_string_aws': {'encrypt': {'keyId': [{'$binary': {'base64': 'AAAAAAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_azure': {'encrypt': {'keyId': [{'$binary': {'base64': 'AZURE+AAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_gcp': {'encrypt': {'keyId': [{'$binary': {'base64': 'GCP+AAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_local': {'encrypt': {'keyId': [{'$binary': {'base64': 'AAAAAAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_kmip': {'encrypt': {'keyId': [{'$binary': {'base64': 'dBHpr8aITfeBQ15grpbLpQ==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_kmip_delegated': {'encrypt': {'keyId': [{'$uuid': '7411e9af-c688-4df7-8143-5e60ae96cba6'}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}}, 'bsonType': 'object'}
88
key_vault_data: [{'_id': {'$binary': {'base64': 'AZURE+AAAAAAAAAAAAAAAA==', 'subType': '04'}}, 'keyMaterial': {'$binary': {'base64': 'n+HWZ0ZSVOYA3cvQgP7inN4JSXfOH85IngmeQxRpQHjCCcqT3IFqEWNlrsVHiz3AELimHhX4HKqOLWMUeSIT6emUDDoQX9BAv8DR1+E1w4nGs/NyEneac78EYFkK3JysrFDOgl2ypCCTKAypkn9CkAx1if4cfgQE93LW4kczcyHdGiH36CIxrCDGv1UzAvERN5Qa47DVwsM6a+hWsF2AAAJVnF0wYLLJU07TuRHdMrrphPWXZsFgyV+lRqJ7DDpReKNO8nMPLV/mHqHBHGPGQiRdb9NoJo8CvokGz4+KE8oLwzKf6V24dtwZmRkrsDV4iOhvROAzz+Euo1ypSkL3mw==', 'subType': '00'}}, 'creationDate': {'$date': {'$numberLong': '1601573901680'}}, 'updateDate': {'$date': {'$numberLong': '1601573901680'}}, 'status': {'$numberInt': '0'}, 'masterKey': {'provider': 'azure', 'keyVaultEndpoint': 'key-vault-csfle.vault.azure.net', 'keyName': 'key-name-csfle'}, 'keyAltNames': ['altname', 'azure_altname']}]
99

1010
tests:

‎source/client-side-encryption/tests/legacy/gcpKMS.json

Lines changed: 11 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎source/client-side-encryption/tests/legacy/gcpKMS.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ database_name: &database_name "default"
44
collection_name: &collection_name "default"
55

66
data: []
7-
json_schema: {'properties': {'encrypted_string_aws': {'encrypt': {'keyId': [{'$binary': {'base64': 'AAAAAAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_azure': {'encrypt': {'keyId': [{'$binary': {'base64': 'AZURE+AAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_gcp': {'encrypt': {'keyId': [{'$binary': {'base64': 'GCP+AAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_local': {'encrypt': {'keyId': [{'$binary': {'base64': 'AAAAAAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_kmip': {'encrypt': {'keyId': [{'$binary': {'base64': 'dBHpr8aITfeBQ15grpbLpQ==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}}, 'bsonType': 'object'}
7+
json_schema: {'properties': {'encrypted_string_aws': {'encrypt': {'keyId': [{'$binary': {'base64': 'AAAAAAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_azure': {'encrypt': {'keyId': [{'$binary': {'base64': 'AZURE+AAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_gcp': {'encrypt': {'keyId': [{'$binary': {'base64': 'GCP+AAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_local': {'encrypt': {'keyId': [{'$binary': {'base64': 'AAAAAAAAAAAAAAAAAAAAAA==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_kmip': {'encrypt': {'keyId': [{'$binary': {'base64': 'dBHpr8aITfeBQ15grpbLpQ==', 'subType': '04'}}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}, 'encrypted_string_kmip_delegated': {'encrypt': {'keyId': [{'$uuid': '7411e9af-c688-4df7-8143-5e60ae96cba6'}], 'bsonType': 'string', 'algorithm': 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'}}}, 'bsonType': 'object'}
88
key_vault_data: [{'_id': {'$binary': {'base64': 'GCP+AAAAAAAAAAAAAAAAAA==', 'subType': '04'}}, 'keyMaterial': {'$binary': {'base64': 'CiQAIgLj0WyktnB4dfYHo5SLZ41K4ASQrjJUaSzl5vvVH0G12G0SiQEAjlV8XPlbnHDEDFbdTO4QIe8ER2/172U1ouLazG0ysDtFFIlSvWX5ZnZUrRMmp/R2aJkzLXEt/zf8Mn4Lfm+itnjgo5R9K4pmPNvvPKNZX5C16lrPT+aA+rd+zXFSmlMg3i5jnxvTdLHhg3G7Q/Uv1ZIJskKt95bzLoe0tUVzRWMYXLIEcohnQg==', 'subType': '00'}}, 'creationDate': {'$date': {'$numberLong': '1601574333107'}}, 'updateDate': {'$date': {'$numberLong': '1601574333107'}}, 'status': {'$numberInt': '0'}, 'masterKey': {'provider': 'gcp', 'projectId': 'devprod-drivers', 'location': 'global', 'keyRing': 'key-ring-csfle', 'keyName': 'key-name-csfle'}, 'keyAltNames': ['altname', 'gcp_altname']}]
99

1010
tests:

0 commit comments

Comments
 (0)