Skip to content

Commit 7d92ed0

Browse files
HappyTetrahedronHappyTetrahedron
authored
Merge pull request #90 from projectsyn/docs/restore-procedure
Document how to perform a restore from k8up backup
2 parents a1ba184 + 89b529b commit 7d92ed0

File tree

4 files changed

+148
-0
lines changed

4 files changed

+148
-0
lines changed

‎docs/modules/ROOT/pages/how-tos/restore-from-backup.adoc

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
= Restore from Backup
2+
3+
When configuring component-vault with `backup.enabled = true`, the component sets up regular backups using k8up.
4+
This how-to explains how Vault can be restored from such a backup.
5+
6+
== Information on the Vault Unseal Key and Root Token
7+
8+
include::partial$vault-warning.adoc[]
9+
10+
== Prerequisites
11+
12+
* `restic` - command line tool to access backups made by k8up
13+
* `vault` - command line tool to interact with Vault
14+
* `kubectl`
15+
* Write access to the cluster's tenant repository
16+
* Read access to the restic repository in which k8up stored the Vault backups
17+
* The Vault instance's *unseal key* and *root token* - these must be backed up manually; they are not part of the automated k8up backup.
18+
19+
== Procedure
20+
21+
=== 1. Set up new Vault instance
22+
23+
. Add the `vault` application to your cluster configuration.
24+
.. If your old instance of vault is still running on the cluster, you can use component instantiation to create a second instance by adding `vault as new-vault` to your application list, and configuring it under `new_vault`.
25+
. Initially disable backups by setting `.backups.enabled` to `false`
26+
. Compile and push the cluster config and wait for Vault to start.
27+
28+
=== 2. Retrieve the Vault snapshot
29+
30+
. Set up the restic credentials (values correspond to the component parameters `backup.bucket` and `backup.password`)
31+
+
32+
[source,shell]
33+
----
34+
export AWS_ACCESS_KEY_ID="S3_KEY" # from component configuration: backup.bucket.accesskey
35+
export AWS_SECRET_ACCESS_KEY="S3_SECRET" # from component confiugration: backup.bucket.secretkey
36+
export RESTIC_REPOSITORY="s3:https://path.to.my/bucket"
37+
export RESTIC_PASSWORD="RESTIC_REPO_KEY" # from component configuration: backup.password
38+
----
39+
. Retrieve the latest Vault snapshot to your local disk
40+
+
41+
[source,shell]
42+
----
43+
mkdir restore
44+
restic restore --target restore/ latest
45+
----
46+
. Verify the snapshot file
47+
+
48+
[source,shell]
49+
----
50+
ls restore
51+
# This should show a file named "[instance name]-backup.snapshot"
52+
----
53+
54+
=== 3. Restore the snapshot
55+
56+
. Expose the Vault pod
57+
+
58+
[source,shell]
59+
----
60+
kubectl port-forward -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-0 8200
61+
----
62+
. In a separate terminal, prepare the environment to access Vault
63+
+
64+
[source,shell]
65+
----
66+
# Get root token to log in
67+
export VAULT_TOKEN="$(kubectl get secret -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-seal -ojsonpath='{.data.vault-root}' | base64 -d)"
68+
export VAULT_ADDR="http://127.0.0.1:8200"
69+
----
70+
. Restore the backup
71+
+
72+
[source,shell]
73+
----
74+
vault operator raft snapshot restore -force restore/${VAULT_INSTANCE_NAME}-backup.snapshot
75+
----
76+
77+
=== 4. Unseal Vault
78+
79+
If you were logged into the Vault UI, you should have gotten logged out now.
80+
This is expected.
81+
82+
. Open your browser at http://localhost:8200
83+
. Use the *Vault Unseal Key* of the Vault instance you've just restored to unseal Vault
84+
. Use the *Vault root token* of the Vault instance you've just restored to log in with the `Token` method
85+
. Verify that the restore worked, and secrets are now restored in Vault.
86+
87+
[IMPORTANT]
88+
====
89+
The unseal key and root token of the Vault instance you're restoring need to have been stored separately.
90+
Without them, the restore procedure cannot be completed.
91+
====
92+
93+
=== 5. Update the Vault Secret
94+
95+
NOTE: Without this step, your Vault instance will not be able to auto-unseal.
96+
97+
. Encode the Vault credentials
98+
+
99+
[source,shell]
100+
----
101+
export VAULT_UNSEAL_KEY="OLD_UNSEAL_KEY"
102+
export VAULT_ROOT_TOKEN="OLD_ROOT_TOKEN"
103+
104+
echo -n "$VAULT_UNSEAL_KEY" | base64 -w0
105+
echo -n "$VAULT_ROOT_TOKEN" | base64 -w0
106+
----
107+
. Update the Vault secret
108+
+
109+
[source,shell]
110+
----
111+
kubectl edit secret -n ${VAULT_INSTANCE_NAME} ${VAULT_INSTANCE_NAME}-seal
112+
----
113+
. Update the `vault-root` and `vault-unseal-0` keys to reflect the values you have just encoded
114+
. Save the secret
115+
. Verify that auto-unseal works:
116+
.. Restart all vault pods simultaneously:
117+
+
118+
[source,shell]
119+
----
120+
kubectl delete pod -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-{0..2}
121+
----
122+
.. Expose the Vault UI
123+
+
124+
[source,shell]
125+
----
126+
kubectl port-forward -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-0 8200
127+
----
128+
.. Verify that http://localhost:8200[the Vault UI] does not prompt you for the unseal key
129+
130+
131+
=== 6. Cleanup
132+
133+
. Reenable `backups.enabled` in the component configuration

‎docs/modules/ROOT/pages/references/parameters.adoc

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,12 @@ default:: `true`
252252

253253
Whether to do backups using k8up.
254254

255+
[WARNING]
256+
.Manual Setup Required
257+
====
258+
include::partial$vault-warning.adoc[]
259+
====
260+
255261
== `backup.schedule`
256262

257263
[horizontal]

‎docs/modules/ROOT/partials/nav.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
* xref:index.adoc[Home]
22
* xref:tutorials/install.adoc[Installation]
33
* xref:how-tos/lieutenant.adoc[Lieutenant Integration]
4+
* xref:how-tos/restore-from-backup.adoc[Restore Vault from Backup]
45
* xref:references/parameters.adoc[Parameters]

‎docs/modules/ROOT/partials/vault-warning.adoc

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
`component-vault` leverages k8up's application specific backups to create Vault snapshots.
2+
Restoring such a snapshot is only possible by providing the Vault https://developer.hashicorp.com/vault/docs/concepts/seal[unseal key] and https://developer.hashicorp.com/vault/docs/concepts/tokens#root-tokens[root token].
3+
As these are sensitive secrets, they are not backed up as part of the component's automated backup process.
4+
5+
When setting up Vault, you must store the Vault unseal key and root token in a safe location in order to be able to restore backups.
6+
7+
In a running Vault instance, both of these secrets can be found in the `[instance name]-seal` secret in the Vault instance's namespace.
8+
In order to have a reliable Vault backup, this secret must be backed up separately to a safe location.

0 commit comments

Comments
 (0)