forked from postgres/postgres
-
Notifications
You must be signed in to change notification settings - Fork 14
PG-1832 Document the archive and restore commands cont #531
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 15 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
a8829b5
PG-1832 Document pg_tde's archive and restore commands
jeltz 2cdba4d
Merge branch 'TDE_REL_17_STABLE' into ak/tde/archive-restore-docs
Andriciuc 0dc4618
Merge branch 'TDE_REL_17_STABLE' into tde-archive-restore-docs-cont
Andriciuc 5af36d5
Merge branch 'TDE_REL_17_STABLE' into tde-archive-restore-docs-cont
Andriciuc 3e3c9f4
update pg-tde-archive-decrypt.md with new commands and clarity in tex…
Andriciuc f8b115f
update cli-tools.md with paragraphs explainint New and extended tools
Andriciuc ee8a470
replace Tools with tool in TOC for CLI tools
Andriciuc 6fe9305
change how it works bullets to numbers
Andriciuc 3420568
Update pg-tde-restore-encrypt tool with new information and better de…
Andriciuc 36d5595
add missing `` to several params
Andriciuc 16b8d3c
updated how it works with better described text
Andriciuc 948c8f1
update text in how it works for restore-encrypt
Andriciuc ded0100
minor text fix for restore-encrypt how it works
Andriciuc 2a92b78
Merge branch 'TDE_REL_17_STABLE' into tde-archive-restore-docs-cont
Andriciuc 0d04930
updated archive-decrypt tool description for clarity
Andriciuc b3b7e89
Update the Features topic buttons for better clarity (#508)
Andriciuc ef0f2fe
Merge branch 'TDE_REL_17_STABLE' into tde-archive-restore-docs-cont
Andriciuc a6680d8
fix a word
Andriciuc 594e194
Merge branch 'release-17.5.3' into tde-archive-restore-docs-cont
Andriciuc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
19 changes: 15 additions & 4 deletions
19
contrib/pg_tde/documentation/docs/command-line-tools/cli-tools.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,18 @@ | ||
# Overview of pg_tde CLI tools | ||
|
||
The `pg_tde` extension introduces new command-line utilities and extends some existing PostgreSQL tools to support encrypted WAL and tables. These include: | ||
The `pg_tde` extension introduces new command-line utilities and extends some existing PostgreSQL tools to support encrypted WAL and tables. | ||
|
||
* [pg_tde_change_key_provider](../command-line-tools/pg-tde-change-key-provider.md): change encryption key provider for a database | ||
* [pg_waldump](../command-line-tools/pg-waldump.md): inspect and decrypt WAL files | ||
* [pg_checksums](../command-line-tools/pg-tde-checksums.md): verify data checksums (non-encrypted files only) | ||
## New tools | ||
|
||
These tools are introduced by `pg_tde` to support key rotation and WAL encryption workflows: | ||
|
||
* [pg_tde_change_key_provider](./pg-tde-change-key-provider.md): change the encryption key provider for a database | ||
* [pg_tde_archive_decrypt](./pg-tde-archive-decrypt.md): decrypts WAL before archiving | ||
* [pg_tde_restore_encrypt](./pg-tde-restore-encrypt.md): a custom restore command for making sure the restored WAL is encrypted | ||
|
||
## Extended tools | ||
|
||
These existing PostgreSQL tools are enhanced to support `pg_tde`: | ||
|
||
* [pg_checksums](./pg-tde-checksums.md): verify data checksums (non-encrypted files only) | ||
* [pg_waldump](./pg-waldump.md): inspect and decrypt WAL files |
55 changes: 55 additions & 0 deletions
55
contrib/pg_tde/documentation/docs/command-line-tools/pg-tde-archive-decrypt.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
# pg_tde_archive_decrypt | ||
|
||
The `pg_tde_archive_decrypt` tool wraps an archive command and decrypts WAL files before archiving. It allows external tools to access unencrypted WAL data, which is required because WAL encryption keys in the two-key hierarchy are host-specific and may not be available on the replay host. | ||
|
||
!!! tip | ||
|
||
For more information on the encryption architecture and key hierarchy, see [Architecture](../architecture/architcture.md). | ||
|
||
This tool is often use in conjunction with [pg_tde_restore_encrypt](./pg-tde-restore-encrypt.md) to support WAL archive. | ||
|
||
## How it works | ||
|
||
1. Decrypts the WAL segment to a temporary file on a RAM disk (`/dev/shm`) | ||
2. Replaces `%p` and `%f` in the archive command with the path and name of the decrypted file | ||
3. Executes the archive command | ||
|
||
!!! note | ||
|
||
To ensure security, encrypt the files stored in your WAL archive using tools like `PgBackRest`. | ||
|
||
## Usage | ||
|
||
```bash | ||
pg_tde_archive_decrypt [OPTION] | ||
pg_tde_archive_decrypt DEST-NAME SOURCE-PATH ARCHIVE-COMMAND | ||
``` | ||
|
||
## Parameter descriptions | ||
|
||
* `DEST-NAME`: name of the WAL file to send to the archive | ||
* `SOURCE-PATH`: path to the original encrypted WAL file | ||
* `ARCHIVE-COMMAND`: archive command to wrap. `%p` and `%f` are replaced with the decrypted WAL file path and WAL file name, respectively. | ||
|
||
## Options | ||
|
||
* `-V, --version`: show version information, then exit | ||
* `-?, --help`: show help information, then exit | ||
|
||
!!! note | ||
|
||
Any `%f` or `%p` parameter in `ARCHIVE-COMMAND` has to be escaped as `%%f` or `%%p` respectively if used as `archive_command` in `postgresql.conf`. | ||
|
||
## Examples | ||
|
||
### Using `cp` | ||
|
||
```ini | ||
archive_command='pg_tde_archive_decrypt %f %p "cp %%p /mnt/server/archivedir/%%f"' | ||
``` | ||
|
||
### Using `PgBackRest` | ||
|
||
```ini | ||
archive_command='pg_tde_archive_decrypt %f %p "pgbackrest --stanza=your_stanza archive-push %%p"' | ||
``` |
49 changes: 49 additions & 0 deletions
49
contrib/pg_tde/documentation/docs/command-line-tools/pg-tde-restore-encrypt.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# pg_tde_restore_encrypt | ||
|
||
The `pg_tde_restore_encrypt` tool wraps a normal restore command from the WAL archive and writes them to disk in a format compatible with `pg_tde`. | ||
|
||
!!! note | ||
|
||
This command is often use together with [pg_tde_archive_decrypt](./pg-tde-archive-decrypt.md). | ||
|
||
## How it works | ||
|
||
1. Replaces `%f` and `%p` in the restore command with the WAL file name and temporary file path (in `/dev/shm`) | ||
2. Runs the restore command to fetch the unencrypted WAL from the archive and write it to the temp file | ||
3. Encrypts the temp file and writes it to the destination path in PostgreSQL’s data directory | ||
|
||
## Usage | ||
|
||
```bash | ||
pg_tde_restore_encrypt [OPTION] | ||
pg_tde_restore_encrypt SOURCE-NAME DEST-PATH RESTORE-COMMAND | ||
``` | ||
|
||
## Parameter descriptions | ||
|
||
* `SOURCE-NAME`: name of the WAL file to retrieve from the archive | ||
* `DEST-PATH`: path where the encrypted WAL file should be written | ||
* `RESTORE-COMMAND`: restore command to wrap; `%p` and `%f` are replaced with the WAL file name and path to write the unencrypted WAL, respectively | ||
|
||
## Options | ||
|
||
* `-V, --version`: show version information, then exit | ||
* `-?, --help`: show help information, then exit | ||
|
||
!!! note | ||
|
||
Any `%f` or `%p` parameter in `RESTORE-COMMAND` has to be escaped as `%%f` or `%%p` respectively if used as `restore_command` in `postgresql.conf`. | ||
|
||
## Examples | ||
|
||
### Using `cp` | ||
|
||
```ini | ||
restore_command='pg_tde_restore_encrypt %f %p "cp /mnt/server/archivedir/%%f %%p"' | ||
``` | ||
|
||
### Using `PgBackRest` | ||
|
||
```ini | ||
restore_command='pg_tde_restore_encrypt %f %p "pgbackrest --stanza=your_stanza archive-get %%f \"%%p\""' | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.