Skip to content

PG-1832 Document the archive and restore commands cont #531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: release-17.5.3
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
a8829b5
PG-1832 Document pg_tde's archive and restore commands
jeltz Aug 11, 2025
2cdba4d
Merge branch 'TDE_REL_17_STABLE' into ak/tde/archive-restore-docs
Andriciuc Aug 13, 2025
0dc4618
Merge branch 'TDE_REL_17_STABLE' into tde-archive-restore-docs-cont
Andriciuc Aug 13, 2025
5af36d5
Merge branch 'TDE_REL_17_STABLE' into tde-archive-restore-docs-cont
Andriciuc Aug 13, 2025
3e3c9f4
update pg-tde-archive-decrypt.md with new commands and clarity in tex…
Andriciuc Aug 13, 2025
f8b115f
update cli-tools.md with paragraphs explainint New and extended tools
Andriciuc Aug 13, 2025
ee8a470
replace Tools with tool in TOC for CLI tools
Andriciuc Aug 13, 2025
6fe9305
change how it works bullets to numbers
Andriciuc Aug 13, 2025
3420568
Update pg-tde-restore-encrypt tool with new information and better de…
Andriciuc Aug 13, 2025
36d5595
add missing `` to several params
Andriciuc Aug 13, 2025
16b8d3c
updated how it works with better described text
Andriciuc Aug 13, 2025
948c8f1
update text in how it works for restore-encrypt
Andriciuc Aug 13, 2025
ded0100
minor text fix for restore-encrypt how it works
Andriciuc Aug 13, 2025
2a92b78
Merge branch 'TDE_REL_17_STABLE' into tde-archive-restore-docs-cont
Andriciuc Aug 13, 2025
0d04930
updated archive-decrypt tool description for clarity
Andriciuc Aug 13, 2025
b3b7e89
Update the Features topic buttons for better clarity (#508)
Andriciuc Aug 20, 2025
ef0f2fe
Merge branch 'TDE_REL_17_STABLE' into tde-archive-restore-docs-cont
Andriciuc Aug 20, 2025
a6680d8
fix a word
Andriciuc Aug 20, 2025
594e194
Merge branch 'release-17.5.3' into tde-archive-restore-docs-cont
Andriciuc Aug 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,18 @@
# Overview of pg_tde CLI tools

The `pg_tde` extension introduces new command-line utilities and extends some existing PostgreSQL tools to support encrypted WAL and tables. These include:
The `pg_tde` extension introduces new command-line utilities and extends some existing PostgreSQL tools to support encrypted WAL and tables.

* [pg_tde_change_key_provider](../command-line-tools/pg-tde-change-key-provider.md): change encryption key provider for a database
* [pg_waldump](../command-line-tools/pg-waldump.md): inspect and decrypt WAL files
* [pg_checksums](../command-line-tools/pg-tde-checksums.md): verify data checksums (non-encrypted files only)
## New tools

These tools are introduced by `pg_tde` to support key rotation and WAL encryption workflows:

* [pg_tde_change_key_provider](./pg-tde-change-key-provider.md): change the encryption key provider for a database
* [pg_tde_archive_decrypt](./pg-tde-archive-decrypt.md): decrypts WAL before archiving
* [pg_tde_restore_encrypt](./pg-tde-restore-encrypt.md): a custom restore command for making sure the restored WAL is encrypted

## Extended tools

These existing PostgreSQL tools are enhanced to support `pg_tde`:

* [pg_checksums](./pg-tde-checksums.md): verify data checksums (non-encrypted files only)
* [pg_waldump](./pg-waldump.md): inspect and decrypt WAL files
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# pg_tde_archive_decrypt

The `pg_tde_archive_decrypt` tool wraps an archive command and decrypts WAL files before archiving. It allows external tools to access unencrypted WAL data, which is required because WAL encryption keys in the two-key hierarchy are host-specific and may not be available on the replay host.

!!! tip

For more information on the encryption architecture and key hierarchy, see [Architecture](../architecture/architcture.md).

This tool is often used in conjunction with [pg_tde_restore_encrypt](./pg-tde-restore-encrypt.md) to support WAL archive.

## How it works

1. Decrypts the WAL segment to a temporary file on a RAM disk (`/dev/shm`)
2. Replaces `%p` and `%f` in the archive command with the path and name of the decrypted file
3. Executes the archive command

!!! note

To ensure security, encrypt the files stored in your WAL archive using tools like `PgBackRest`.

## Usage

```bash
pg_tde_archive_decrypt [OPTION]
pg_tde_archive_decrypt DEST-NAME SOURCE-PATH ARCHIVE-COMMAND
```

## Parameter descriptions

* `DEST-NAME`: name of the WAL file to send to the archive
* `SOURCE-PATH`: path to the original encrypted WAL file
* `ARCHIVE-COMMAND`: archive command to wrap. `%p` and `%f` are replaced with the decrypted WAL file path and WAL file name, respectively.

## Options

* `-V, --version`: show version information, then exit
* `-?, --help`: show help information, then exit

!!! note

Any `%f` or `%p` parameter in `ARCHIVE-COMMAND` has to be escaped as `%%f` or `%%p` respectively if used as `archive_command` in `postgresql.conf`.

## Examples

### Using `cp`

```ini
archive_command='pg_tde_archive_decrypt %f %p "cp %%p /mnt/server/archivedir/%%f"'
```

### Using `PgBackRest`

```ini
archive_command='pg_tde_archive_decrypt %f %p "pgbackrest --stanza=your_stanza archive-push %%p"'
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# pg_tde_restore_encrypt

The `pg_tde_restore_encrypt` tool wraps a normal restore command from the WAL archive and writes them to disk in a format compatible with `pg_tde`.

!!! note

This command is often use together with [pg_tde_archive_decrypt](./pg-tde-archive-decrypt.md).

## How it works

1. Replaces `%f` and `%p` in the restore command with the WAL file name and temporary file path (in `/dev/shm`)
2. Runs the restore command to fetch the unencrypted WAL from the archive and write it to the temp file
3. Encrypts the temp file and writes it to the destination path in PostgreSQL’s data directory

## Usage

```bash
pg_tde_restore_encrypt [OPTION]
pg_tde_restore_encrypt SOURCE-NAME DEST-PATH RESTORE-COMMAND
```

## Parameter descriptions

* `SOURCE-NAME`: name of the WAL file to retrieve from the archive
* `DEST-PATH`: path where the encrypted WAL file should be written
* `RESTORE-COMMAND`: restore command to wrap; `%p` and `%f` are replaced with the WAL file name and path to write the unencrypted WAL, respectively

## Options

* `-V, --version`: show version information, then exit
* `-?, --help`: show help information, then exit

!!! note

Any `%f` or `%p` parameter in `RESTORE-COMMAND` has to be escaped as `%%f` or `%%p` respectively if used as `restore_command` in `postgresql.conf`.

## Examples

### Using `cp`

```ini
restore_command='pg_tde_restore_encrypt %f %p "cp /mnt/server/archivedir/%%f %%p"'
```

### Using `PgBackRest`

```ini
restore_command='pg_tde_restore_encrypt %f %p "pgbackrest --stanza=your_stanza archive-get %%f \"%%p\""'
```
6 changes: 4 additions & 2 deletions contrib/pg_tde/documentation/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -193,11 +193,13 @@ nav:
- "Functions": functions.md
- "Streaming Replication with tde_heap": replication.md
- "TDE operations":
- "pg_tde CLI Tools":
- "pg_tde CLI tools":
- "Overview": command-line-tools/cli-tools.md
- "pg_tde_change_key_provider": command-line-tools/pg-tde-change-key-provider.md
- "pg_waldump": command-line-tools/pg-waldump.md
- "pg_tde_archive_decrypt": command-line-tools/pg-tde-archive-decrypt.md
- "pg_tde_restore_encrypt": command-line-tools/pg-tde-restore-encrypt.md
- "pg_checksums": command-line-tools/pg-tde-checksums.md
- "pg_waldump": command-line-tools/pg-waldump.md
- "Uninstall pg_tde": how-to/uninstall.md
- "Configure Multi-tenancy": how-to/multi-tenant-setup.md
- "Encryption Enforcement": how-to/enforcement.md
Expand Down