Spin up throwaway PostgreSQL containers for development and testing, with content-addressed seed caching so re-runs skip work they have already done. Supports Docker and Podman with automatic backend detection.
PostgreSQL test setup is usually dominated by re-running the same migrations and seed inserts on every cold start. pg-ephemeral caches each seed step as an OCI image keyed by its content chain: change one seed and only that seed (and anything after it) re-runs; change nothing and the container boots from the final cached image with no setup work at all. See Seed Caching for mechanics and How it compares to testcontainers for the wider picture.
pg-ephemeral ships in three forms so it fits whichever test suite is calling it:
- Rust library —
pg-ephemeralon crates.io; driveDefinitiondirectly from#[tokio::test]integration tests. See Rust Library. - Standalone CLI — the
pg-ephemeralbinary (cargo install pg-ephemeralor release tarball) for any language that can shell out.pg-ephemeral host run-env -- <cmd>exposesPG*andDATABASE_URLto the wrapped process. - Ecosystem integrations —
pg-ephemeralon npm and thepg-ephemeralRuby gem, each bundling the native binary behind an idiomatic API (PgEphemeral.with_connection/withConnection). Want a wrapper for another language? Open an issue — requests welcome.
# Launch psql against an ephemeral database (default command)
pg-ephemeral
# Run a command with PG* / DATABASE_URL set
pg-ephemeral host run-env -- your-dev-tool
# Run an interactive shell on the container
pg-ephemeral container shellThe same binary ships through three other entry points; substitute the
appropriate prefix in any pg-ephemeral … command in this README:
| Source | Invocation |
|---|---|
| Standalone binary (cargo install / release tarball) | pg-ephemeral … |
Node.js project (pg-ephemeral on npm) |
npx pg-ephemeral … |
Ruby project (pg-ephemeral rubygem) |
bundle exec pg-ephemeral … |
Without a config file pg-ephemeral creates a single main instance using the latest
supported PostgreSQL image on the auto-detected container backend. See
examples/01-default
for the no-config workflow.
Runnable example projects for every common workflow live in
pg-ephemeral/examples —
each subdirectory has a database.toml plus a focused walk-through.
Place a database.toml in the working directory (or pass --config-file <path>).
File paths in the config are resolved relative to the config file's location, not the
process working directory. This means tests can be run from any subdirectory without
changing the paths in database.toml.
image = "17.1"
[instances.main.seeds.schema]
type = "sql-file"
path = "schema.sql"
[instances.main.seeds.data]
type = "script"
script = "psql -c \"INSERT INTO users (name) VALUES ('alice'), ('bob')\""
[instances.main.seeds.indexes]
type = "sql-file"
path = "indexes.sql"
[instances.main.seeds.dynamic]
type = "command"
command = "sh"
arguments = ["-c", "psql -c \"INSERT INTO users (name) VALUES ('dynamic-$RANDOM')\""]
cache = { type = "none" }| Field | Description |
|---|---|
image |
PostgreSQL version / image tag (e.g. "17.1") |
backend |
"docker", "podman", or omit for auto-detection (see below) |
cache_registry |
OCI registry prefix for cache images (e.g. "ghcr.io/myorg"). See Sharing cache across machines. |
ssl_config |
SSL configuration with hostname field (example) |
wait_available_timeout |
How long to wait for PostgreSQL to accept connections (e.g. "30s") |
When no backend is set in database.toml, pg-ephemeral uses
ociman (crates.io)
to auto-detect Docker or Podman. Individual users can override the default without changing the project
config:
| Method | Description |
|---|---|
OCIMAN_BACKEND env variable |
Set to "docker" or "podman" for explicit selection |
~/.config/ociman.toml |
Set default_backend = "podman" (or "docker") to change the preference |
The resolution order is: OCIMAN_BACKEND env variable, then
~/.config/ociman.toml, then auto-detection (Docker first, Podman fallback).
Set PostgreSQL server parameters per instance with an [instances.<name>.parameters]
table. Each entry is passed to the server as a -c <name>=<value> flag at container
launch, so any GUC settable on the command line is fair game (shared_preload_libraries,
work_mem, log_statement, …).
image = "17.1"
[instances.main.parameters]
shared_preload_libraries = "pg_cron"
log_statement = "all"
work_mem = "16MB"Parameters are folded into the seed cache key, so changing a parameter invalidates the cached images that depend on it and the affected seeds re-run.
When ssl_config is set, pg-ephemeral controls the ssl, ssl_cert_file, ssl_key_file,
and ssl_ca_file parameters itself; setting any of them in parameters is rejected at
launch. Parameters are per-instance only — there is no top-level parameters table.
Seeds run in declaration order inside the container. Whenever a seed step exits,
remaining database connections are terminated before the container is stopped.
Each seed has a type:
| Type | Fields | Description |
|---|---|---|
sql-file |
path, optional git_revision |
Apply a SQL file. With git_revision, reads the file from that git commit. path is resolved relative to the config file's directory. (example) |
sql-statement |
statement |
Apply an inline SQL statement. Equivalent to sql-file but the SQL is embedded directly in the config instead of read from disk. (example) |
csv-file |
path, table, optional delimiter |
Load a CSV file into a table using COPY ... FROM STDIN. The first row must be column headers matching column names in the target table; columns may appear in any order and omitted columns use their table defaults. The column delimiter defaults to , and can be overridden with delimiter. The line delimiter is hardcoded to \n. path is resolved relative to the config file's directory. table requires schema and table fields. (example) |
script |
script |
Run a shell script on the host with sh -e -c. PG environment variables are available. |
command |
command, arguments, cache |
Run an arbitrary command on the host. If command is a relative path (contains /), it is resolved relative to the config file's directory; bare names like psql are looked up via PATH. |
container-script |
script |
Run a shell script inside the container with sh -e -c. PostgreSQL is not running during execution. Use this to install extensions or perform other image customizations (example). |
Official PostgreSQL Docker images ship with contrib extensions but not third-party ones
like pg_cron, PostGIS, or pgvector. The container-script seed type installs packages
(or performs any other image customization) by running a script inside the container.
A runnable end-to-end version of the snippet below lives at
examples/06-container-script-pg-cron.
PostgreSQL is not started during a container-script seed. This avoids snapshotting dirty
database state (WAL files, pid files) into the cached image. The seed cache system builds
a new image via docker build with a generated Dockerfile, so installed packages persist
across runs as regular image layers.
Extensions that require shared_preload_libraries need the setting present before PostgreSQL
starts. Place an init script in /docker-entrypoint-initdb.d/ to configure this:
image = "17"
[instances.main.seeds.install-pg-cron]
type = "container-script"
script = """
apt-get update && apt-get install -y --no-install-recommends postgresql-17-cron \
&& printf '#!/bin/bash\necho "shared_preload_libraries = '"'"'pg_cron'"'"'" \
>> "$PGDATA/postgresql.conf"\n' \
> /docker-entrypoint-initdb.d/pg-cron.sh \
&& chmod +x /docker-entrypoint-initdb.d/pg-cron.sh
"""
[instances.main.seeds.enable-pg-cron]
type = "script"
script = 'psql -c "CREATE EXTENSION pg_cron"'Both seeds are cached. After the first run, subsequent invocations boot directly from the cached image with pg_cron already installed and enabled.
Define multiple named instances under [instances.<name>]. Top-level fields serve as
defaults for all instances. Use --instance <name> on the CLI to target a specific one.
See
examples/04-multi-instance
for a worked-out two-instance config.
pg-ephemeral caches seed results as container images so repeated runs skip already-applied
seeds. A walk-through with a chain-breaking cache = { type = "none" } seed lives at
examples/07-seed-cache.
Each seed's cache key is a SHA-256 chain of:
- pg-ephemeral version
- base image
- SSL configuration
- all preceding seeds' content
When the cache key matches an existing image the seed is a hit and the container boots
from that image directly. Seeds are cached in order; an uncacheable seed (e.g.
cache = { type = "none" }) breaks the chain and all subsequent seeds run without caching.
# Show cache status for all seeds
pg-ephemeral cache status
# JSON output (instance + version + summary rollup + per-seed status,
# cache_image, and reason/broken_by for uncacheable seeds)
pg-ephemeral cache status --json
# Print credentials baked into a cached seed image (no container booted).
# Defaults to the last declared seed; --seed-name picks an earlier layer.
pg-ephemeral cache credentials
pg-ephemeral cache credentials --seed-name schema
# Print the full pg-ephemeral metadata stored on a cached image
pg-ephemeral cache inspect pg-ephemeral/main:<sha>
# Pre-populate the cache without running an interactive session
pg-ephemeral cache populate
# Pull cache images from the configured registry (requires cache_registry)
pg-ephemeral cache pull
# Push locally-cached stages to the configured registry (requires cache_registry)
pg-ephemeral cache push
# Remove cached images
pg-ephemeral cache reset
# Force-remove cached images (even if referenced by stopped containers)
pg-ephemeral cache reset --forcecache status --json is intentionally lean — it reports cache state, not
full image manifests. Use cache inspect <reference> (with a reference
copied from cache status --json) when you need the embedded
superuser/seed-chain/SSL metadata.
For command and script type seeds, the cache field controls how the cache key is
computed. The seed's own content (command + arguments, or script body) is always folded
into the cache key; the strategy layers additional inputs on top:
| Strategy | Description |
|---|---|
{ type = "command-hash" } |
Cache key is the seed content only (default). |
{ type = "key-command", command = "...", arguments = [...] } |
Run a separate command; its stdout is folded into the cache key alongside the seed content. |
{ type = "key-script", script = "..." } |
Run a script; its stdout is folded into the cache key alongside the seed content. |
{ type = "none" } |
Disable caching. Breaks the cache chain for this and all subsequent seeds. |
By default, cache images are named pg-ephemeral/<instance>:<hex> and live
only in the local Docker/Podman image store. Set cache_registry to a remote
OCI registry prefix and every cache image gains that prefix, so references
become push/pullable addresses in a registry you can share across machines
(CI runners, developer laptops, production build hosts):
image = "17.1"
cache_registry = "ghcr.io/myorg"
[instances.main.seeds.schema]
type = "sql-file"
path = "schema.sql"The cache_registry value can be any valid OCI registry name — just a host
(ghcr.io), a host plus namespace (ghcr.io/myorg), or a private registry
(registry.example.com:5000/team). pg-ephemeral cache status will now
report references like ghcr.io/myorg/pg-ephemeral/main:<hex>.
The cache key hash is not affected by cache_registry. Two machines
pointed at different registries still compute the same hex for the same
content, and switching a project from no registry to a registry (or between
registries) does not invalidate any existing cache.
Once cache_registry is set, use the two dedicated subcommands to move
cache images between the local image store and the remote registry:
# Pull the newest cached stage from the registry that exists remotely.
# Walks the seed chain from tip backwards and stops on the first hit.
pg-ephemeral cache pull
# Populate anything still missing locally, then push everything that's
# now cached locally to the registry. Typical CI shape:
pg-ephemeral cache pull && pg-ephemeral cache populate && pg-ephemeral cache pushRegistry authentication is handled entirely by the underlying container
CLI (docker login, podman login, or cred-helper integration) — no
pg-ephemeral-specific setup required.
You can override the registry on a single invocation with --cache-registry
without editing database.toml:
pg-ephemeral --cache-registry ghcr.io/myorg cache pullA session is a long-running pg-ephemeral container kept alive between CLI
invocations. The normal flow tears the container down at the end of each
command; sessions skip that, letting you reuse one PostgreSQL instance across
many psql / script / shell invocations with full state continuity.
# Start a detached session named "foo" from the "main" instance
pg-ephemeral session start --name foo --instance main
# List running sessions
pg-ephemeral session list
# Attach interactively — same UX as the top-level bare commands
pg-ephemeral session psql --name foo
pg-ephemeral session shell --name foo
pg-ephemeral session run-env --name foo -- your-dev-tool
# Stop and remove
pg-ephemeral session stop --name fooAttached forms mirror the top-level surface. Bare session psql / shell /
run-env / schema-dump / pgbench run in transparent mode (cwd
bind-mounted, host-UID, in-container unix socket). Use session host <sub>
for a host-side process (TCP via published port) or session container <sub>
for an explicit in-container exec without the cwd bind mount.
The current working directory at session start time is bind-mounted into the
container — attached commands can read and write host files at the same path
they see on the host. Mounts are baked in at start; attaches from a cwd
outside that tree fail at the container's chdir layer.
A session is diverged when its stored seed-hash chain no longer matches
what the current database.toml would produce — typically because the base
image, a seed file, or a git-revisioned seed has changed since the session
started.
# Report sync/diverged status for every running session
pg-ephemeral session status
# Single session, vertical layout
pg-ephemeral session status --name fooDiverged sessions still work, but the data they hold reflects the config as it was at start time. To refresh, stop and start again:
pg-ephemeral session stop --name foo
pg-ephemeral session start --name foo --instance mainpg-ephemeral bin -- <tool> [args…] runs a tool from the configured image
against your working directory without booting PostgreSQL. The current
directory is bind-mounted into the container at the same path, the tool runs
there as the container entrypoint, and its stdout/stderr stream straight to
your terminal. No database is started.
This gives you the image's version-pinned tooling (pg_dump,
pg_verifybackup, pg_waldump, …) without installing it on the host — so the
PostgreSQL version your config already pins also governs the tools you run
against local files.
# Inspect the pinned tool / operate on local files (no database)
pg-ephemeral bin -- pg_dump --version
pg-ephemeral bin -- pg_verifybackup ./backup
# Pick the image via the instance or an explicit override
pg-ephemeral bin --instance reporting -- pg_dump --version
pg-ephemeral --image 17 bin -- pg_dump --versionUse -- to separate pg-ephemeral's own options from the tool and its
arguments.
bin is terminal-transparent: host stdin is forwarded and a PTY is allocated
when you run it from a terminal, so interactive tools and stdin piping behave
like a local install.
Your host PG* environment is forwarded into the container, so a tool reaches
the same routable database your shell is configured for with no extra
flags — PGHOST, PGUSER, PGPASSWORD, PGDATABASE, PGSSLMODE, … all
apply (host-specific variables like PATH / HOME are not forwarded):
# Uses the forwarded PG* env; writes the dump into the bind-mounted cwd
pg-ephemeral bin -- pg_dump -f dump.sqlSome PG* variables name host file paths (PGSSLROOTCERT, PGSSLCERT,
PGPASSFILE, PGSERVICEFILE, …). They are forwarded as-is, so they resolve
only when the file is reachable in the container at that path — i.e. inside
the working directory (bind-mounted at the same path). A cert or pass-file
under your cwd works; one elsewhere on the host (e.g. ~/.pgpass) will not be
found.
bin does not (yet) reach a host-local database — a tunnel bound to
localhost (SDM, cloud-sql-proxy, kubectl port-forward, …) — since the
container can't see the host's loopback. That's a planned follow-up; for an
interactive session against an ephemeral instance, use the bare psql /
shell commands.
pg-ephemeral can be used as a Rust library for integration tests or any code that needs a throwaway PostgreSQL instance.
async fn example() {
let backend = ociman::backend::resolve::auto().await.unwrap();
let definition = pg_ephemeral::Definition::new(
backend,
pg_ephemeral::Image::default(),
"test".parse().unwrap(),
)
.apply_file(
"schema".parse().unwrap(),
"schema.sql".into(),
)
.unwrap()
.apply_script(
"seed-data".parse().unwrap(),
r#"psql -c "INSERT INTO users (name) VALUES ('alice')""#,
pg_ephemeral::SeedCacheConfig::CommandHash,
)
.unwrap();
definition
.with_container(async |container| {
container
.with_connection(async |conn| {
let row: (i64,) = sqlx::query_as("SELECT count(*) FROM users")
.fetch_one(&mut *conn)
.await
.unwrap();
assert_eq!(row.0, 1);
})
.await;
})
.await
.unwrap();
}with_container handles the full lifecycle: populate the seed cache, boot a container
(from the latest cache hit if available), apply any remaining uncached seeds, run the
closure, and stop the container.
Seeds are added to a Definition via builder methods:
# async fn example() {
# let backend = ociman::backend::resolve::auto().await.unwrap();
let definition = pg_ephemeral::Definition::new(
backend,
pg_ephemeral::Image::default(),
"test".parse().unwrap(),
)
// Apply a SQL file from disk
.apply_file("schema".parse().unwrap(), "schema.sql".into())
.unwrap()
// Apply a SQL file from a specific git revision
.apply_file_from_git_revision(
"baseline".parse().unwrap(),
"schema.sql".into(),
"abc1234",
)
.unwrap()
// Apply an inline SQL statement
.apply_sql_statement(
"create-users".parse().unwrap(),
"CREATE TABLE users (id INT)",
)
.unwrap()
// Run an inline shell script with explicit cache strategy
.apply_script(
"seed-data".parse().unwrap(),
r#"psql -c "INSERT INTO users (name) VALUES ('alice')""#,
pg_ephemeral::SeedCacheConfig::CommandHash,
)
.unwrap()
// Run an arbitrary command with explicit cache strategy
.apply_command(
"migrations".parse().unwrap(),
pg_ephemeral::Command::new("migrate", ["up"]),
pg_ephemeral::SeedCacheConfig::CommandHash,
)
.unwrap()
// Run a script inside the container (for installing extensions, etc.)
.apply_container_script(
"install-pg-cron".parse().unwrap(),
"apt-get update && apt-get install -y --no-install-recommends postgresql-17-cron",
)
.unwrap();
# }The Definition builder supports additional options:
# async fn example() {
# let backend = ociman::backend::resolve::auto().await.unwrap();
let definition = pg_ephemeral::Definition::new(
backend,
"17.1".parse().unwrap(),
"test".parse().unwrap(),
)
// Extend the timeout for slow CI environments
.wait_available_timeout(std::time::Duration::from_secs(30))
// Enable cross-container access (for testing from other containers)
.cross_container_access(true)
// Set a PostgreSQL server parameter (passed as `-c name=value`)
.parameter("work_mem".parse().unwrap(), "16MB".parse().unwrap())
// Enable SSL with a generated certificate
.ssl_config(pg_ephemeral::definition::SslConfig::Generated {
hostname: "localhost".parse().unwrap(),
});
# }Inside with_container, the Container provides several ways to connect:
# async fn example() {
# let backend = ociman::backend::resolve::auto().await.unwrap();
# let definition = pg_ephemeral::Definition::new(
# backend, pg_ephemeral::Image::default(), "test".parse().unwrap(),
# );
definition
.with_container(async |container| {
// Direct sqlx connection
container
.with_connection(async |conn| {
sqlx::query("SELECT 1").execute(&mut *conn).await.unwrap();
})
.await;
// Get pg_client::Config for custom connection setup
let _config = container.client_config();
// Get libpq-style environment variables (PGHOST, PGPORT, etc.)
let _env = container.pg_env();
// Get DATABASE_URL string
let _url = container.database_url();
})
.await
.unwrap();
# }The pg-ephemeral Ruby gem bundles the binary and provides a native API:
# Yields a PG::Connection to an ephemeral database
PgEphemeral.with_connection do |conn|
conn.exec("SELECT 1")
end
# Or get the server URL for manual connection management
PgEphemeral.with_server do |server|
puts server.url # => "postgres://postgres:...@127.0.0.1:54321/postgres"
endThe gem is available for x86_64-linux, aarch64-linux, and arm64-darwin.
- RubyGems: pg-ephemeral
- Source and docs: integrations/ruby
The pg-ephemeral npm package installs the platform binary via optional
dependencies and provides a TypeScript API:
import { withConnection } from 'pg-ephemeral';
await withConnection(async (client) => {
await client.query('SELECT 1');
});Platform binaries are available for linux-x64, linux-arm64, and darwin-arm64.
- npm: pg-ephemeral
- Source and docs: integrations/npm
Any language can integrate via host run-env or the integration server protocol:
Environment variables — run a command with PG* and DATABASE_URL set:
pg-ephemeral host run-env -- python manage.py test
pg-ephemeral host run-env -- npx prisma migrate deploySee examples/05-run-env for a runnable end-to-end script.
Integration server — for programmatic control over the container lifecycle:
pg-ephemeral integration-server --result-fd 3 --control-fd 4Boots a container, writes a JSON line with connection details to the result pipe FD, then waits for EOF on the control pipe FD before shutting down. The parent process creates the pipes and passes the inherited file descriptors. Close the control pipe write end to stop the server.
pg-ephemeral [OPTIONS] [COMMAND]
Commands:
host Operations executed on the host (psql, run-env, shell, schema-dump)
container Operations executed inside the container (psql, run-env, shell, schema-dump)
psql Run interactive psql (default)
run-env Run a command with PG* and DATABASE_URL set
schema-dump Dump schema to stdout
shell Run an interactive shell
bin Run an image tool against the cwd, without booting PostgreSQL
host Operations executed on the host (psql, run-env, shell, schema-dump)
container Operations executed inside the container (psql, run-env, shell, schema-dump)
cache Cache management (status, credentials, inspect, populate, pull, push, reset)
session Named long-running sessions (list, start, stop, status,
psql/shell/run-env/schema-dump/pgbench, host, container)
integration-server Run integration server (pipe-based control protocol)
list List defined instances
meta Backend introspection (info)
platform Platform support checks
Bare commands (`psql`, `run-env`, `schema-dump`, `shell`) run in transparent
mode: cwd is bind-mounted into the container at the same path and the command
executes as the host user. Use `host <sub>` for a host-side process, or
`container <sub>` for an in-container exec without the cwd bind mount.
When invoked with no subcommand, pg-ephemeral defaults to `psql`.
Options:
--config-file <PATH> Config file path (default: database.toml)
--no-config-file Use defaults, ignore any config file
--backend <BACKEND> Override backend (docker, podman)
--cache-registry <NAME> Override cache_registry from config (e.g. ghcr.io/myorg)
--image <IMAGE> Override PostgreSQL image
--ssl-hostname <HOST> Enable SSL with the specified hostname
--instance <NAME> Target instance (default: main)
| Feature | pg-ephemeral | testcontainers |
|---|---|---|
| Seed caching | Content-addressed OCI image chain, only changed seeds re-run | None |
| Seed types | SQL files, git revisions, host commands, host scripts, container scripts | SQL files via Docker entrypoint init |
| Git-aware seeds | Seed from any git revision; apply migrations against schema from main |
No git integration |
| Extension installation | First-class container-script cached via docker build |
Manual custom image or exec |
| SSL/TLS | Auto-generated CA + server certs with verify-full | Manual certificate setup |
| Authentication | Random password per session, production-like auth | Static hardcoded password or trust mode |
| Version-controlled schema | Scripted pg_dump via CLI and Rust API |
Manual |
| CLI | psql, run-env, shell, cache management, schema-dump | Library only |
| Config files | TOML with per-instance overrides and path resolution | Programmatic only |
| Container runtime | Docker + Podman | Docker only |
| Multi-language integration | Single binary with FD-based integration protocol | Native library per language (30+ services each) |
testcontainers is a general-purpose container testing framework with a large ecosystem covering 30+ services and native libraries for Java, Go, .NET, Python, Node.js, and Rust. pg-ephemeral is purpose-built for PostgreSQL testing workflows with deep caching and seed management.
Runnable example projects covering common workflows live at pg-ephemeral/examples:
- 01-default — zero-config, default subcommand.
- 02-sql-file-seed —
sql-fileandsql-statementseeds. - 03-csv-load —
csv-fileseeds (default and tab delimiters). - 04-multi-instance — multiple named instances with overrides.
- 05-run-env —
run-envas the universal integration. - 06-container-script-pg-cron — install
pg_cronviacontainer-script. - 07-seed-cache — cache mechanics,
cache = { type = "none" }, JSON shape. - 08-ssl — SSL with auto-generated CA + verify-full.
- Docker Engine 20.10+ / Docker Desktop 4.34+, or Podman 5.3+
- PostgreSQL client tools (
psql) for host-side commands
Release builds use split-debuginfo = "packed" to separate debug information from the binary:
- Linux: Debug info stored in
.dwpfile alongside the binary - macOS: Debug info stored in
.dSYMbundle alongside the binary
This provides smaller binaries while preserving full backtraces with file paths and line numbers.