Skip to content

Commit 1c8417c

Browse files
ericcurtinericcurtin
authored
docs(container-gateway): fix Docker driver setup for containerized gateway (NVIDIA#1419)
The existing docs omitted or misstated several requirements when running the gateway as a container with the Docker compute driver: - OPENSHELL_GRPC_ENDPOINT is required; the Docker driver uses only the scheme (http/https) — host and port are substituted automatically with host.openshell.internal and the gateway's own bind port - Supervisor binary must be extracted to a host path before starting the gateway; bind-mount sources are resolved by the host Docker daemon so the path must be identical inside and outside the gateway container - Docker socket access requires adding the docker group (UID 1000 default) - Port binding should remain 127.0.0.1; Docker driver adds a bridge listener automatically - add --server-san host.openshell.internal to generate-certs for mTLS - Complete the mTLS docker run with all Docker driver requirements - Add deploy/docker/gateway.toml — TOML config for the Docker driver - Add deploy/docker/docker-compose.yml referencing the TOML - Add docs/get-started/tutorials/docker-compose.mdx tutorial page - Remote gateway registration instructions (--remote flag) Address reviewer feedback: - Move Docker Compose tutorials card to the bottom of the list - Replace inline YAML snippet in Docker Compose section with a reference to deploy/docker/ to avoid drift - Clarify OPENSHELL_DB_URL is safe in compose.yml (plain SQLite path, no credentials); the TOML block targets credential-bearing DSNs - Note that ./ in source: resolves relative to the compose file directory - Clarify that only the scheme from OPENSHELL_GRPC_ENDPOINT matters - Add note that the tilde volume mount resolves to the same absolute path on both host and container
1 parent b7ce0be commit 1c8417c

5 files changed

Lines changed: 473 additions & 27 deletions

File tree

‎deploy/docker/docker-compose.yml‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
# OpenShell gateway — docker-compose setup (Docker compute driver)
5+
#
6+
# Prerequisites:
7+
# - Docker Desktop (Windows / macOS) or Docker Engine + Compose plugin (Linux)
8+
# - The openshell CLI installed on your workstation
9+
#
10+
# Quick start:
11+
#
12+
# 1. Start the gateway:
13+
# docker compose up -d
14+
#
15+
# 2. Register the gateway with the CLI (one-time):
16+
# openshell gateway add http://localhost:8080 --name openshell-docker
17+
#
18+
# 3. Configure an AI provider (example: Anthropic):
19+
# ANTHROPIC_API_KEY=sk-ant-... \
20+
# openshell provider create --type anthropic --from-existing
21+
#
22+
# 4. Create a sandboxed agent — Claude Code or OpenClaw:
23+
# openshell sandbox create -- claude
24+
# openshell sandbox create --from openclaw
25+
#
26+
# Sandbox containers are managed by the gateway, not by this Compose file.
27+
# Each `openshell sandbox create` call launches a fresh container; the gateway
28+
# tracks their lifecycle.
29+
#
30+
# Configuration:
31+
# All gateway and driver settings live in gateway.toml in this directory.
32+
# Three values cannot be expressed in the TOML file and remain as env vars:
33+
# - OPENSHELL_DB_URL (explicitly blocked from the config file to prevent
34+
# secrets from being committed to VCS)
35+
# - XDG_DATA_HOME / HOME (OS-level path-resolution vars outside the
36+
# gateway config schema)
37+
#
38+
# command: [] note:
39+
# The gateway image's default CMD is ["--bind-address", "0.0.0.0", "--port",
40+
# "8080"]. CLI flags beat TOML in the merge order, so without clearing the
41+
# CMD the TOML's bind_address = "127.0.0.1:8080" is silently ignored and the
42+
# gateway binds 0.0.0.0. Setting command: [] lets the TOML file own all
43+
# gateway settings.
44+
#
45+
# Data directory note:
46+
# /var/lib/openshell is bind-mounted at the SAME absolute path in both the
47+
# host and the container. This is required so that the supervisor binary
48+
# extracted from the supervisor image can be passed to Docker as a host-side
49+
# bind-mount source when sandbox containers are created. Named volumes
50+
# cannot be used here because Docker resolves bind-mount sources against the
51+
# host filesystem, not the container filesystem.
52+
#
53+
# Linux note:
54+
# host.docker.internal and host.openshell.internal are not automatically
55+
# added on Linux Docker. Add the following under the gateway service:
56+
# extra_hosts:
57+
# - "host.docker.internal:host-gateway"
58+
# - "host.openshell.internal:host-gateway"
59+
60+
services:
61+
gateway:
62+
image: ghcr.io/nvidia/openshell/gateway:${IMAGE_TAG:-latest}
63+
restart: unless-stopped
64+
65+
# Clear the default CMD so gateway.toml owns all settings (see note above).
66+
command: []
67+
68+
# This setup is Docker-outside-of-Docker (DooD), not Docker-in-Docker (DinD).
69+
# The gateway uses the host's Docker socket to create sibling containers on the
70+
# host, rather than running a nested Docker daemon. DooD does NOT require
71+
# --privileged; it only needs read/write access to /var/run/docker.sock.
72+
#
73+
# Run as UID 0 so the gateway can:
74+
# - write the extracted supervisor binary to /var/lib/openshell
75+
# - access /var/run/docker.sock (typically owned by root or the docker group)
76+
# Distroless images have no /etc/passwd, so the numeric UID must be used.
77+
# This is appropriate for local development. Production deployments
78+
# should use a dedicated non-root UID with explicit docker-group membership.
79+
user: "0"
80+
81+
ports:
82+
# gRPC / control-plane API (used by the openshell CLI and sandbox callbacks)
83+
# The Docker driver injects host.openshell.internal:<gateway-port> into sandbox
84+
# containers as the callback endpoint. The gateway's internal port is 8080, so
85+
# host port 8080 must be published at the same number so that
86+
# host.openshell.internal:8080 routes to the gateway container.
87+
# gateway.toml binds to 127.0.0.1 — the Docker driver adds the bridge listener
88+
# automatically so sandbox containers can reach the gateway without 0.0.0.0.
89+
- "127.0.0.1:${OPENSHELL_PORT:-8080}:8080"
90+
# Health endpoint (GET /healthz, GET /readyz)
91+
- "127.0.0.1:${OPENSHELL_HEALTH_PORT:-8081}:8081"
92+
93+
volumes:
94+
# Docker socket — lets the gateway create and manage sandbox containers.
95+
- /var/run/docker.sock:/var/run/docker.sock
96+
97+
# Data directory — must be a bind-mount with source == target so that
98+
# paths written inside the container are resolvable by Docker when it
99+
# creates sandbox containers (see note above).
100+
# /var/lib/openshell is intentionally not namespaced to a sub-path
101+
# (e.g. /var/lib/openshell/gateway): the path must match exactly on
102+
# both the host and inside the container, and a single gateway per host
103+
# is the expected topology.
104+
- type: bind
105+
source: /var/lib/openshell
106+
target: /var/lib/openshell
107+
bind:
108+
create_host_path: true
109+
110+
# TOML config — all gateway and driver settings live here.
111+
# Docker Compose resolves ./ relative to the directory that contains this
112+
# docker-compose.yml file (deploy/docker/), not the CWD of the caller.
113+
- type: bind
114+
source: ./gateway.toml
115+
target: /etc/openshell/gateway.toml
116+
read_only: true
117+
118+
environment:
119+
# Point the gateway at the TOML config file mounted above.
120+
OPENSHELL_GATEWAY_CONFIG: /etc/openshell/gateway.toml
121+
122+
# Database URL cannot be set in the TOML config file — it is explicitly
123+
# blocked there to prevent credential-bearing URLs (e.g. postgresql://user:pass@host/db)
124+
# from being committed to VCS. A plain SQLite path contains no credentials, so it
125+
# is safe to include here. Swap this for a real DSN via an env file or secret if
126+
# you switch to an external database.
127+
OPENSHELL_DB_URL: "sqlite:/var/lib/openshell/gateway.db?mode=rwc"
128+
129+
# XDG path variables are OS-level; they are not part of the gateway config
130+
# schema. Setting them ensures the extracted supervisor binary lands in the
131+
# bind-mounted directory so its path is resolvable by the host Docker daemon.
132+
XDG_DATA_HOME: /var/lib/openshell
133+
HOME: /var/lib/openshell

‎deploy/docker/gateway.toml‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
# OpenShell gateway TOML configuration — Docker compute driver.
5+
#
6+
# This file is the primary configuration source for docker-compose.yml in this
7+
# directory. It is mounted read-only at /etc/openshell/gateway.toml inside the
8+
# gateway container and loaded via OPENSHELL_GATEWAY_CONFIG.
9+
#
10+
# Why docker-compose.yml sets command: []:
11+
# The gateway image's default CMD passes --bind-address 0.0.0.0 --port 8080
12+
# as explicit CLI flags. CLI flags beat the TOML file in the merge order, so
13+
# bind_address = "127.0.0.1:8080" below would be silently ignored without
14+
# clearing the CMD first.
15+
#
16+
# grpc_endpoint note:
17+
# host.docker.internal is automatically resolvable from containers on
18+
# Docker Desktop (Windows / macOS). On Linux, add extra_hosts to the
19+
# gateway service:
20+
# extra_hosts:
21+
# - "host.docker.internal:host-gateway"
22+
# - "host.openshell.internal:host-gateway"
23+
24+
[openshell]
25+
version = 1
26+
27+
[openshell.gateway]
28+
# Bind to loopback only. The Docker driver adds an extra listener on the
29+
# bridge interface automatically so sandbox containers can reach the gateway.
30+
bind_address = "127.0.0.1:8080"
31+
health_bind_address = "127.0.0.1:8081"
32+
log_level = "info"
33+
compute_drivers = ["docker"]
34+
disable_tls = true
35+
36+
[openshell.drivers.docker]
37+
# Default image pulled for `openshell sandbox create` without --from.
38+
default_image = "ghcr.io/nvidia/openshell-community/sandboxes/base:latest"
39+
# Supervisor image from which the openshell-sandbox binary is extracted on
40+
# first start. The binary is cached to XDG_DATA_HOME and reused on restart.
41+
supervisor_image = "ghcr.io/nvidia/openshell/supervisor:latest"
42+
# Only pull images that are not already cached locally.
43+
image_pull_policy = "IfNotPresent"
44+
# Prefix applied to sandbox container names.
45+
sandbox_namespace = "openshell"
46+
# Address sandbox containers use to call back to the gateway.
47+
# The Docker driver replaces the host with host.openshell.internal and the
48+
# port with the gateway's own bind port (8080). Only the scheme survives.
49+
# The gateway must be published on port 8080 on the Docker host so that
50+
# host.openshell.internal:8080 resolves to the gateway container.
51+
grpc_endpoint = "http://host.openshell.internal:8080"

‎docs/about/container-gateway.mdx‎

Lines changed: 79 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -12,37 +12,89 @@ Use this approach when you want to run the OpenShell gateway as a container inst
1212

1313
The gateway image is published at `ghcr.io/nvidia/openshell/gateway`.
1414

15+
## Prerequisites for the Docker Driver
16+
17+
When the gateway runs as a container and creates Docker-backed sandboxes, the gateway container
18+
communicates with the host Docker daemon via the mounted socket. This requires three things beyond
19+
a basic `docker run`:
20+
21+
1. **Docker socket access.** The gateway process must be able to read and write the Docker socket.
22+
Add the `docker` group (or the GID of `/var/run/docker.sock`) so the socket is accessible
23+
without running as root.
24+
25+
2. **gRPC endpoint.** Sandbox containers call back to the gateway over the `OPENSHELL_GRPC_ENDPOINT`
26+
address. The Docker driver substitutes `host.openshell.internal` as the host and the gateway's
27+
own bind port as the port — only the **scheme** (`http` or `https`) is preserved. Use
28+
`http://host.openshell.internal:8080` when TLS is disabled and `https://host.openshell.internal:8080`
29+
when mTLS is enabled. The docker driver automatically binds the gateway to the bridge network
30+
interface so sandbox containers can reach it — you do not need to expose the port on `0.0.0.0`.
31+
32+
3. **Supervisor binary on the host.** The gateway bind-mounts the `openshell-sandbox` supervisor
33+
binary into each sandbox container. Because bind-mount paths are resolved by the host Docker
34+
daemon (not inside the gateway container), the binary must exist at a path on the **host**
35+
filesystem and be mounted at the **same absolute path** inside the gateway container. That way
36+
the path the gateway records internally matches what Docker can find on the host when it
37+
creates sandbox containers.
38+
1539
## Quick Start
1640

17-
This example runs the gateway locally with TLS disabled. It is suitable for development on a single machine. Binding to `127.0.0.1` prevents remote access without authentication.
41+
Extract the supervisor binary to the host once, then start the gateway:
42+
43+
```shell
44+
mkdir -p ~/openshell/supervisor
45+
docker create --name tmp-supervisor ghcr.io/nvidia/openshell/supervisor:latest
46+
docker cp tmp-supervisor:/openshell-sandbox ~/openshell/supervisor/openshell-sandbox
47+
docker rm tmp-supervisor
48+
chmod +x ~/openshell/supervisor/openshell-sandbox
49+
```
50+
51+
Start the gateway:
1852

1953
```shell
2054
docker run -d \
2155
--name openshell-gateway \
2256
--restart unless-stopped \
57+
--group-add docker \
2358
-p 127.0.0.1:8080:8080 \
2459
-v openshell-state:/var/openshell \
2560
-v /var/run/docker.sock:/var/run/docker.sock \
61+
-v ~/openshell/supervisor/openshell-sandbox:~/openshell/supervisor/openshell-sandbox:ro \
2662
-e OPENSHELL_DRIVERS=docker \
63+
-e OPENSHELL_GRPC_ENDPOINT=http://host.openshell.internal:8080 \
64+
-e OPENSHELL_DOCKER_SUPERVISOR_BIN=~/openshell/supervisor/openshell-sandbox \
2765
-e OPENSHELL_DB_URL=sqlite:/var/openshell/openshell.db \
2866
-e OPENSHELL_DISABLE_TLS=true \
2967
ghcr.io/nvidia/openshell/gateway:latest
3068
```
3169

32-
Register the gateway with the CLI:
70+
The volume mount uses `~/openshell/supervisor/openshell-sandbox` for both the host and container
71+
paths. The shell expands `~` in both halves before passing the argument to Docker, so both sides
72+
resolve to the same absolute path (e.g., `/home/user/openshell/supervisor/openshell-sandbox`).
73+
This satisfies the same-path requirement so the host Docker daemon can find the binary when
74+
creating sandbox containers.
75+
76+
Register the gateway with the CLI. If running on the same machine, use `--local`:
3377

3478
```shell
3579
openshell gateway add http://127.0.0.1:8080 --local --name local
3680
```
3781

82+
If registering from a different machine on the same network, use the host IP and `--remote`:
83+
84+
```shell
85+
openshell gateway add http://HOST_IP:8080 --remote --name remote
86+
```
87+
3888
Confirm the CLI can reach the gateway:
3989

4090
```shell
4191
openshell status
4292
```
4393

4494
<Warning>
45-
Disabling TLS removes authentication. Binding to `127.0.0.1` limits access to the local machine. If you expose the port on `0.0.0.0`, enable TLS and local mTLS user authentication, or put the gateway behind a trusted proxy with its own authentication.
95+
Disabling TLS removes authentication. This example binds to `127.0.0.1` so only local
96+
connections are accepted. To accept remote connections, enable mTLS or restrict access with
97+
a firewall rule.
4698
</Warning>
4799

48100
## Full mTLS Setup
@@ -58,7 +110,9 @@ docker run --rm \
58110
-v "$HOME/.local/state/openshell:/home/openshell/.local/state/openshell" \
59111
-v "$HOME/.config/openshell:/home/openshell/.config/openshell" \
60112
ghcr.io/nvidia/openshell/gateway:latest \
61-
generate-certs --output-dir /home/openshell/.local/state/openshell/tls
113+
generate-certs \
114+
--output-dir /home/openshell/.local/state/openshell/tls \
115+
--server-san host.openshell.internal
62116
```
63117

64118
This writes the server and client certificates under `~/.local/state/openshell/tls/`, writes sandbox JWT signing keys under `~/.local/state/openshell/tls/jwt/`, and copies the client bundle to `~/.config/openshell/gateways/openshell/mtls/` so the CLI picks it up automatically.
@@ -69,10 +123,14 @@ Start the gateway with mTLS enabled:
69123
docker run -d \
70124
--name openshell-gateway \
71125
--restart unless-stopped \
126+
--group-add docker \
72127
-p 127.0.0.1:8080:8080 \
73128
-v "$HOME/.local/state/openshell:/home/openshell/.local/state/openshell" \
74129
-v /var/run/docker.sock:/var/run/docker.sock \
130+
-v ~/openshell/supervisor/openshell-sandbox:~/openshell/supervisor/openshell-sandbox:ro \
75131
-e OPENSHELL_DRIVERS=docker \
132+
-e OPENSHELL_GRPC_ENDPOINT=https://127.0.0.1:8080 \
133+
-e OPENSHELL_DOCKER_SUPERVISOR_BIN=~/openshell/supervisor/openshell-sandbox \
76134
-e OPENSHELL_DB_URL=sqlite:/home/openshell/.local/state/openshell/openshell.db \
77135
-e OPENSHELL_LOCAL_TLS_DIR=/home/openshell/.local/state/openshell/tls \
78136
-e OPENSHELL_TLS_CERT=/home/openshell/.local/state/openshell/tls/server/tls.crt \
@@ -93,39 +151,33 @@ openshell gateway add https://127.0.0.1:8080 --local --name local
93151

94152
## Docker Compose
95153

96-
Save the following as `compose.yml`. This uses the TLS-disabled configuration bound to localhost, suitable for local development.
97-
98-
```yaml
99-
services:
100-
gateway:
101-
image: ghcr.io/nvidia/openshell/gateway:latest
102-
restart: unless-stopped
103-
ports:
104-
- "127.0.0.1:8080:8080"
105-
volumes:
106-
- openshell-state:/var/openshell
107-
- /var/run/docker.sock:/var/run/docker.sock
108-
environment:
109-
OPENSHELL_DRIVERS: docker
110-
OPENSHELL_DB_URL: "sqlite:/var/openshell/openshell.db"
111-
OPENSHELL_DISABLE_TLS: "true"
112-
113-
volumes:
114-
openshell-state:
115-
```
154+
The [`deploy/docker/`](https://github.com/NVIDIA/OpenShell/tree/main/deploy/docker) directory in
155+
the repository contains a production-ready Compose setup with full inline documentation:
116156

117-
Start the gateway:
157+
| File | Purpose |
158+
|---|---|
159+
| `docker-compose.yml` | Gateway service, volumes, and environment variables |
160+
| `gateway.toml` | TOML configuration mounted into the container |
161+
162+
Clone or copy those files, then start the gateway:
118163

119164
```shell
120-
docker compose up -d
165+
docker compose -f deploy/docker/docker-compose.yml up -d
121166
```
122167

123-
Register the gateway with the CLI:
168+
Register the gateway with the CLI. If registering from the same machine:
124169

125170
```shell
126171
openshell gateway add http://127.0.0.1:8080 --local --name local
127172
```
128173

174+
If registering from a different machine on the same network, replace `HOST_IP` with the
175+
machine's LAN address:
176+
177+
```shell
178+
openshell gateway add http://HOST_IP:8080 --remote --name remote
179+
```
180+
129181
## Using Podman
130182

131183
Replace `docker` with `podman` in the commands above. Mount the Podman socket instead of the Docker socket and set the driver to `podman`:

0 commit comments

Comments
 (0)