Skip to content

Production Deployment

Jump to bottom
swayam25 edited this page Aug 26, 2026 · 3 revisions

Written so that someone who has never touched the box before can rebuild it from an empty server. It follows what is actually running, not a generic template.

The reverse proxy layer has its own page: Traefik and Pangolin.

What you are building

One VPS. Every public request enters through Traefik on :80 / :443, and Traefik decides where it goes based on routes that Pangolin hands it. Pangolin is the admin layer: a web dashboard where you say "api.clashwithjpa.com goes to container jpa-server on port 3000", and it turns that into Traefik configuration plus TLS plus optional login-in-front-of-it.

flowchart TD
    internet(["Internet"])

    subgraph edge["pangolin stack"]
        traefik["traefik<br/><i>:80 / :443, TLS termination</i>"]
        pango["pangolin<br/><i>dashboard :3002, API :3001/:3000</i>"]
    end

    subgraph app["clashwithjpa.com stack (prod profile)"]
        web["jpa-web<br/><i>nginx :80, prerendered SvelteKit</i>"]
        server["jpa-server<br/><i>Hono :3000</i>"]
        db[("jpa-db<br/>Postgres 18 :5432")]
        redis[("jpa-redis<br/>Redis 8 :6379")]
        minio[("jpa-minio<br/>MinIO :9000 / :9001")]
    end

    subgraph tools["standalone tools"]
        dozzle["dozzle :8080"]
        drizzle["drizzle-gate :4983"]
        backup["pgbackweb :8085"]
        rybbit["rybbit-client :3002<br/>rybbit-backend :3001"]
    end

    internet --> traefik
    traefik -->|polls every 5s| pango
    traefik --> web
    traefik --> server
    traefik --> minio
    traefik --> dozzle
    traefik --> drizzle
    traefik --> backup
    traefik --> rybbit
    web -->|browser calls| server
    server --> db
    server --> redis
    server --> minio
    backup -->|pg_dump| db
Loading

Two Docker networks do the work:

Network Created by Who is on it Why
pangolin you, by hand traefik, pangolin, jpa-web, jpa-server, jpa-db, jpa-minio, and every tool below Anything Traefik must reach, plus jpa-db so the backup tool can dump it
clashwithjpa-network you, by hand jpa-web, jpa-server, jpa-db, jpa-redis, jpa-minio The app's own private wiring. jpa-redis lives only here, nothing external needs it

Note

Both networks are declared external: true in their compose files, which means Compose will not create them for you. Creating them is a manual one-time step, covered in step 2. Skipping it fails with network <name> declared as external, but could not be found.

Prerequisites

What Type Notes
Ubuntu Required The live box is Ubuntu 24.04 LTS on arm64 with a public IPv4. Any modern Linux with Docker works.
Docker Required Engine plus the Compose plugin. Everything runs in containers.
Git Required just prod pulls the deployed commit through it.
Just Required Command runner behind every deploy and database recipe.
Cloudflare Required The domain's nameservers, and an API token Traefik uses to prove ownership for the wildcard certificate.
Pangolin Required Ingress manager. Installed in step 3, it brings Traefik with it.
Discord Required Sign-in is Discord OAuth only, see Environment Variables.
Clash of Clans Required Game data. Use the RoyaleAPI proxy so you do not need a static IP.
Turnstile Required Captcha on the application form. Real keys scoped to the live hostname, not the test pair.
Sentry Optional Error reporting. Two projects, one browser and one server.

Step 1: Point DNS at the box

In Cloudflare, on the zone for your domain, create two records pointing at the server's public IPv4:

Type Name Content Proxy status
A @ your server IP DNS only (grey cloud)
A * your server IP DNS only (grey cloud)

The wildcard is what lets you add a new subdomain in the Pangolin dashboard without touching DNS again. Every subdomain in this guide resolves through it.

Important

Keep both records DNS only. The live site is not proxied through Cloudflare, Traefik terminates TLS itself with a Let's Encrypt certificate. Turning the orange cloud on puts Cloudflare's certificate in front and changes which header carries the real client IP. The API already prefers cf-connecting-ip when it is present, so it would keep working, but nothing here depends on it and it is one more thing to get wrong.

Then create the API token Traefik will use for the certificate. Cloudflare dashboard → My Profile → API Tokens → Create Token → Edit zone DNS template:

  • Permissions: Zone · DNS · Edit
  • Zone Resources: Include · Specific zone · your domain

Copy the token once, it is not shown again. It goes in pangolin/.env in step 3.

Note

The certificate is issued over the DNS-01 challenge, not HTTP-01. That is the only challenge type that can issue a *.yourdomain.com wildcard, and it is why Traefik needs DNS write access rather than just port 80.

Step 2: Prepare the server

Install Docker Engine and the Compose plugin from Docker's own repository, not the distro package, then add yourself to the docker group so you are not typing sudo all day:

sudo usermod -aG docker $USER
newgrp docker

Install Just and make sure git and curl are present.

Create the two Docker networks. Nothing else in this guide starts until these exist:

docker network create pangolin
docker network create clashwithjpa-network

Now the firewall, and read this part carefully because it is the one thing people get wrong.

Only 80, 443 and your SSH port should be reachable from the internet. The app's compose file also publishes 7100 to 7110 on the host for debugging, and those must stay private.

Warning

A host firewall configured with ufw does not cover published Docker ports. Docker inserts its own DNAT rules and the traffic traverses the FORWARD chain, skipping the INPUT chain that ufw writes into. On the live box INPUT rejects everything except SSH, and 7100 to 7110 are still bound to 0.0.0.0 behind it.

What actually keeps them closed there is the cloud provider's firewall (an Oracle Cloud VCN security list). Use whatever equivalent your host gives you (AWS security group, Hetzner firewall, DigitalOcean cloud firewall) and allow only 80, 443 and SSH inbound. Verify from your laptop, not from the server:

for p in 80 443 7100 7101 7102 7105 7106 7110; do
    printf "%s: " "$p"
    timeout 5 bash -c "</dev/tcp/YOUR_SERVER_IP/$p" 2>/dev/null && echo OPEN || echo filtered
done

80 and 443 must be OPEN, everything else filtered. If a 71xx port comes back OPEN, your Postgres and MinIO are on the public internet right now.

If your provider has no network-level firewall, add rules to Docker's own DOCKER-USER chain instead of ufw, or bind the debug ports to loopback by changing 7101:5432 to 127.0.0.1:7101:5432 in apps/server/docker-compose.yaml.

Step 3: Pangolin and Traefik

Covered in full on its own page: Traefik and Pangolin. Come back here once https://pangolin.clashwithjpa.com loads and you have an admin account.

Step 4: Deploy the app

Clone the repo on the server:

git clone https://github.com/clashwithjpa/clashwithjpa.com jpa
cd jpa

Copy the three env files:

cp apps/server/.env.example apps/server/.env
cp apps/server/.env.server-db.example apps/server/.env.server-db
cp apps/web/.env.example apps/web/.env

Environment Variables documents every key, the production value for the ones that differ, and how to obtain the Discord, Clash of Clans and Turnstile credentials. Two settings there need updating to match the live hostnames:

  1. In the Discord application, OAuth2 → Redirects, add https://api.clashwithjpa.com/api/auth/callback/discord.
  2. In Cloudflare Turnstile, create a widget scoped to clashwithjpa.com and swap the test keys for the real pair.

Important

Set SENTRY_SPOTLIGHT=0 and PUBLIC_SENTRY_SPOTLIGHT=0. Left at 1, the server drops its Sentry DSN and sends errors to a Spotlight container that production does not run, so every server-side error is silently discarded.

Then deploy:

just prod

That recipe does four things: git pull, docker compose ... down, rebuild both images, and up -d --force-recreate with the prod profile and both docker-compose.yaml + docker-compose.prod.yaml layered together. The overlay file is what joins jpa-web, jpa-server, jpa-minio and jpa-db to the pangolin network so Traefik can see them.

Five containers come up:

Container Image Host port Container port
jpa-web clashwithjpa/web 7110 80
jpa-server clashwithjpa/server 7100 3000
jpa-db postgres:18-alpine 7101 5432
jpa-redis redis:8-alpine 7102 6379
jpa-minio minio/minio 7105 / 7106 9000 / 9001

Those host ports are for debugging over SSH and must not be reachable from the internet, see the firewall warning in step 2.

Two things happen automatically on first boot, so there is no manual step for either:

  • Migrations. The server's entrypoint runs pending Drizzle migrations before starting, on every boot. See Database.
  • The MinIO bucket. The server creates a bucket named uploads and applies a public-read policy to it at startup, in upload.ts. That public policy is deliberate: uploaded images are served straight from cdn.clashwithjpa.com to visitors.

Step 5: Publish the resources in Pangolin

Nothing is reachable yet. Traefik has no route for your domain until you add resources.

First, confirm the base domain is registered. In the dashboard, Domains. The installer adds it from config.yml, where it shows as Config Managed:

Pangolin domains page

Then Resources → Public, and add one per row below. For each: give it a name, pick the domain and subdomain, then add a target with the container name as the host and the container's internal port, not the 71xx host port. Traefik connects over the Docker network.

Domain Target host Port Serves Login required
clashwithjpa.com jpa-web 80 Prerendered SvelteKit site No
api.clashwithjpa.com jpa-server 3000 Hono API, with the Scalar docs at / No
cdn.clashwithjpa.com jpa-minio 9000 Uploaded files. Must match MINIO_PUBLIC_URL No
console.clashwithjpa.com jpa-minio 9001 MinIO admin console Yes

Once the standalone tools are added too, the full list looks like this:

Pangolin public resources page

Putting a login in front of a resource

Anything that is an admin surface should not be open to the internet. In the resource's settings, enable SSO. That attaches the Badger middleware, and any request without a valid Pangolin session gets a 302 to https://pangolin.clashwithjpa.com/auth/resource/<id>?redirect=... instead of the app.

On the live box that is on for the MinIO console, Dozzle, Drizzle Gateway and PG Back Web, and off for the site, the API, the uploads host and analytics. Access is granted through the Admin role.

Caution

The MinIO console, Drizzle Gateway (a database browser) and PG Back Web (which holds database dumps) each grant complete control of your data to anyone who reaches them. Turn SSO on for those three before you point DNS at the box, not after.

Give it five seconds after saving (that is Traefik's poll interval), then check:

curl -sI https://clashwithjpa.com | head -1
curl -sI https://api.clashwithjpa.com | head -1

Step 6: Analytics

The site loads a Rybbit tracking script, hardcoded in apps/web/src/app.html:

<script
    src="https://analytics.clashwithjpa.com/api/script.js"
    data-site-id="<site-id>"
    defer
></script>

If you are deploying your own instance, either point that at your own Rybbit and replace the site ID, or delete the line. Left as is, every visitor's browser makes a request to analytics.clashwithjpa.com.

Rybbit itself is a separate stack, see Server Services. It needs two targets on one resource, because the tracking endpoint and the dashboard are different containers:

Path Target host Port
/api/* rybbit-backend 3001
(default) rybbit-client 3002

Step 7: Backups

PG Back Web runs as its own stack and reaches jpa-db:5432 over the pangolin network. That is the reason jpa-db is attached to a network it otherwise would not need. Add the database as a connection there, then a destination and a schedule with a retention window. Restoring is covered in Database.

Also worth backing up, separately from the database:

  • pangolin/config/db/db.sqlite: every resource, user and role
  • pangolin/config/letsencrypt/acme.json: certificates and the ACME account key
  • the three .env files in the app repo, which are gitignored and exist nowhere else
  • the jpa-minio-data Docker volume, which holds every uploaded image

Updating

just prod

Same command as the first deploy and safe to rerun: pull, rebuild, recreate, migrations included.

For the other stacks, docker compose pull && docker compose up -d inside their own directory.

Tip

Editing a .env and running docker restart does not apply the change. Docker reads env files when a container is created, not when it starts. just prod already passes --force-recreate, so use it. For a single service without pulling:

docker compose -f docker-compose.yaml -f docker-compose.prod.yaml \
  --env-file apps/server/.env --env-file apps/web/.env \
  --profile prod up -d --force-recreate --no-build jpa-server

Clone this wiki locally