Troubleshooting

Local development

Symptom	Cause and fix
network clashwithjpa-network declared as external, but could not be found	docker network create clashwithjpa-network. Once per machine.
The API throws on boot naming an env var	That key is missing from apps/server/.env. See Environment Variables.
Sign in fails with invalid_redirect_uri	http://localhost:3000/api/auth/callback/discord is not in the Discord app's OAuth2 redirect list.
Turnstile always fails	The site key and secret must be the matching pair. The test keys only validate against each other.
Schema changes have no effect	just generate writes the migration, just migrate applies it. Both are needed.
just db-reset left the app broken	The volume took the schema with it. Run just migrate again.
Type errors from packages/*-client after an API change	Those are generated by kubb. Regenerate them rather than editing by hand, see Repository Structure.
Drizzle Gateway cannot connect to Postgres	Use host jpa-db port 5432, not localhost:7101. Inside the container localhost is the container itself.
Drizzle Gateway forgot every saved connection	The drizzle-gateway-data volume was dropped. Saved connections live there, not in the database.

Production

Symptom	Cause and fix
network pangolin declared as external, but could not be found	docker network create pangolin. Same for clashwithjpa-network.
Traefik answers 404 page not found	No route matches that Host. The resource is missing or disabled in Pangolin, or you saved it less than 5 seconds ago.
502 Bad Gateway	The route exists but the target is unreachable. Usually the container is down, or it is not on the pangolin network. Check docker network inspect pangolin.
Browser shows a certificate warning	The wildcard was never issued. docker compose logs traefik in the pangolin directory. Nearly always a Cloudflare token with the wrong scope.
Site loads but every API call fails CORS	JPA_APP_URL / JPA_AUTH_URL do not match the real origins. Both are read at server start, so recreate the container after fixing them.
Discord login returns invalid_redirect_uri	The production callback URL is not in the Discord app's OAuth2 redirect list.
Uploaded images 404 in the browser but the upload succeeded	MINIO_PUBLIC_URL is wrong, or cdn.clashwithjpa.com points at port 9001 (the console) instead of 9000 (the S3 API).
Frontend changes do not appear after a deploy	PUBLIC_* variables are compiled in. Rebuild with just prod, do not just restart.
Errors never reach the Sentry dashboard	SENTRY_SPOTLIGHT=1 in apps/server/.env. Set it to 0 and recreate the container. A restart will not pick up an env file change.
An env file change had no effect	Docker reads env files at container creation, not at start. Use --force-recreate.
just prod fails on git pull	Local edits on the server. Never edit files in place there. The .env files are gitignored, everything else should come from a commit.

Useful reads

docker ps                                        # what is up
docker logs -f jpa-server                        # API logs
docker compose -f docker-compose.yaml -f docker-compose.prod.yaml --profile prod ps
docker network inspect pangolin --format '{{range .Containers}}{{.Name}} {{end}}'

To check what a running container actually has in its environment, which is not necessarily what is in the .env file on disk:

docker inspect jpa-server --format '{{range .Config.Env}}{{println .}}{{end}}' | grep SENTRY

Or just open logs.clashwithjpa.com, which is Dozzle showing all of the above in a browser.