Skip to content

Commit 2752317

Browse files
Anmol1696Anmol1696
authored
Merge pull request #389 from constructive-io/anmol/docker-jobs
jobs: add docker compose for local jobs
2 parents 93df86b + 5c1350f commit 2752317

File tree

3 files changed

+295
-0
lines changed

3 files changed

+295
-0
lines changed

‎DEVELOPMENT_JOBS.md‎

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
# Jobs Development Setup
2+
3+
This guide covers a local development workflow for the jobs stack:
4+
5+
- Postgres + `launchql-ext-jobs`
6+
- LaunchQL API server
7+
- `simple-email` function
8+
- `knative-job-service`
9+
10+
It assumes:
11+
12+
- You have Docker / Docker Compose v2 installed.
13+
- You are using `pgpm` (not `lql`) for database initialization.
14+
- You have the latest `pgpm` installed (`npm i -g pgpm` or equivalent).
15+
16+
---
17+
18+
## 1. Start Postgres (and Minio)
19+
20+
From the `constructive/` directory:
21+
22+
```sh
23+
docker compose up -d postgres
24+
```
25+
26+
This uses `docker-compose.yml` and creates a Docker network called `constructive-net` that other services will join.
27+
28+
---
29+
30+
## 2. Configure your local Postgres env (pgenv)
31+
32+
Add this helper to your shell config (for example in `~/.zshrc`):
33+
34+
```sh
35+
pgenv() {
36+
export PGHOST=localhost
37+
export PGPORT=5432
38+
export PGUSER=postgres
39+
export PGPASSWORD=password
40+
export PGDATABASE=launchql
41+
echo "PostgreSQL environment variables set"
42+
}
43+
```
44+
45+
Then in a new shell (or after re-sourcing `~/.zshrc`), run:
46+
47+
```sh
48+
pgenv
49+
```
50+
51+
This ensures all subsequent `pgpm` and `psql` commands point at the same local database.
52+
53+
---
54+
55+
## 3. Bootstrap roles and database with pgpm
56+
57+
Make sure `pgpm` is installed and up to date.
58+
59+
From the `constructive/` directory (with `pgenv` applied):
60+
61+
1. Bootstrap admin users:
62+
63+
```sh
64+
pgpm admin-users bootstrap --yes
65+
pgpm admin-users add --test --yes
66+
```
67+
68+
2. Create the `launchql` database (if it does not already exist):
69+
70+
```sh
71+
createdb launchql
72+
```
73+
74+
3. Deploy the main app and jobs packages into `launchql`:
75+
76+
```sh
77+
pgpm deploy --yes --database "$PGDATABASE" --package app-svc-local
78+
pgpm deploy --yes --database "$PGDATABASE" --package db-meta
79+
pgpm deploy --yes --database "$PGDATABASE" --package launchql-database-jobs
80+
```
81+
82+
At this point, the app schema and `database-jobs` should be installed and `app_jobs.*` should be available in the `launchql` database.
83+
84+
---
85+
86+
## 4. Start jobs stack (API + worker + function)
87+
88+
With Postgres initialized, bring up the jobs-related services using `docker-compose.jobs.yml`:
89+
90+
```sh
91+
docker compose -f docker-compose.jobs.yml up
92+
```
93+
94+
Or run detached:
95+
96+
```sh
97+
docker compose -f docker-compose.jobs.yml up -d
98+
```
99+
100+
This starts:
101+
102+
- `launchql-server` – GraphQL API server
103+
- `simple-email` – Knative-style HTTP function
104+
- `knative-job-service` – jobs runtime (callback server + worker + scheduler)
105+
106+
By default, all three services use the published image:
107+
108+
```text
109+
ghcr.io/constructive-io/launchql:b88e3d1
110+
```
111+
112+
If you want to test a local build instead, build the image from the `constructive/` workspace and update `image:` in `docker-compose.jobs.yml` to point to your local tag, for example:
113+
114+
```sh
115+
docker build -t constructive-local .
116+
```
117+
118+
Then in `docker-compose.jobs.yml`:
119+
120+
```yaml
121+
image: constructive-local
122+
```
123+
124+
All services are attached to the shared `constructive-net` network and talk to the `postgres` container by hostname `postgres`.
125+
126+
---
127+
128+
## 5. Enqueue a test job (simple-email)
129+
130+
With the jobs stack running, you can enqueue a test job from your host into the Postgres container:
131+
132+
```sh
133+
docker exec -it postgres \
134+
psql -U postgres -d launchql -c "
135+
SELECT app_jobs.add_job(
136+
'00000000-0000-0000-0000-000000000001'::uuid,
137+
'simple-email',
138+
json_build_object(
139+
'to', '[email protected]',
140+
'subject', 'Hello from LaunchQL jobs',
141+
'html', '<p>Hi from simple-email (dry run)</p>'
142+
)::json
143+
);
144+
"
145+
```
146+
147+
You should then see the job picked up by `knative-job-service` and the email payload logged by the `simple-email` container in `docker compose -f docker-compose.jobs.yml logs -f`.
148+
149+
---
150+
151+
## 6. Inspect logs and iterate
152+
153+
To watch logs while you develop:
154+
155+
```sh
156+
docker compose -f docker-compose.jobs.yml logs -f
157+
```
158+
159+
Useful containers:
160+
161+
- `launchql-server`
162+
- `simple-email`
163+
- `knative-job-service`
164+
- `postgres` (from `docker-compose.yml`)
165+
166+
If you change Docker images, environment variables, or code inside the image, restart the stack:
167+
168+
```sh
169+
docker compose -f docker-compose.jobs.yml down
170+
docker compose -f docker-compose.jobs.yml up --build
171+
```
172+
173+
---
174+
175+
## 7. Stopping services
176+
177+
To stop only the jobs stack:
178+
179+
```sh
180+
docker compose -f docker-compose.jobs.yml down
181+
```
182+
183+
To stop everything, including Postgres and Minio:
184+
185+
```sh
186+
docker compose down
187+
```

‎docker-compose.jobs.yml‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
services:
2+
# LaunchQL API server (GraphQL)
3+
launchql-server:
4+
container_name: launchql-server
5+
image: ghcr.io/constructive-io/launchql:b88e3d1
6+
# The image entrypoint already runs the LaunchQL CLI (`lql`).
7+
# We only need to provide the subcommand and flags here.
8+
entrypoint: ["lql", "server", "--port", "3000", "--origin", "*", "--strictAuth", "false"]
9+
environment:
10+
NODE_ENV: development
11+
# Server
12+
PORT: "3000"
13+
SERVER_HOST: "0.0.0.0"
14+
SERVER_TRUST_PROXY: "true"
15+
SERVER_ORIGIN: "*" # allow all origins in dev
16+
SERVER_STRICT_AUTH: "false"
17+
# Postgres connection (matches postgres service)
18+
PGHOST: postgres
19+
PGPORT: "5432"
20+
PGUSER: postgres
21+
PGPASSWORD: password
22+
PGDATABASE: launchql
23+
# API meta configuration (static mode for dev)
24+
API_ENABLE_META: "true"
25+
API_EXPOSED_SCHEMAS: "collections_public,meta_public"
26+
API_ANON_ROLE: "administrator"
27+
API_ROLE_NAME: "administrator"
28+
API_DEFAULT_DATABASE_ID: "dbe"
29+
ports:
30+
- "3000:3000"
31+
networks:
32+
- constructive-net
33+
34+
# Simple email function (Knative-style HTTP function)
35+
simple-email:
36+
container_name: simple-email
37+
image: ghcr.io/constructive-io/launchql:b88e3d1
38+
# Override the image entrypoint (LaunchQL CLI) and run the Node function directly.
39+
entrypoint: ["node", "functions/simple-email/dist/index.js"]
40+
environment:
41+
NODE_ENV: development
42+
LOG_LEVEL: info
43+
# Mailgun / email provider configuration for @launchql/postmaster
44+
# Replace with real credentials for local testing.
45+
MAILGUN_API_KEY: "change-me-mailgun-api-key"
46+
MAILGUN_DOMAIN: "mg.constructive.io"
47+
MAILGUN_FROM: "[email protected]"
48+
ports:
49+
# Expose function locally (optional)
50+
- "8081:8080"
51+
networks:
52+
- constructive-net
53+
54+
# Jobs runtime: callback server + worker + scheduler
55+
knative-job-service:
56+
container_name: knative-job-service
57+
image: ghcr.io/constructive-io/launchql:b88e3d1
58+
# Override the image entrypoint and run the jobs runtime directly.
59+
entrypoint: ["node", "jobs/knative-job-service/dist/run.js"]
60+
depends_on:
61+
- simple-email
62+
environment:
63+
NODE_ENV: development
64+
65+
# Postgres (jobs extension lives in this DB)
66+
PGUSER: postgres
67+
PGHOST: postgres
68+
PGPASSWORD: password
69+
PGPORT: "5432"
70+
PGDATABASE: launchql
71+
JOBS_SCHEMA: app_jobs
72+
73+
# Worker configuration
74+
JOBS_SUPPORT_ANY: "false"
75+
JOBS_SUPPORTED: "simple-email"
76+
HOSTNAME: "knative-job-service-1"
77+
78+
# Callback HTTP server (job completion callbacks)
79+
INTERNAL_JOBS_CALLBACK_PORT: "8080"
80+
INTERNAL_JOBS_CALLBACK_URL: "http://knative-job-service:8080"
81+
82+
# Function gateway base URL (used by worker when no dev map is present)
83+
# Not used in dev when INTERNAL_GATEWAY_DEVELOPMENT_MAP is set, but required for validation.
84+
KNATIVE_SERVICE_URL: "dev.internal"
85+
INTERNAL_GATEWAY_URL: "http://simple-email:8080"
86+
87+
# Development-only map from task identifier -> function URL
88+
# Used by @launchql/knative-job-worker when NODE_ENV !== 'production'.
89+
# This lets the worker call the simple-email container directly in docker-compose.
90+
INTERNAL_GATEWAY_DEVELOPMENT_MAP: '{"simple-email":"http://simple-email:8080"}'
91+
92+
ports:
93+
- "8080:8080"
94+
networks:
95+
- constructive-net
96+
97+
networks:
98+
constructive-net:
99+
external: true
100+
name: constructive-net

‎docker-compose.yml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ services:
1313
- ./bin:/sql-bin
1414
- ./packages:/sql-packages
1515
- ./extensions:/sql-extensions
16+
networks:
17+
- constructive-net
1618

1719
minio:
1820
container_name: minio
@@ -25,3 +27,9 @@ services:
2527
expose:
2628
- "9000"
2729
command: server /data
30+
networks:
31+
- constructive-net
32+
33+
networks:
34+
constructive-net:
35+
name: constructive-net

0 commit comments

Comments
 (0)