Dev#606
Merged
Merged
Conversation
Adds a self-contained pipeline that captures the current hand-crafted
demo state (78 users, 288 posts, 6 active + 48 past mentorships, 45
ratings, 100 messages, 389 binary uploads) into a portable snapshot
and replays it on a fresh DB without any HTTP/API calls.
- scripts/seed.sh: single bash entry point (setup/dump/load/status).
Bootstraps a local .venv on first run, pre-flights docker, and
dispatches to seed_snapshot.py.
- scripts/seed_snapshot.py: psycopg COPY + docker-cp based dump/load.
Writes seed-owned rows in FK order, restores backend uploads tree.
Idempotent — DELETE-then-COPY semantics on load.
- scripts/seed_snapshot_data/: committed snapshot artifact (~109 MB).
Lets a fresh checkout reach demo state in ~1.5 seconds.
- scripts/seed_demo_*.py: the full Phase A-V builder that produced
the snapshot, kept around for future regenerations. Phase A now
auto-applies coordinates + backfills bio/goals/availability from
the markdown corpus so the snapshot is reproducible end-to-end.
- docker-compose.yml: three new env defaults required by the seed
flow (APP_RATELIMIT_ENABLED, FOLLOW_GRAPH_SYNC_ENABLED, FOLLOW_RANKER).
- .gitignore: ignore .venv/, .env.*, seed-pipeline runtime artifacts.
Fresh-machine flow:
docker compose up -d
./scripts/seed.sh load
Add a docker-compose service that loads the snapshot into Postgres and
into the shared uploads volume when the stack is brought up with
`--profile demo`. Without the flag the service is dormant.
- docker-compose.yml: new `seed` service, profile-gated. Uses
python:3.12-slim, depends on backend health, mounts the scripts/
tree read-only plus the uploads_data volume, runs seed_snapshot.py
in `--uploads-mode local` so it writes directly to the volume
(no docker daemon access inside compose).
- scripts/seed_snapshot.py: support running inside compose. DSN now
auto-detects POSTGRES_HOST (so the same script works from the host
and from inside the `seed` service). New `--uploads-mode local`
flag short-circuits the `docker cp` path and copies files with
shutil instead.
- README.md: document the `--profile demo` flow plus the host-side
`./scripts/seed.sh load` wrapper.
Fresh-machine deploy flow now compresses to:
docker compose --profile demo up -d
…and the stack comes up already seeded with the hand-crafted demo
state. Without the profile, behaviour is unchanged: clean DB.
Without these flags the `--profile demo` stack boots, but the UI
quietly falls back to the legacy ranker, blank mentor explanations,
and a 401 on any registration attempt. Make the demo defaults explicit
and explain each knob.
- .env.example: pre-set every flag the demo dataset wants
(APP_RATELIMIT_ENABLED=false, APP_EMAIL_ENABLED=false,
APP_SPAM_ENABLED=false; MENTOR_ADVANCED_RANKER=true,
MENTOR_EXPLANATION_ENABLED=true, MENTOR_EXPLANATION_TIMEOUT_MS=15000;
FEED_ADVANCED_RANKER=true; FOLLOW_RANKER=advanced). The
docker-compose `:-` fallbacks still ship prod-safe defaults, so
forgetting to copy .env.example yields a prod-shaped boot.
- README.md: new "Environment variable reference" section with three
tables — seed-friendliness, advanced ranker stack, and keys to set
manually (chiefly OPENAI_API_KEY for LLM explanations + semantic
affinity).
Re-runnable demo flow now matches the snapshot's intent end-to-end:
cp .env.example .env
docker compose --profile demo up -d
Two friction points uncovered while dry-running the README on a fresh
checkout:
1. Quick start and the seed section listed two different commands
(`docker compose up --build` and `docker compose --profile demo up -d`).
Collapse them into one canonical fresh-clone command:
docker compose --profile demo up --build -d
`--profile demo` enables the one-shot seed service; `--build` covers
the first-run image build (backend, frontend, custom Neo4j-GDS).
Omit the profile to get a clean DB instead.
2. `docker compose down` without `--profile demo` leaves the seed
container orphaned with a stale network reference; the next `up`
then fails with "network not found". Document explicitly that the
teardown command must include the profile too, and surface the
`run --rm seed` recipe for re-seeding against a running stack.
Also collapse the duplicated "Seed local data" block into a single
"Seeding options" section that points back at the Quick start for the
primary command and only lists the host-side wrapper / legacy seeder
as alternates.
The production droplet runs the same docker-compose.yml as local dev (bundled Postgres + Neo4j, prod-grade .env), so the demo profile works identically there. Wire it into the existing deploy workflow: - .github/workflows/deploy.yml: add `--profile demo` to the compose invocation so the one-shot `seed` service runs after the backend reports healthy. Also pin command_timeout: 30m explicitly (it had rotted out of this branch but is needed for cold rebuilds), and add `set -e` so the workflow fails loudly if any step breaks. - README.md: rewrite the Production deployment section to describe the actual deploy path (GitHub Actions → SSH → docker compose --profile demo up --build -d) and list the env flags the droplet needs to surface the full advanced-ranker UI. The standalone docker-compose.prod.yml flow stays documented as the hardened alternative. The seed is idempotent and scoped to %@seed.test rows. Real users registered through the UI on the live site are never touched.
The orphan-cleanup queries in AttachmentRepository previously checked
only `messages` and `feed_post_attachments`, but four tables hold a
foreign key to `attachments(id)`:
- messages.attachment_id (V15, ON DELETE SET NULL)
- feed_post_attachments.attachment_id (V41, NO ACTION)
- task_assignment_attachments.attachment_id (V23, ON DELETE CASCADE)
- task_submission_attachments.attachment_id (V23, ON DELETE CASCADE)
Because the task-attachment junctions cascade on attachment delete,
the daily 03:15 UTC orphan sweep would silently drop task assignment
PDFs and task submission files older than the 24h retention window:
1. `findOrphansOlderThan` returns the attachment row as a candidate
(passes the messages/feed_post_attachments NOT EXISTS check).
2. `deleteIfStillOrphan` deletes the attachment row.
3. The CASCADE on `task_*_attachments.attachment_id` drops the
junction row, removing the task's file reference entirely.
4. Result: the file vanishes from the task detail view on both
web and mobile, with no audit trail.
Add `NOT EXISTS` clauses for both task junctions to both queries, and
expand the javadoc to enumerate the canonical list of FK sources so
future schema additions stay in sync.
No migration is needed (the queries are in code, not in
flyway_schema_history). Existing orphan rows in production that have
already been swept are unrecoverable, but the seed snapshot
re-uploads its task PDFs on every deploy so the demo dataset
self-heals.
feat(scripts): reproducible demo seed via snapshot dump/load
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.