Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,16 @@ Temporary Items
/web/cache/sessions/sess_*
*.smx

# Uploaded demo evidence (production) + synthetic demo files written by
# `./sbpp.sh db-seed` (dev). `.gitkeep` is the structural marker so the
# directory exists for fresh tarball installs and tests that probe
# `is_dir(SB_DEMOS)`. Both upload paths land MD5-hash basenames here;
# checking them in would (a) blow up the working tree on every db-seed
# and (b) leak real ban evidence into the source tree on a self-hoster
# install.
/web/demos/*
!/web/demos/.gitkeep

# Local stash file used by the install-screenshot capture flow
# (`docs/README.md` → "Capturing screenshots"). Gitignored so an
# interrupted run never leaks the panel's DB credentials into a
Expand Down
59 changes: 48 additions & 11 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -618,24 +618,61 @@ two idiomatic shapes (#1273):

`./sbpp.sh db-seed` populates the dev DB (`sourcebans`) with a deterministic,
realistic synthetic dataset across bans, comms, servers, admins, groups,
submissions, protests, comments, notes, banlog, and the audit log. Use it
when you need the panel surfaces (banlist, dashboard, drawer, moderation
queues, audit log) to render with real-looking data instead of empty
states. Acceptance audits and screenshot work both depend on it.
submissions, protests, comments (per-type: B / C / S / P), notes, banlog,
the audit log, AND demo evidence files (rows + on-disk files under
`SB_DEMOS`). Use it when you need the panel surfaces (banlist, dashboard,
drawer, moderation queues, audit log, "Review Demo" download flow) to
render with real-looking data instead of empty states. Acceptance audits
and screenshot work both depend on it.

- Lives at `web/tests/Synthesizer.php` (`Sbpp\Tests\Synthesizer`); CLI
driver at `web/tests/scripts/seed-dev-db.php`. Both are dev-only —
the synthesizer refuses any `DB_NAME` other than `sourcebans`, so
`sourcebans_test` / `sourcebans_e2e` stay untouched and the E2E
suite (which builds its own rows per spec) is unaffected.
- Idempotent: every run truncates the synth-owned tables first
(preserving `sb_settings` / `sb_mods` from `data.sql`), re-seeds the
CONSOLE + `admin/admin` rows, then inserts the synthetic dataset.
- Idempotent: every run snapshots the filenames currently referenced
by `:prefix_demos` BEFORE the TRUNCATE wipes the table, executes
the truncates (preserving `sb_settings` / `sb_mods` from `data.sql`),
unlinks the snapshotted files AFTER (so rows + files drop visibly
together from the panel's POV — closes a transient pre-fix window
where rows pointed at files that had just been removed), re-seeds
the CONSOLE + `admin/admin` rows, then inserts the new synthetic
dataset (rows AND on-disk demo payloads). The wipe scope is the
set of filenames currently in `:prefix_demos`; anything else in
`SB_DEMOS` (the tracked `.gitkeep`, manual panel uploads, a dev's
SDK demoviewer working copy) survives untouched. `./sbpp.sh
db-reset` carries a sibling cleanup that strips MD5-named
basenames from `web/demos/` (those rows vanished with the DB
volume so they'd otherwise be permanent orphans).
- Deterministic given a fixed `--seed` — `mt_srand($seed)` pins PHP's
RNG so two devs hit the same names/reasons/timestamps. Default seed
is `Synthesizer::DEFAULT_SEED` (pinned in code).
- Three tiers: `--scale=small` (~30 bans, fast iteration), `medium`
(default, ~200 bans), `large` (~2000 bans for pagination / perf).
RNG so two devs hit the same names/reasons/timestamps AND so
`array_rand` picks the same parent IDs for demo attachment across
runs. Demo filenames ride `md5(seed|demtype|demid)` so re-seeds
round-trip identically and the next run's cleanup pass finds +
unlinks the prior set cleanly (filenames stay 32-char lowercase
hex, indistinguishable from `UploadHandler::handle()`'s
`renameToHash` output). Default seed is `Synthesizer::DEFAULT_SEED`
(pinned in code).
- Three tiers: `--scale=small` (30 bans + 15 demos, fast iteration),
`medium` (default, 200 bans + 80 demos), `large` (2000 bans + 800
demos for pagination / perf). Comment budget per scale fans
exactly 60% Ban (`B`) / 20% Comm (`C`) / 10% Submission (`S`) /
10% Protest (`P`) (medium → 120/40/20/20); demo budget fans
exactly 80% ban-attached (`B`) / 20% submission-attached (`S`)
(medium → 64/16). Counts are exact (no rejection-sampling
shortfall) — `insertDemos()` uses `array_rand` for unique picks
upfront. The splits give each surface enough rows to render
without leaving the moderation-queue detail cards empty.
- Demo payloads are opaque ~1 KiB text blobs carrying the synth
marker + parent ID, served verbatim by `getdemo.php` with
`Content-Type: application/octet-stream`. They won't replay in the
SDK demoviewer (would need a real Source-engine demo binary);
they DO exercise the panel's "Review Demo" download chrome end-to-end
(link → 200 → file download with `Content-Disposition`). On-disk
files land in `web/demos/`, which is gitignored alongside the
production demo-upload directory (the panel's `UploadHandler` writes
there too with `renameToHash: true`) so synthetic and real-production
uploads share one storage shape.
- Do NOT reach for this from `Fixture::truncateAndReseed()` (the e2e
hot path) — those two diverge by design. The synthesizer is for
manual dev / screenshot work; the e2e fixture stays minimal so each
Expand Down
43 changes: 32 additions & 11 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,27 +86,48 @@ matching docs source file on disk.
./sbpp.sh e2e # Playwright E2E gate (lazy chromium install) against the running stack
./sbpp.sh db-dump backup.sql # mysqldump to host file
./sbpp.sh db-load fixtures.sql # pipe a SQL file into the DB
./sbpp.sh db-reset # drop just the DB volume and re-seed
./sbpp.sh db-seed # populate the dev DB with realistic synthetic data
./sbpp.sh db-reset # drop the DB volume + wipe MD5-named demo files from web/demos/
./sbpp.sh db-seed # populate the dev DB with realistic synthetic data + write demo files to web/demos/
./sbpp.sh rebuild # `--no-cache` rebuild of the web image
```

`db-seed` lights up every data-driven panel surface — banlist + commslist
beyond a single page, dashboard "Latest …" cards, the drawer's history /
comments / notes panes, admin moderation queues (submissions and
protests), admin audit log, multiple groups/admins/servers — without
touching `data.sql` / `struc.sql` (the install path stays minimal).
Idempotent: every run truncates the synthesizer-owned tables first and
re-seeds. Deterministic given `--seed=<int>` so two devs see the same
data; pinned default seed in code. Three scale tiers:
comments / notes panes (B/C/S/P comment types covered, so the moderation
queues and protest threads paint with real text), admin moderation
queues, admin audit log, multiple groups/admins/servers, and demo files
on disk under `web/demos/` (so the banlist's "Review Demo" link returns
an actual download body) — without touching `data.sql` / `struc.sql`
(the install path stays minimal). Idempotent: every run truncates the
synthesizer-owned tables first and re-seeds. Deterministic given
`--seed=<int>` so two devs see the same data; pinned default seed in
code. Three scale tiers:

```sh
./sbpp.sh db-seed # default scale (~200 bans, ~100 comms)
./sbpp.sh db-seed --scale=small # ~30 bans, fast iteration
./sbpp.sh db-seed --scale=large # ~2000 bans, pagination / perf
./sbpp.sh db-seed # default scale (~200 bans, ~100 comms, ~80 demos)
./sbpp.sh db-seed --scale=small # ~30 bans, ~15 demos, fast iteration
./sbpp.sh db-seed --scale=large # ~2000 bans, ~800 demos, pagination / perf
./sbpp.sh db-seed --seed=42 # alternate RNG seed
```

Demo files: each `:prefix_demos` row lands a paired ~1 KiB opaque file
in `web/demos/` (host bind-mount; gitignored alongside production
upload evidence). Filenames are MD5(seed|demtype|demid) so re-runs with
the same seed produce stable basenames and the truncate-time wipe can
clean them up by re-reading the table. The synth payload is text, not a
playable Source-engine demo — the panel chrome and `getdemo.php` stream
the bytes verbatim with `Content-Type: application/octet-stream`, which
is enough for the download affordance to work end-to-end, but the SDK
demoviewer won't replay them. Manual uploads (panel-side or anything
else not referenced by `:prefix_demos`) are never touched by the wipe.

`db-reset` drops the entire DB volume AND wipes MD5-named files from
`web/demos/` (orphans after the DB drop — the `_demos` rows that
pointed at them vanished with the volume). The wipe is scoped to
`[0-9a-f]{32}` basenames so `.gitkeep` stays put; non-MD5 manual
uploads (the icon / mapimg flows preserve original filenames) survive
too, though those don't normally land in `web/demos/`.

Refusal guard: the seeder strictly refuses any `DB_NAME` other than
`sourcebans`, so the PHPUnit DB (`sourcebans_test`) and Playwright DB
(`sourcebans_e2e`) cannot be wiped by accident. The Synthesizer source
Expand Down
19 changes: 18 additions & 1 deletion sbpp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ Run things inside containers:
Database:
db-dump [file] Dump the DB to file (default: dump-YYYYMMDD-HHMMSS.sql).
db-load <file> Load a SQL dump into the dev DB.
db-reset Drop the DB volume and re-seed (faster than full reset).
db-reset Drop the DB volume + wipe MD5-named files from web/demos/
(orphans after the DB drop). Faster than full reset.
db-seed [args] Populate the dev DB with realistic synthetic data.
Accepts --scale=small|medium|large (default medium) and
--seed=<int> (default pinned in code). Idempotent: every
Expand Down Expand Up @@ -228,6 +229,22 @@ case "$cmd" in
db-reset)
dc rm -fsv db
docker volume rm "$(basename "$PWD")_dbdata" 2>/dev/null || true
# `db-reset` wipes the DB but `web/demos/` is bind-mounted from
# the host worktree (not a named volume) so demo files written
# by a previous `db-seed` (or by any panel-side upload that
# happened against the dev stack) would otherwise survive as
# permanent orphans — the `_demos` rows that pointed at them
# vanish with the DB volume, and the next `db-seed`'s
# referenced-only wipe reads zero rows from the fresh DB and
# skips them. Strict MD5-name regex matches both shapes the
# demo write path produces (`Sbpp\Tests\Synthesizer::deterministicDemoFilename`
# for synth files, `Sbpp\Upload\UploadHandler::handle()` with
# `renameToHash: true` for panel uploads) while leaving the
# `.gitkeep` structural marker in place. Demos are dev-only
# ban evidence here; production installs never run `sbpp.sh`.
if [[ -d web/demos ]]; then
find web/demos -maxdepth 1 -type f -regextype posix-extended -regex '.*/[0-9a-f]{32}$' -delete 2>/dev/null || true
fi
dc up -d db
;;
db-seed)
Expand Down
Loading
Loading