docs(shadow): trends + alerting guide, CLI reference, ADR-0018, roadmap v1.5

Author: abhay-codes07

docs/ROADMAP.md

@@ -132,4 +132,16 @@ regression test, replayed nightly; drift pages you before your users notice.
       empty corpus; `--report` / `--update-baseline` / `--threshold`
 - [x] Nightly GitHub Action template (`examples/workflows/volo-nightly.yml`)
 - [x] Acceptance: a seeded nondeterminism regression trips the alert (test-proven)
-- [ ] M14: drift trends + dashboard screens + webhook/Slack alerting
+
+## v1.5.0 — M14: drift trends + dashboard + alerting ✅
+Goal: the sentinel's memory — reliability over time, visible and loud.
+
+- [x] `SnapshotHistory` (append-only JSONL; every `shadow check` appends snapshot + drift verdict)
+      with fleet-average and per-trace trend series (ADR-0018)
+- [x] `volo shadow trend` — ASCII sparkline per dimension; `--trace` follows one banked trace
+- [x] Webhook alerting on `shadow check` (`--webhook` / `VOLO_SHADOW_WEBHOOK`) — Slack-compatible
+      payload, best-effort delivery (never masks the exit-3 alert)
+- [x] API: `GET /shadow/history` (+ per-trace) — the trend feed
+- [x] Dashboard: `/shadow` screen — fleet-average sparklines per dimension, drifted-night chips,
+      banked-corpus table (bible §8.3 aesthetic, reuses the CI sparkline)
+- [ ] M15: red-team corpus v1 (attack scenarios + safety annex on the report)

docs/adr/0018-shadow-trend-history.md

@@ -0,0 +1,47 @@
+# ADR 0018: trend history is append-only JSONL; alerts stay exit-code-first
+
+- Status: accepted
+- Date: 2026-07-04
+
+## Context
+
+M14 gives the drift sentinel (ADR-0017) a memory and a voice: reliability-over-time for the
+dashboard/CLI, and a push alert. The history format is a data contract (the API, dashboard, and
+users' tooling read it), so it deserves a recorded decision.
+
+## Decision
+
+1. **History is append-only JSONL** (`./.volo/shadow-history.jsonl`): one
+   `{at, snapshot, drift}` line per `volo shadow check`, including the baseline-establishing
+   run (`drift: null`). Torn lines are skipped on read, so a crashed run can't poison the
+   history. Committable and greppable, consistent with ADR-0017's no-database stance.
+2. **Two derived series, computed on read:** `fleet_series` (each dimension averaged across all
+   banked traces, per check — the dashboard headline) and `trace_series(run_id)` (one banked
+   trace over time). Nothing is precomputed or stored twice.
+3. **Alerting stays exit-code-first.** The webhook (`--webhook` / `VOLO_SHADOW_WEBHOOK`) is a
+   loud *secondary* path: a Slack-compatible payload (`text` headline + full report under
+   `volo`), delivered best-effort — a dead webhook logs a warning and never masks the exit-3
+   alert. Stdlib `urllib` only; no HTTP dependency.
+4. **Dashboard/API surface:** `GET /shadow/history` returns `{checks: fleet_series, corpus:
+   bank inventory}`; `GET /shadow/history/{run_id}` returns one trace's series. The `/shadow`
+   web screen renders fleet-average sparklines per dimension plus drifted-night chips, reusing
+   the CI sparkline component.
+
+## Consequences
+
+- The history file grows without bound (~1–2 KB per check per 10 traces); rotation/compaction
+  is deliberately deferred until a real corpus shows the growth rate. Append-only means
+  rotation is a safe external `mv`.
+- Averaging the fleet hides a single trace regressing among many healthy ones in the *chart*;
+  the alert does not average — `compare` runs per trace, so the exit code still fires. The
+  per-trace series exists for the drill-down.
+- Slack compatibility via `text` keeps zero config for the most common webhook; consumers
+  needing richer formats read the `volo` key.
+
+## Alternatives considered
+
+- **SQLite for history** — rejected for now: the query needs (append, scan) fit JSONL; SQLite
+  adds locking/migration surface for no current query we can't do in one pass.
+- **Alert-only-webhook (no exit code change)** — rejected: CI schedulers act on exit codes;
+  webhooks fail silently.
+- **Storing derived series** — rejected: two sources of truth; recompute is O(history) and fast.

website/cli.mdx

@@ -81,9 +81,18 @@ uv run volo shadow list
 ## `volo shadow check`
 Replay the whole corpus against the current agent build; compare the reliability surface to the
 last snapshot. Exit 0 = no drift, **exit 3 = drift alert** (dimension dropped > threshold or a
-verdict flipped). First run establishes the baseline.
+verdict flipped). First run establishes the baseline. Every check appends to the trend history;
+`--webhook` (or `VOLO_SHADOW_WEBHOOK`) POSTs a Slack-compatible alert on drift.
 ```bash
-uv run volo shadow check --agent <module:fn> [--threshold 0.05] [--report drift.json] [--update-baseline]
+uv run volo shadow check --agent <module:fn> [--threshold 0.05] [--report drift.json] \
+  [--update-baseline] [--history ./.volo/shadow-history.jsonl] [--webhook URL]
+```
+
+## `volo shadow trend`
+Reliability-over-time from the check history: one ASCII sparkline per dimension, fleet average
+by default, `--trace <run_id>` for a single banked trace. Feeds the `/shadow` dashboard screen.
+```bash
+uv run volo shadow trend [--history PATH] [--trace RUN_ID]
 ```
 
 ## `volo scenarios`

website/shadow.mdx

@@ -59,6 +59,36 @@ shadow check: 2 drift finding(s) -> ALERT
 `--threshold` tunes sensitivity (default 0.05), `--report drift.json` writes the machine-readable
 verdict, `--update-baseline` accepts the new surface as normal after an intentional change.
 
+## Trends: reliability over time
+
+Every check appends its snapshot to a JSONL history (default `./.volo/shadow-history.jsonl`), so
+the surface becomes a time series for free:
+
+```bash
+uv run volo shadow trend
+# shadow trend: fleet average (12 trace(s)) over 30 check(s)
+# shadow trend: decision_determinism            [@@@@@@@@@@@@@@@@@@@@@@@@=====] latest 0.500
+# shadow trend: faithfulness                    [@@@@@@@@@@@@@@@@@@@@@@@@@@@@@] latest 0.940
+# shadow trend: 5 check(s) drifted
+```
+
+`--trace <run_id>` follows one banked trace instead of the fleet average. The same history feeds
+the dashboard: the **Shadow** screen (`/shadow`) plots each dimension's fleet average per check,
+flags drifted nights, and lists the banked corpus.
+
+## Alerting
+
+Exit code 3 is the primary alert — any scheduler can page on it. For a loud path, add a webhook
+(Slack-compatible: incoming webhooks read the `text` field; the full drift report rides along
+under `volo`):
+
+```bash
+uv run volo shadow check --agent my_package.agent:run \
+  --webhook "$SLACK_WEBHOOK_URL"        # or set VOLO_SHADOW_WEBHOOK
+```
+
+Webhook delivery is best-effort by design: a dead webhook never masks the exit-3 alert itself.
+
 ## Run it every night
 
 A ready-made workflow lives at