feat(observability): stream scoped service health transitions

[tang-vu]

Why this matters

Automation currently has to poll full status payloads and infer whether service health actually changed. This adds an authenticated, cache-backed Server-Sent Events contract that emits only meaningful state transitions. It avoids probing containers per client, ignores noisy latency-only changes, supports resumable consumers, and can terminate after a bounded event count for scripts and health collectors.

Fleet consumers rarely own every service. Repeating service= now scopes the snapshot and transition fingerprint to the requested services, so an automation watching llama-server is not awakened by unrelated dashboard or extension churn.

Overlap check

Searched open and closed upstream PRs for SSE service health, service event stream, and filtered health streams, plus the changed production files. No active PR implements this endpoint or filter. PR #2906 is an independent pull-based Prometheus surface; PRs #2731, #2736, and #2819 change CSRF, key lifecycle, and Docker preflight behavior in other regions. This existing canonical branch was strengthened rather than creating a second observability PR.

What changed

• add authenticated GET /api/events/services with standard SSE framing
• emit deterministic state IDs, initial snapshots, and added/changed/removed transitions
• honor Last-Event-ID so reconnecting clients wait for the next real change
• repeat service=<id> to scope both snapshots and subsequent transitions
• emit heartbeat comments while idle and disable proxy buffering
• bound polling/heartbeat parameters and support max_events for finite automation
• consume the existing background health cache instead of multiplying network probes

Regression coverage

The HTTP boundary tests exercise authentication, SSE headers/framing, bounded snapshots, repeated service filters, and query bounds. Tracker/stream tests prove latency-only deduplication, add/change/remove transitions, cache readiness, and resumption after Last-Event-ID.

Concrete pre-change evidence: requesting service=open-webui&service=llama-server returned the unrelated dashboard-api state as well. The new regression now returns exactly the two requested services in deterministic order.

Validation

• python -m pytest -q tests/test_service_events.py tests/test_main.py — 82 passed.
• python -m py_compile routers/events.py tests/test_service_events.py main.py — passed.
• Earlier full dashboard-api run on this branch: 2,110 passed, 14 skipped; one unrelated Windows symlink test could not create a symlink due local WinError 1314.
• git diff --check — passed.

Tradeoffs and rollback

Filters match current service IDs exactly and unknown IDs produce an empty scoped snapshot; the endpoint does not guess aliases. Filtering happens after reading the shared cache, so it creates no additional probes. Rollback removes only the event API/consumer contract and does not mutate service state. The stream is authenticated and intentionally remains read-only.

[tang-vu]

Batch compatibility receipt (2026-08-24)

Built the inspectable synthetic branch tang-vu:validation/batch-20260824-architecture from upstream main (6ff9b4fc) and merged exact candidate heads in this order:

1. fix(dashboard): keep extension progress polls single-flight #3065 bad25e7f
2. feat(extensions): preview and confirm lifecycle change plans #2903 829dd48f
3. feat(observability): stream scoped service health transitions #2911 d8aed4c6
4. feat(extensions): inspect and gate dependency graphs #2909 6e9bd227
5. feat(release): publish commit-bound deterministic runtime SBOM #2894 9115d147
6. refactor(cli): extract read-only formatting primitives #3071 bff5cbfc

No manual merge resolution was required, including the shared Extensions.jsx / Extensions.test.jsx surface in #3065 and #2903. Synthetic head: 2c82aaec.

Combined validation:

• Dashboard: 28 files / 262 tests passed; ESLint passed; production Vite build passed
• Dashboard API: 350 passed, 5 skipped across service events, main routing, extension plans, lifecycle, and dependency suites
• dependency graph + manifest gates: 4 pytest cases, 8 shell cases, full canonical/library schema contract passed
• runtime SBOM: 5 tests passed; shipped dependency-pin validation passed
• CLI: JSON public boundary, Phase C smoke, Bash syntax, ShellCheck, 4/4 BATS formatting contract, and enable/reconciliation fixture passed

The stale Browserslist notice and authentication-dependent Phase C skips are unchanged environmental disclosures. #2903 remains draft because synthetic compatibility does not replace its declared live host-agent lifecycle gate and independent human review.