M7: run Element Call's own Playwright suite against Spindle

[hanthor]

The Element Call team already has an end-to-end Matrix RTC test suite, it runs in their CI on every PR, and it is swappable onto a different homeserver by changing one image reference. That is a better acceptance test for M7 than anything written here, because it is a real client driving a real call — and because passing it is the actual claim we want to make.

What they run

.github/workflows/test.yaml, the playwright job (checked at element-hq/element-call@a56e934):

- run: |
    docker compose -f docker-compose-dev.yml -f docker-compose-playwright.yml pull
    docker compose -f docker-compose-dev.yml -f docker-compose-playwright.yml up -d
- env: { USE_DOCKER: 1 }
  run: pnpm exec playwright test

The stack it brings up is two homeservers, two Element Web instances, a LiveKit SFU, and two lk-jwt-service instances — federation included, because some specs are cross-server.

The specs are the interesting part: create-call.spec.ts, reconnect.spec.ts, sfu-reconnect-bug.spec.ts ("When creator left, avoid reconnect to the same SFU"), restricted-sfu.spec.ts, access.spec.ts, errors.spec.ts, spa-call-sticky.spec.ts ("One to One call using matrix rtc 2.0 aka sticky events", "One to One rejoin after improper leave does not crash EC"), plus widget/ and mobile/.

"Rejoin after improper leave" is precisely the ghost-participant case #36 exists for, tested by the client that has to live with it.

The mechanism

docker-compose-playwright.yml overrides only the homeserver:

services:
  synapse:
    image: ghcr.io/element-hq/synapse:latest@sha256:da325af…
    volumes: [./backend/playwright_homeserver.yaml:/data/cfg/homeserver.yaml:Z]

So the work is a compose override pointing synapse and synapse-1 at Spindle's image, plus the config Spindle needs in place of playwright_homeserver.yaml.

The feature checklist, taken from their config rather than guessed

backend/playwright_homeserver.yaml is the exact set of homeserver features their suite requires:

Their setting	What it is	Spindle
msc3266_enabled	room summary	✅ served, advertised as im.nheko.summary
msc4222_enabled	state_after on sync	✅ served and advertised
max_event_delay_duration: 24h	MSC4140 delayed events	✅ #262/#264 — and the default cap is already exactly 24h
msc4143_enabled	MatrixRTC focus / m.rtc.member	❌ missing
msc4354_enabled	sticky events (Matrix RTC 2.0)	❌ missing
matrix_rtc.transports[].livekit_service_url	how a client discovers the SFU's JWT service	❌ missing
rc_delayed_event_mgmt	rate-limit bucket for delayed-event management	❌ no such bucket

Three of seven are already there, and the delayed-event one landed today. That table is the real M7 backlog, and it is checkable rather than inferred from the MSCs.

Staging

Getting the whole suite green is the end state, not the first PR.

1. Stand the stack up against Spindle and see what happens. A compose override plus a CI job, running only the specs that need login and room creation (landing.spec.ts, the sign-up/login half of access.spec.ts). This is worth doing before any of the missing features: it exercises .well-known, discovery, registration, login and room creation against a real client, and whatever breaks there breaks for every other spec too.
2. matrix_rtc discovery + MSC4143. Then a call can be created, which unlocks create-call.spec.ts.
3. MSC4354 sticky events. Unlocks spa-call-sticky.spec.ts, including the improper-leave case.
4. The federated specs, which need the second homeserver talking to the first.

Acceptance

• A docker-compose-spindle.yml override in this repository (not a fork of theirs — pin their revision the way scripts/complement.sh pins Complement's, so an upstream change cannot repaint our results without a diff here).
• A CI job that runs a named subset and an allowlist ratchet over it, in the same shape as complement-check.py — passing specs become protected, and a spec that starts passing is a reviewed promotion rather than an automatic one.
• The results table above kept honest: each row closed by the PR that implements it.

Why this over writing our own

Everything in crates/spindle-server/tests/delayed_events.rs is Spindle asserting things about Spindle. It is worth having and it is not evidence of interoperability. A suite written by the team whose client has to work against us, run unmodified, is.

Related: #36 (delayed events, partially closed by #262/#264), #35 (M7).

[hanthor]

Stage 1 is on main (#378): contrib/element-call/ puts Spindle in both of Synapse's seats in upstream's own two-homeserver Playwright stack, pinned at a03f23e, with scripts/element-call-check.py enforcing an allowlist ratchet in the shape of the Complement one, and an element-call-e2e job (nightly and on demand) in compliance.yml.

The feature table in this issue is now fully served on Spindle's side: MSC4143 transports and matrix_rtc discovery landed earlier, MSC4354 sticky events in #377, and rc_delayed_event_mgmt is moot with the limiter off in the rig config. The one remaining gap is the specs that create users through /_synapse/admin/v1/register with a shared secret (widget and restricted-SFU), which stay out of scope.

The allowlist starts empty because the rig was authored without a Docker host. Triggering the workflow now for the first recorded run; the passing specs from it get promoted in a follow-up.

---

Generated by Claude Code

[hanthor]

First recorded run, on main at 47c67e4 (workflow run 34014884679, chromium, element-call pin a03f23e):

element-call-check: 4 tests ran, 4 passed, 0 failed, 0 protected
  access.spec.ts :: As a guest, create a call, share link and other join
  access.spec.ts :: Sign up a new account, then login, then logout
  landing.spec.ts :: Landing page
  landing.spec.ts :: has title

The guest-call spec is the one that matters: a call created, the invite link shared, a second guest joining, and two video tiles on both pages. That is LiveKit, lk-jwt-service and Spindle's transport discovery composing in a real browser with no client patches, which stage 2 of this issue said would need MSC4143 first; it is already there. All four are being promoted into the allowlist (PR #380). Next subsets to name in ELEMENT_CALL_SPECS: create-call.spec.ts, spa-call-sticky.spec.ts (the improper-leave case now that #377 serves sticky events), reconnect.spec.ts.

---

Generated by Claude Code