Skip to content

feat: Jitsi support - #971

Open
nabossha wants to merge 8 commits into
attendee-labs:mainfrom
nabossha:feature/jitsi-adapter
Open

feat: Jitsi support#971
nabossha wants to merge 8 commits into
attendee-labs:mainfrom
nabossha:feature/jitsi-adapter

Conversation

@nabossha

Copy link
Copy Markdown

Adds a Jitsi bot adapter, so Attendee bots can join and record meetings on Jitsi
deployments. Closes #340, where the request was open with no owner and a maintainer
noted there were no technical obstacles and that an outside implementation was welcome.

Bumps version.json from 1.64.1 to 1.65.0 for the minor release.

Recognising a Jitsi URL

Jitsi is domainless: every platform above it in normalize_meeting_url_raw owns a fixed
domain, but a Jitsi room is just https://<any-host>/<room>. Matching on URL shape alone
would swallow unrelated links, so recognition is opt-in through a new
JITSI_MEETING_DOMAINS environment variable, a comma-separated allowlist defaulting to
meet.jit.si. Operators running kMeet, Hostpoint Meet or their own instance add their
hosts to it. The Jitsi branch is checked last, so Zoom, Google Meet and Teams always take
precedence, and the value is read per call rather than at import time so it can be
overridden per environment and per test.

Talking to the app, not the page

Jitsi is heavily whitelabelled — kMeet and Hostpoint Meet restyle the entire interface —
so a DOM-driven adapter would break per deployment. Everything after the prejoin screen
therefore goes through the app itself: join state, lobby knocking and password state are
read from window.APP.store, CONFERENCE_FAILED reasons are recorded off the
lib-jitsi-meet room, and participants, audio tracks and speech events come from
window.APP.conference. The DOM is used for exactly three prejoin controls (app-choice,
name input, join button), matched on data-testid attributes that are identical across
vanilla Jitsi, kMeet and Hostpoint Meet, plus one language-independent icon selector for
kMeet's "join from browser or from the app" screen.

Media is muted through #config.startWithAudioMuted=true&config.startWithVideoMuted=true
in the URL fragment rather than by clicking toggles, so the bot is never briefly live on
camera. Normalised meeting URLs are stored without fragments, so this cannot double up.

Joining as a guest

v1 joins as an anonymous guest only; there is no Jitsi login flow. A room password is
supplied per bot through a new jitsi_settings.room_password, validated by JSON schema
like the other platform settings blocks and documented in docs/openapi.yml. The same
value handles both locked rooms and lobbies, because room.join(password) bypasses an
enabled lobby as well. A missing password on a locked room fails as
could_not_join / room_password; a rejected one fails the same way after a ten second
grace period, which is needed because the passwordRequired state persists briefly after
a correct submission. A wrong password in the lobby case simply leaves the bot knocking,
where a moderator can still admit it. On vanilla Jitsi, where an authenticated moderator
must start the room, the bot surfaces the existing waiting_for_host failure.

Audio only

The payload sends per-participant audio as binary frame type 5, feeding the existing
audio-chunk transcription path, and mixed audio as type 3 when requested. The MP3 itself
is still captured by ffmpeg from the pulseaudio sink monitor, so the payload only taps
tracks and never mutes or reroutes audible playback. Speech start/stop events are derived
from DOMINANT_SPEAKER_CHANGED, which means one speaker at a time — Jitsi does not expose
concurrent per-participant speech state the way the Zoom SDK does. Empty dominant-speaker
ids are handled so a null speaker is never started.

Since Jitsi has no recording-permission concept, the bot moves straight to recording after
joining rather than waiting for a grant.

Two Chrome relaxations

Both are scoped to the bot's own browser, which only ever loads the configured
meeting URL:

  • --disable-features=LocalNetworkAccessChecks,PrivateNetworkAccessChecks — recent Chrome
    blocks a public page from opening ws://localhost:<port>, which is exactly the existing
    payload-to-python transport every web adapter relies on.
  • Page.setBypassCSP via CDP — kMeet, unlike Google Meet, ships a strict connect-src
    CSP that would otherwise block that same websocket.

Happy to gate either behind the meeting type if you would rather not have them apply
broadly.

Out of scope for this PR

No video recording (send_video is stubbed against a botOutputManager that arrives
later), no captions or transcription from Jitsi itself — Jigasi is not assumed, so
meeting_closed_captions defaults to "no external provider" and captions simply never
arrive — no signed-in or moderator bots, and one dominant speaker rather than concurrent
speech events.

Validation

Tested against three deployments: kMeet by Infomaniak and Hostpoint Meet with live
meetings, and the meet.jit.si API surface. The adapter has been running in production on
a self-hosted Attendee instance recording real kMeet meetings.

  • bots/tests/test_meeting_url_utils.py — URL recognition and normalisation, plus
    JITSI_MEETING_DOMAINS override behaviour
  • bots/tests/test_jitsi_bot.py — five integration tests tagged jitsi_tests: adapter
    factory dispatch, record-and-auto-leave when all participants leave, bot removed from
    meeting, missing room password, and the silence timeout
  • .github/workflows/ci.ymljitsi_tests added to the teams_and_zoom_web job (now
    teams_zoom_web_and_jitsi) and excluded from others, matching how the other platforms
    are split
  • ruff check and ruff format --check clean across the changed files

🤖 Generated with Claude Code

nandobosshart and others added 7 commits August 20, 2026 23:47
…AINS

Jitsi is domainless, so URLs are only recognized for domains explicitly
listed in the JITSI_MEETING_DOMAINS env var (comma-separated, default
meet.jit.si). Normalized URL is https://<domain>/<room> with query and
#config fragments stripped. Known platforms always take precedence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JitsiBotAdapter(WebBotAdapter, JitsiUIMethods) with guest-join flow via
the prejoin data-testids (identical on vanilla Jitsi, kMeet and Hostpoint
Meet) and join detection via the APP.conference api. Chromedriver payload
is a stub until the media/events payload lands. Audio: 48kHz float mixed
audio, no gstreamer pipeline (browser-side recording like Meet/Teams).
Default transcription settings map to meeting_closed_captions, which
never arrive on Jitsi — transcription happens downstream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Optional API field following the teams_settings pattern (schema + field +
validator, stored in bot.settings, model accessor, factory kwarg). Used to
join locked rooms and to bypass an enabled lobby.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Join detection is a poll loop over the app api and redux state (isJoined,
features/lobby.knocking, passwordRequired) plus a CONFERENCE_FAILED recorder
armed after the kMeet app-choice screen (that click reloads the page).
Error mapping: locked room without/with wrong password ->
UiIncorrectPasswordException; lobby denied -> UiRequestToJoinDeniedException
(event fast path + state fallback when knocking ends without a join);
lobby timeout -> UiCouldNotJoinMeetingWaitingRoomTimeoutException after
abort_join_attempt; authenticationRequired -> waiting-for-host timeout;
missing window.APP -> UiMeetingNotFoundException. Media stays muted via
config URL fragment. Name input is re-located per attempt and verified
(React remounts make held references stale).

All five paths verified end-to-end against real kMeet rooms with headless
Chrome and fake media devices.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bridges window.APP.conference to the websocket contract: UsersUpdate,
ChatMessage, ChatStatusChange(ready_to_send), SilenceStatus,
ParticipantSpeechStartStopEvent (from DOMINANT_SPEAKER_CHANGED) and
MeetingStatusChange as JSON, plus per-participant audio as binary frame
type 5 (480-sample float32 @48khz, participantId == deviceId) which drives
the existing audio-chunk transcription path. Remote audio is only tapped,
never muted — the MP3 is captured by ffmpeg from the pulseaudio sink.

Also needed for the payload's ws://localhost connection on kMeet:
- Page.setBypassCSP before navigation (kMeet ships a strict connect-src CSP)
- --disable-features=LocalNetworkAccessChecks (public page -> localhost)

Stale-room listeners are gated on the active room so a lobby rejoin cannot
end the session via a stale CONFERENCE_LEFT.

All events verified end-to-end against real kMeet rooms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Five tests following the teams/meet pattern, feeding real payload frames
through handle_websocket: adapter factory (incl. room_password), happy path
with only-participant auto-leave plus chat/participant db assertions, kick
(removed_from_meeting), locked room without password (COULD_NOT_JOIN /
FATAL_ERROR), silence auto-leave. Tagged jitsi_tests and wired into the
teams_zoom_web_and_jitsi CI leg (excluded from others).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@nabossha
nabossha requested a review from a team as a code owner August 21, 2026 08:42
@nabossha nabossha mentioned this pull request Aug 21, 2026
BotOutputManager.displayImage() draws the bot image onto the shared canvas
and calls turnOnWebcam. Wire turnOnWebcam/turnOffWebcam to
APP.conference.muteVideo(false/true): unmuting makes jitsi request
getUserMedia({video}), which the BotOutputManager interceptor answers with
the canvas track, so the bot image is published as a normal camera track.
Verified on kMeet and Hostpoint Meet with the real 1.2 MB bot image sent the
way WebBotAdapter.send_raw_image does; mute/unmute cycles cleanly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@nabossha

Copy link
Copy Markdown
Author

Pushed one more commit (d093aa5): the bot image now shows up as the bot's camera on Jitsi as well. turnOnWebcam/turnOffWebcam are wired to APP.conference.muteVideo(false/true) — unmuting makes jitsi request getUserMedia({video}), which the existing BotOutputManager interceptor answers with the canvas track, so displayImage() works without any DOM interaction. Verified on kMeet and Hostpoint Meet. Video playback (send_video) remains out of scope as described above.

@noah-duncan

noah-duncan commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Hi @nandobosshart thanks for this PR, it looks high quality at first glance. I can't make any guarantees about when we can merge this in as it is a big PR, demand for Jitsi support is almost non-existent among our paying customers and we currently have high demand for other things from our paying customers.

But Jitsi is philosophically in alignment with Attendee so we will try to get this merged in eventually.

Thank for your work!

Edit: Reading the PR in more detail you did a great job of not adding any complexity to the core logic, so this shouldn't be too hard to merge.

@nabossha

Copy link
Copy Markdown
Author

@noah-duncan thanks for the feedback. Would really appreciate a merge as there are multiple providers in Switzerland which over sovereign meeting-solutions based on Jitsi and they are becoming more popular quickly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Jitsi support

3 participants