Skip to content

Commit

Permalink
Merge branch 'master' into XEP-0045_Status-code-purpose-wording
Browse files Browse the repository at this point in the history
  • Loading branch information
@iNPUTmice
iNPUTmice authored Aug 23, 2024
2 parents 0c658e8 + 4677fc1 commit 17ded3f
Show file tree
Hide file tree
Showing 5 changed files with 431 additions and 29 deletions.
284 changes: 284 additions & 0 deletions inbox/av_conferences.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,284 @@
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE xep SYSTEM 'xep.dtd' [
<!ENTITY % ents SYSTEM 'xep.ent'>
%ents;
]>
<?xml-stylesheet type='text/xsl' href='xep.xsl'?>
<xep>
<header>
<title>Jingle Audio/Video Conferences</title>
<abstract>This specification defines a way to hold multiparty conferences with an SFU via Jingle.</abstract>
&LEGALNOTICE;
<number>xxxx</number>
<status>ProtoXEP</status>
<type>Standards Track</type>
<sig>Standards</sig>
<approver>Council</approver>
<dependencies>
<spec>XMPP Core</spec>
<spec>XEP-0001</spec>
<spec>XEP-0166</spec>
<spec>XEP-0167</spec>
<spec>XEP-0298</spec>
</dependencies>
<supersedes/>
<supersededby/>
<shortname>av-conferences</shortname>
<author>
<firstname>Jérôme</firstname>
<surname>Poisson</surname>
<email>[email protected]</email>
<jid>[email protected]</jid>
</author>
<revision>
<version>0.0.1</version>
<date>2024-07-29</date>
<initials>jp</initials>
<remark><p>First draft.</p></remark>
</revision>
</header>

<section1 topic='Introduction' anchor='intro'>
<p>Audio/Video calls are possible with a single destination via &xep0166; and &xep0167;, and associated XEPs. It may be desirable to have calls with multiple people at the same time. Three main strategies exist to achieve that:</p>
<ul>
<li><p>A mesh network where each peer contacts each other peer: this is straightforward to implement and doesn't require any extra service, but it is heavy on network bandwidth and doesn't scale.</p></li>
<li><p>An MCU (Multipoint Conferencing Unit) which uses a central service that generally mixes the contents of participants: this is lighter on network bandwidth but heavy on computing resources and hard to scale.</p></li>
<li><p>A SFU (Selective Forwarding Unit) which uses a service that redistributes streams of participants as efficiently as possible: this is lighter on computing resources, acceptable in terms of network bandwidth, and can scale.</p></li>
</ul>
<p>The mesh strategy is covered by &xep0272;. There have been attempts to specify MCU and SFU strategies, notably with &xep0340;, but it is unmaintained and unused.</p>
<p>This specification proposes a simple way to implement a SFU strategy with a service called in a similar way to one-to-one calls.</p>
</section1>
<section1 topic='Requirements' anchor='reqs'>
<p>The design goals of this XEP are:</p>
<ul>
<li>Be as simple as possible to implement for clients that already implement one-to-one A/V calls.</li>
<li>Be compatible with simulcast.</li>
<li>Use a simple way to identify the originating XMPP entity of a stream.</li>
<li>Use a simple way to configure the A/V conference room.</li>
</ul>
</section1>

<section1 topic='Glossary' anchor='glossary'>
<ul>
<li><strong>A/V</strong>: Audio/visual.</li>
<li><strong>SFU</strong>: Selective Forwarding Unit, a service that redistributes streams of participants as efficiently as possible.</li>
<li><strong>SFU service</strong>: An XMPP entity that services A/V conferences with a SFU.</li>
<li><strong>Room</strong>: A JID of an SFU service associated with a specific multiparty call.</li>
<li><strong>Joining entity</strong>: An XMPP entity that participates in an A/V conference room.</li>
<li><strong>Simulcast</strong>: Serving multiple versions of the same media content at different quality levels simultaneously, allowing the receiving end to select the most suitable version.</li>
</ul>
</section1>

<section1 topic='Overview' anchor='overview'>
<p>A/V conferences works using a Jingle session per stream: all sessions use unidirectional streams. An A/V conference is joined by calling the JID of an conference entity, and by sending our own stream. The conference service then call us for each stream it wants to send us.</p>
<p>&xep0298; is used to associate stream with the originating entity.</p>
<p>&xep0050; is used with a well-known node to configure the room.</p>
</section1>
<section1 topic='Overview' anchor='overview'>
<p>A/V conferences work by using a Jingle session per stream: all sessions use unidirectional streams. To join an A/V conference, a client calls the JID of a conference entity and sends its own stream. The conference service then contacts the client for each stream it wants to send.</p>
<p>&xep0298; is used to associate a stream with the originating entity.</p>
<p>&xep0050; is used with a well-known node to configure the room.</p>
</section1>


<section1 topic='Joining a Conference' anchor='joining'>
<p>To join a conference, a user calls the conference room JID using &xep0167; as for a one-to-one call. The calling entity sends its audio/video streams (typically webcam and microphone) in an unidirectional way. It MUST have its content 'senders' attribute set to "initiator" as explained in &xep0166; <link url="https://xmpp.org/extensions/xep-0166.html#def-content">§ 7.3</link>.</p>
<p>A joining entity MAY send the same audio or video stream with different quality for simulcast purposes. It is up to the SFU service to then select which one to forward.</p>
<p>A SFU service MUST use &xep0298; and notably it MUST set the 'isfocus' attribute of the &lt;conference-info&gt; element to "true".</p>
</section1>

<section1 topic='Receiving Streams' anchor='receiving'>
<p>The SFU service MAY call the joining entity as many times as necessary using &xep0167; Jingle sessions from the joined room. The joining entity SHOULD accept each session coming from a joined room. Each session MUST be unidirectional (stream sent from SFU service to joining entity) and MUST have its content 'senders' attribute set to "initiator" as explained in &xep0166; <link url="https://xmpp.org/extensions/xep-0166.html#def-content">§ 7.3</link>. Those streams are the streams of other participants as selected by the SFU.</p>
<p>To identify the streams' origin, the SFU service must specify it using &xep0298;, in particular by specifying the &lt;user&gt; element and its 'entity' attribute.</p>
</section1>

<section1 topic='Examples'>
<section2 topic='Juliet Joins a Conference'>
<example caption="Juliet initiates a Jingle session with the conference room"><![CDATA[
<iq id="av_iq_1" type="set" from="[email protected]/balcony" to="[email protected]">
<jingle xmlns="urn:xmpp:jingle:1"
sid="av_jingle_session_1"
action="session-initiate"
initiator="[email protected]/balcony">
<group xmlns="urn:xmpp:jingle:apps:grouping:0" semantics="BUNDLE">
<content name="video0"/>
<content name="audio1"/>
</group>
<content creator="initiator" name="video0" senders="initiator">
<description xmlns="urn:xmpp:jingle:apps:rtp:1" media="video">
<!-- video stream description -->
</description>
<transport xmlns="urn:xmpp:jingle:transports:ice-udp:1"
ufrag="ufrag_1"
pwd="ice_pwd_1">
<fingerprint xmlns="urn:xmpp:jingle:apps:dtls:0" hash="sha-256" setup="actpass">
<!-- some fingerprint -->
</fingerprint>
</transport>
</content>
<content creator="initiator" name="audio1" senders="initiator">
<description xmlns="urn:xmpp:jingle:apps:rtp:1" media="audio">
<!-- audio stream description -->
</description>
<transport xmlns="urn:xmpp:jingle:transports:ice-udp:1"
ufrag="ufrag_1"
pwd="ice_pwd_1">
<!-- transport child elements -->
</transport>
</content>
</jingle>
</iq>
<iq to="[email protected]/balcony" from="[email protected]" id="av_iq_1" type="result"/>
]]></example>

<example caption="Conference room accepts the session"><![CDATA[
<iq id="room_iq_1" type="set" from="[email protected]" to="[email protected]/balcony">
<jingle xmlns="urn:xmpp:jingle:1"
sid="av_jingle_session_1"
action="session-accept"
responder="[email protected]">
<group xmlns="urn:xmpp:jingle:apps:grouping:0" semantics="BUNDLE">
<content name="audio1"/>
<content name="video0"/>
</group>
<content creator="initiator" name="video0">
<description xmlns="urn:xmpp:jingle:apps:rtp:1" media="video">
<!-- video stream description -->
</description>
<!-- transport element -->
</content>
<content creator="initiator" name="audio1">
<description xmlns="urn:xmpp:jingle:apps:rtp:1" media="audio">
<!-- audio stream description -->
</description>
<!-- transport element -->
</content>
<conference-info xmlns='urn:xmpp:coin:1' isfocus='true'/>
</jingle>
</iq>
]]></example>
</section2>

<section2 topic='Romeo Joins the Conference'>
<example caption="Conference room initiates a session with Juliet to send Romeo's stream"><![CDATA[
<iq id="room_iq_2" type="set" from="[email protected]" to="[email protected]/balcony">
<jingle xmlns="urn:xmpp:jingle:1"
sid="av_jingle_session_2"
action="session-initiate"
initiator="[email protected]">
<group xmlns="urn:xmpp:jingle:apps:grouping:0" semantics="BUNDLE">
<content name="0"/>
<content name="1"/>
</group>
<content creator="initiator" name="0" senders="initiator">
<description xmlns="urn:xmpp:jingle:apps:rtp:1" media="audio">
<!-- audio stream description -->
</description>
<transport xmlns="urn:xmpp:jingle:transports:ice-udp:1"
ufrag="ufrag_2"
pwd="ice_pwd_2">
<fingerprint xmlns="urn:xmpp:jingle:apps:dtls:0" hash="sha-256" setup="actpass">
<!-- some fingerprint -->
</fingerprint>
</transport>
</content>
<content creator="initiator" name="1" senders="initiator">
<description xmlns="urn:xmpp:jingle:apps:rtp:1" media="video">
<!-- video stream description -->
</description>
<transport xmlns="urn:xmpp:jingle:transports:ice-udp:1"
ufrag="ufrag_2"
pwd="ice_pwd_2">
<fingerprint xmlns="urn:xmpp:jingle:apps:dtls:0" hash="sha-256" setup="actpass">
<!-- some fingerprint -->
</fingerprint>
</transport>
</content>
<conference-info xmlns='urn:xmpp:coin:1' isfocus='true'>
<users>
<user entity='xmpp:[email protected]' />
</users>
</conference-info>
</jingle>
</iq>
]]></example>
</section2>
</section1>

<section1 topic="Configuration" anchor='configuration'>
<p>It may be desirable to have configuration options on the SFU service or on one of its rooms. For instance, a conference room may offer the option to record the session. While the definition of those options is beyond the scope of this specification, the configuration MUST be done via &xep0050; on the well-known node "urn:xmpp:av-conferences:config".</p>
</section1>

<section1 topic='Business Rules' anchor='rules'>
<p>Only other peers' streams are sent to a joining entity. A SFU service MUST NOT send back a stream to the entity that initially sent it.</p>
<p>A stream MAY originate from a non-XMPP entity (e.g., using SFU's own user interface or with a gateway to another protocol). That means that the &lt;user&gt; 'entity' attribute may be something else than an "xmpp:" URL.</p>
</section1>

<section1 topic='Discovering Support' anchor='disco'>
<p>If a client or a service supports the protocol specified in this XEP, it MUST advertise it by including the "urn:xmpp:jingle:av-conferences:0" discovery feature in response to a &xep0030; information request.</p>

<example caption="Service Discovery information request"><![CDATA[
<iq type='get'
from='[email protected]/balcony'
to='[email protected]/orchard'
id='disco1'>
<query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>]]></example>
<example caption="Service Discovery information response"><![CDATA[
<iq type='result'
from='[email protected]/orchard'
to='[email protected]/balcony'
id='disco1'>
<query xmlns='http://jabber.org/protocol/disco#info'>
...
<feature var='urn:xmpp:jingle:av-conferences:0'/>
...
</query>
</iq>]]></example>

<p>If an entity is a A/V conferences service as specified by this XEP, it MUST have an identity with a category of "conference" and a value of "audio-video" in a response to a &xep0030; information request.</p>

<example caption="Service Discovery information request"><![CDATA[
<iq type='get'
from='[email protected]/balcony'
to='[email protected]'
id='disco2'>
<query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>]]></example>
<example caption="Service Discovery information response"><![CDATA[
<iq type='result'
from='[email protected]'
to='[email protected]/balcony'
id='disco2'>
<query xmlns='http://jabber.org/protocol/disco#info'>
...
<identity category='conference' type='audio-video'/>
...
<feature var='urn:xmpp:jingle:av-conferences:0'/>
...
</query>
</iq>]]></example>

</section1>

<section1 topic="Security Considerations" anchor="security">
<p>Streams are sent to the SFU service for redistribution. Although it is technically possible to have the stream encrypted end-to-end, this specification assumes that streams are decoded at the SFU level and are therefore not end-to-end encrypted, unlike one-to-one calls. Future specifications may outline a method for maintaining end-to-end encryption, but this is beyond the scope of this XEP; developers and end-users MUST assume that streams are decrypted at the SFU service level.</p>
<p>The general security considerations for one-to-one calls apply, with the added risk that more parties are participating in the call. However, the IP address of a device sending a stream is only known by the SFU service, not by other peers, unlike in one-to-one or &xep0272; calls.</p>
</section1>

<section1 topic='IANA Considerations' anchor='iana'>
<p>TODO</p>
</section1>
<section1 topic='XMPP Registrar Considerations' anchor='registrar'>
<p>TODO</p>
</section1>
<section1 topic='XML Schema' anchor='schema'>
<p>TODO</p>
</section1>

<section1 topic='Acknowledgements' anchor='acks'>
<p>Thanks to NLNet foundation/NGI Assure for funding the work on this specification.</p>
</section1>
</xep>
19 changes: 12 additions & 7 deletions xep-0045.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,15 @@
&stpeter;
<revision>
<version>1.34.7</version>
<date>2024-07-10</date>
<date>2024-07-09</date>
<initials>gk</initials>
<remark><p>Status code purpose no longer hints that recipient is the affected user</p></remark>
<remark>
<ul>
<li>Remove references to using resourceparts when banning users.</li>
<li>Explicitly disallow Ban List modifications that clash with 'Banning a User' conditions.</li>
<li>Status code purpose no longer hints that recipient is the affected user</li>
</ul>
</remark>
</revision>
<revision>
<version>1.34.6</version>
Expand Down Expand Up @@ -3332,14 +3338,13 @@
type='result'/>
]]></example>
<p>The service MUST then remove the affected occupants (if they are in the room) and send updated presence (including the appropriate status code) from them to all the remaining occupants as described in the "Banning a User" use case. (The service MUST also remove each banned user's reserved nickname from the list of reserved roomnicks, if appropriate.)</p>
<p>When an entity is banned from a room, an implementation SHOULD match JIDs in the following order (these matching rules are the same as those defined for privacy lists in &xep0016;):</p>
<p>When an entity is banned from a room, an implementation SHOULD match JIDs in the following order:</p>
<ol start='1'>
<li>&lt;user@domain/resource&gt; (only that resource matches)</li>
<li>&lt;user@domain&gt; (any resource matches)</li>
<li>&lt;domain/resource&gt; (only that resource matches)</li>
<li>&lt;domain&gt; (the domain itself matches, as does any user@domain or domain/resource)</li>
<li>&lt;user@domain&gt;</li>
<li>&lt;domain&gt; (the domain itself matches, as does any user@domain)</li>
</ol>
<p>Some administrators might wish to ban all users associated with a specific domain from all rooms hosted by a MUC service. Such functionality is a service-level feature and is therefore out of scope for this document; see <cite>XEP-0133</cite>.</p>
<p>As specified in <link url='#ban'>Banning a User</link>, users cannot be banned under certain conditions. For example: admins and owners cannot ban themselves, and a user cannot be banned by an admin with a lower affiliation. When a request to modify the ban list includes one or more modifications that is prohibited by the definitions in <link url='#ban'>Banning a User</link>, then the service SHOULD NOT apply any of the requested changes and MUST deny the request using an error which SHOULD be either &conflict; or &notallowed;.</p>
</section2>

<section2 topic='Granting Membership' anchor='grantmember'>
Expand Down
10 changes: 9 additions & 1 deletion xep-0054.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,20 @@
<status>Active</status>
<type>Historical</type>
<sig>Standards</sig>
<approver>Council</approver>
<dependencies>
<spec>XMPP Core</spec>
</dependencies>
<supersedes/>
<supersededby/>
<shortname>vcard-temp</shortname>
&stpeter;
<revision>
<version>1.3.0</version>
<date>2024-06-27</date>
<initials>gk</initials>
<remark><p>Updated error cases to be compatible with &xmppim;.</p></remark>
</revision>
<revision>
<version>1.2</version>
<date>2008-07-16</date>
Expand Down Expand Up @@ -242,7 +249,8 @@
</vCard>
</iq>
]]></example>
<p>If no vCard exists or the user does not exist, the server MUST return a stanza error, which SHOULD be either &unavailable; or &notfound; (but the server MUST return the same error condition in both cases to help prevent directory harvesting attacks).</p>
<p>If no vCard exists or the user does not exist, the server MUST return a stanza error. The server MUST return the same error condition in both cases to help prevent directory harvesting attacks. &xmppim; dictates that &unavailable; MUST be used when the user account identified by the 'to' attribute of an IQ stanza does not exist. Hence, &unavailable; MUST be used in both cases.</p>
<p>Note: versions prior to 1.3 of this specification defined that either &unavailable; or &notfound; SHOULD be was used in these cases. For backwards-compatibility, applications SHOULD appropriately handle both errors.</p>
<example caption="No vCard"><![CDATA[
<iq id='v3'
to='[email protected]/roundabout'
Expand Down
Loading

0 comments on commit 17ded3f

Please sign in to comment.