diff --git a/index.bs b/index.bs
index 7d43920..4ab70f8 100644
--- a/index.bs
+++ b/index.bs
@@ -91,37 +91,44 @@ argument, ensure that the codec is disabled and produces no output.
### Stream creation ### {#stream-creation}
+Each {{RTCRtpSender}} or {{RTCRtpReceiver}} has its own media thread on which media flows,
+from a source on which to read encoded data to a sink on which to write encoded data.
+The source is the packetizer for {{RTCRtpSender}} and the decoder for {{RTCRtpReceiver}}.
+It operates in the [=media thread=] and is modeled as a {{ReadableStream}}.
+The sink is the encoder for {{RTCRtpSender}} and the depacketizer for {{RTCRtpReceiver}}.
+It operates in the [=media thread=] and is modeled as a {{WritableStream}}.
+
At construction of each {{RTCRtpSender}} or {{RTCRtpReceiver}}, run the following steps:
-2. Initialize [=this=].`[[transform]]` to null.
-3. Initialize [=this=].`[[readable]]` to a new {{ReadableStream}}.
-4. Set up [=this=].`[[readable]]`. [=this=].`[[readable]]` is provided frames using the [=readEncodedData=] algorithm given |this| as parameter.
-5. Set [=this=].`[[readable]]`.`[[owner]]` to |this|.
-6. Initialize [=this=].`[[writable]]` to a new {{WritableStream}}.
-7. Set up [=this=].`[[writable]]` with its [=WritableStream/set up/writeAlgorithm=] set to [=writeEncodedData=] given |this| as parameter.
-8. Set [=this=].`[[writable]]`.`[[owner]]` to |this|.
-9. Initialize [=this=].`[[pipeToController]]` to null.
-10. Initialize [=this=].`[[lastReceivedFrameTimestamp]]` to zero.
-11. [=Queue a task=] to run the following steps:
- 1. If [=this=].`[[pipeToController]]` is not null, abort these steps.
- 2. Set [=this=].`[[pipeToController]]` to a new {{AbortController}}.
-
- 3. Call pipeTo with [=this=].`[[readable]]`, [=this=].`[[writable]]`, preventClose equal to true, preventAbort equal to true, preventCancel equal to true and [=this=].`[[pipeToController]]`.signal.
+1. Initialize [=this=].`[[transform]]` to null
.
+1. Initialize [=this=].`[[lastReceivedFrameTimestamp]]` to zero.
+1. Initialize [=this=].`[[pipeToController]]` to null
.
+1. Initialize [=this=].`[[source]]` to a new {{ReadableStream}}.
+1. Set up [=this=].`[[source]]`. [=this=].`[[source]]` is provided frames using the [=readSourceData=] algorithm given |this| as parameter.
+1. Initialize [=this=].`[[sink]]` to a new {{WritableStream}}.
+1. Set up [=this=].`[[sink]]` with its [=WritableStream/set up/writeAlgorithm=] set to [=writeSinkData=] given |this| as parameter.
+1. [=Queue a task=] to run the following step:
+ 1. [=Queue a task=] on [=this=]'s media thread to run the following steps:
+ 1. If [=this=].`[[pipeToController]]` is not null, abort these steps.
+ 1. Set [=this=].`[[pipeToController]]` to a new {{AbortController}}.
+
+ 1. Call pipeTo with [=this=].`[[source]]`,
+ [=this=].`[[sink]]`, preventClose equal to true, preventAbort equal to true, preventCancel equal to true and [=this=].`[[pipeToController]]`.signal.
### Stream processing ### {#stream-processing}
-The readEncodedData algorithm is given a |rtcObject| as parameter. It is defined by running the following steps:
+The readSourceData algorithm given |rtcObject| runs the following steps in the [=media thread=]:
1. Wait for a frame to be produced by |rtcObject|'s encoder if it is a {{RTCRtpSender}} or |rtcObject|'s packetizer if it is a {{RTCRtpReceiver}}.
-2. Let |frame| be the newly produced frame.
-3. Set |frame|.`[[owner]]` to |rtcObject|.
-4. [=ReadableStream/Enqueue=] |frame| in |rtcObject|.`[[readable]]`.
+1. Let |frame| be the newly produced frame.
+1. Set |frame|.`[[owner]]` to |rtcObject|.
+1. [=ReadableStream/Enqueue=] |frame| in |rtcObject|.`[[source]]`.
-The writeEncodedData algorithm is given a |rtcObject| as parameter and a |frame| as input. It is defined by running the following steps:
+The writeSinkData algorithm given |rtcObject| and |frame| runs the following steps in the [=media thread=]:
1. If |frame|.`[[owner]]` is not equal to |rtcObject|, abort these steps and return [=a promise resolved with=] undefined. A processor cannot create frames, or move frames between streams.
-2. If the |frame|'s {{RTCEncodedVideoFrame/timestamp}} is equal to or larger than |rtcObject|.`[[lastReceivedFrameTimestamp]]`, abort these steps and return [=a promise resolved with=] undefined. A processor cannot reorder frames, although it may delay them or drop them.
-3. Set |rtcObject|.`[[lastReceivedFrameTimestamp]]` to the |frame|'s {{RTCEncodedVideoFrame/timestamp}}.
-4. Enqueue the frame for processing as if it came directly from the encoded data source, by running one of the following steps:
- * If |rtcObject| is a {{RTCRtpSender}}, enqueue it to |rtcObject|'s packetizer, to be processed [=in parallel=].
- * If |rtcObject| is a {{RTCRtpReceiver}}, enqueue it to |rtcObject|'s decoder, to be processed [=in parallel=].
+1. If the |frame|'s {{RTCEncodedVideoFrame/timestamp}} is equal to or larger than |rtcObject|.`[[lastReceivedFrameTimestamp]]`, abort these steps and return [=a promise resolved with=] undefined. A processor cannot reorder frames, although it may delay them or drop them.
+1. Set |rtcObject|.`[[lastReceivedFrameTimestamp]]` to the |frame|'s {{RTCEncodedVideoFrame/timestamp}}.
+1. Enqueue the frame for processing as if it came directly from the encoded data source, by running one of the following steps in the [=media thread=]:
+ * If |rtcObject| is a {{RTCRtpSender}}, enqueue it to |rtcObject|'s packetizer.
+ * If |rtcObject| is a {{RTCRtpReceiver}}, enqueue it to |rtcObject|'s decoder.
5. Return [=a promise resolved with=] undefined.
## Extension attribute ## {#attribute}
@@ -132,29 +139,33 @@ The transform getter step
1. Return [=this=].`[[transform]]`.
The `transform` setter steps are:
-2. Let |transform| be the argument to the setter.
-3. Let |checkedTransform| set to |transform| if it is not null or to an [=identity transform stream=] otherwise.
-3. Let |reader| be the result of [=ReadableStream/getting a reader=] for |checkedTransform|.`[[readable]]`.
-4. Let |writer| be the result of [=WritableStream/getting a writer=] for |checkedTransform|.`[[writable]]`.
-5. Initialize |newPipeToController| to a new {{AbortController}}.
-6. If [=this=].`[[pipeToController]]` is not null, run the following steps:
- 1. [=AbortSignal/Add=] the [=chain transform algorithm=] to [=this=].`[[pipeToController]]`.signal.
- 2. [=AbortSignal/signal abort=] [=this=].`[[pipeToController]]`.signal.
-7. Else, run the [=chain transform algorithm=] steps.
-8. Set [=this=].`[[pipeToController]]` to |newPipeToController|.
-9. Set [=this=].`[[transform]]` to |transform|.
-
-The chain transform algorithm steps are defined as:
-1. If |newPipeToController|'s [=AbortSignal/aborted flag=] is true, abort these steps.
-2. [=ReadableStreamDefaultReader/Release=] |reader|.
-3. [=WritableStreamDefaultWriter/Release=] |writer|.
-4. Assert that |newPipeToController| is the same object as |rtcObject|.`[[pipeToController]]`.
-
-5. Call pipeTo with |rtcObject|.`[[readable]]`, |checkedTransform|.`[[writable]]`, preventClose equal to false, preventAbort equal to false, preventCancel equal to true and |newPipeToController|.signal.
-6. Call pipeTo with |checkedTransform|.`[[readable]]`, |rtcObject|.`[[writable]]`, preventClose equal to true, preventAbort equal to true, preventCancel equal to false and |newPipeToController|.signal.
+1. Let |transform| be the argument to the setter.
+1. If |transform| is not null
, run the following steps:
+ 1. If |transform|.`[[readable]]` is [=ReadableStream/locked=], [=throw=] a {{TypeError}}.
+ 1. [=WritableStream/getting a writer|Get a writer=] for |transform|.`[[writable]]`.
+ 1. [=ReadableStream/getting a reader|Get a reader=] for |transform|.`[[readable]]`.
+1. [=Queue a task=] in [=this=]'s [=media thread=] to run the following steps:
+ 1. let |pipeToController| be a new {{AbortController}}.
+ 1. Let |internalTransform| be an [=identity transform stream=].
+ 1. If |transform| is an {{SFrameTransform}}, set |internalTransform| to an [=SFrame transform stream=] given |transform|.
+ 1. If |transform| is an {{RTCRtpScriptTransform}}, set |internalTransform| to a [=script transform stream=] given |transform|.
+ 1. Run the [=chain transform algorithm=] with [=this=], |internalTransform|.`[[readable]]`, |internalTransform|.`[[writable]]` and |pipeToController|.
+1. Set [=this=].`[[transform]]` to |transform|.
+
+The chain transform algorithm, given |rtcObject|, |readable|, |writable| and |pipeToController|, runs these steps in rtcObject's [=media thread=]:
+1. If |pipeToController| [=AbortSignal/aborted flag=] is true, abort these steps.
+1. If |rtcObject|.`[[pipeToController]]` is not null
, run the following steps:
+ 1. [=AbortSignal/Add=] the [=chain transform algorithm=] with |rtcObject|, |readable|, |writable| and |pipeToController|, to |rtcObject|.`[[pipeToController]]`.signal.
+ 1. [=AbortSignal/signal abort=] |rtcObject|.`[[pipeToController]]`.signal.
+1. Else run the following steps:
+
+ 1. Call pipeTo with |rtcObject|.`[[source]]`, |writable|, preventClose equal to false, preventAbort equal to false, preventCancel equal to true and |pipeToController|.signal.
+ 1. Call pipeTo with |readable|, |rtcObject|.`[[sink]]`, preventClose equal to true, preventAbort equal to true, preventCancel equal to false and |pipeToController|.signal.
+1. Set |rtcObject|.`[[pipeToController]]` to |pipeToController|.
This algorithm is defined so that transforms can be updated dynamically.
There is no guarantee on which frame will happen the switch from the previous transform to the new transform.
+If a new transform overwrites an old transform, all frames will go either through the old or the new transform from the source to the sink.
If a web application sets the transform synchronously at creation of the {{RTCRtpSender}} (for instance when calling addTrack), the transform will receive the first frame generated by the {{RTCRtpSender}}'s encoder.
Similarly, if a web application sets the transform synchronously at creation of the {{RTCRtpReceiver}} (for instance when calling addTrack, or at track event handler), the transform will receive the first full frame generated by the {{RTCRtpReceiver}}'s packetizer.
@@ -183,40 +194,43 @@ SFrameTransform includes GenericTransformStream;
The new SFrameTransform(options)
constructor steps are:
-1. Let |transformAlgorithm| be an algorithm which takes a |frame| as input and runs the SFrame transform algorithm with |this| and |frame|.
-2. Set |this|.`[[transform]]` to a new {{TransformStream}}.
-3. Set up [=this=].`[[transform]]` with [=TransformStream/set up/transformAlgorithm=] set to |transformAlgorithm|.
-4. Let |options| be the method's first argument.
-5. Set |this|.`[[role]]` to |options|["{{SFrameTransformOptions/role}}"].
-6. Set |this|.`[[readable]]` to |this|.`[[transform]]`.`[[readable]]`.
-7. Set |this|.`[[writable]]` to |this|.`[[transform]]`.`[[writable]]`.
-
-## Algorithm ## {#sframe-transform-algorithm}
-
-The SFrame transform algorithm, given |sframe| as a SFrameTransform object and |frame|, runs these steps:
-1. Let |role| be |sframe|.`[[role]]`.
-2. If |frame|.`[[owner]]` is a {{RTCRtpSender}}, set |role| to 'encrypt'.
-3. If |frame|.`[[owner]]` is a {{RTCRtpReceiver}}, set |role| to 'decrypt'.
-4. Let |data| be undefined.
-5. If |frame| is a {{BufferSource}}, set |data| to |frame|.
-6. If |frame| is a {{RTCEncodedAudioFrame}}, set |data| to |frame|.{{RTCEncodedAudioFrame/data}}
-7. If |frame| is a {{RTCEncodedVideoFrame}}, set |data| to |frame|.{{RTCEncodedVideoFrame/data}}
-8. If |data| is undefined, abort these steps.
-9. Let |buffer| be the result of running the SFrame algorithm with |data| and |role| as parameters. This algorithm is defined by the SFrame specification and returns an {{ArrayBuffer}}.
-10. If |frame| is a {{BufferSource}}, set |frame| to |buffer|.
-11. If |frame| is a {{RTCEncodedAudioFrame}}, set |frame|.{{RTCEncodedAudioFrame/data}} to |buffer|.
-12. If |frame| is a {{RTCEncodedVideoFrame}}, set |frame|.{{RTCEncodedVideoFrame/data}} to |buffer|.
-13. [=ReadableStream/Enqueue=] |frame| in |sframe|.`[[transform]]`.
+1. Set |this|.`[[role]]` to |options|["{{SFrameTransformOptions/role}}"].
+1. Let |sframeTransform| be a [=SFrame transform stream=] given |this|.
+1. Set |this|.`[[readable]]` to |sframeTransform|.`[[readable]]`.
+1. Set |this|.`[[writable]]` to |sframeTransform|.`[[writable]]`.
+
+## Algorithms ## {#sframe-transform-algorithms}
+
+A SFrame transform stream given |transform| is created by running the following steps:
+1. Let |transformAlgorithm| be an algorithm which takes a |frame| as input and runs the [=SFrame transform algorithm=] with |transform| and |frame|.
+1. Set |transform|.`[[sframeTransform]]` to a new {{TransformStream}}.
+1. [=TransformStream/Set up=] |transform|.`[[sframeTransform]]` with [=TransformStream/set up/transformAlgorithm=] set to |transformAlgorithm|.
+1. Return |sframeTransform|.
+
+The SFrame transform algorithm, given |sframeTransform| and |frame|, runs these steps:
+1. Let |role| be |sframeTransform|.`[[role]]`.
+1. If |frame|.`[[owner]]` is a {{RTCRtpSender}}, set |role| to 'encrypt'.
+1. If |frame|.`[[owner]]` is a {{RTCRtpReceiver}}, set |role| to 'decrypt'.
+1. Let |data| be undefined.
+1. If |frame| is a {{BufferSource}}, set |data| to |frame|.
+1. If |frame| is a {{RTCEncodedAudioFrame}}, set |data| to |frame|.{{RTCEncodedAudioFrame/data}}
+1. If |frame| is a {{RTCEncodedVideoFrame}}, set |data| to |frame|.{{RTCEncodedVideoFrame/data}}
+1. If |data| is undefined, abort these steps.
+1. Let |buffer| be the result of running the SFrame algorithm with |data| and |role| as parameters. This algorithm is defined by the SFrame specification and returns an {{ArrayBuffer}}.
+1. If |frame| is a {{BufferSource}}, set |frame| to |buffer|.
+1. If |frame| is a {{RTCEncodedAudioFrame}}, set |frame|.{{RTCEncodedAudioFrame/data}} to |buffer|.
+1. If |frame| is a {{RTCEncodedVideoFrame}}, set |frame|.{{RTCEncodedVideoFrame/data}} to |buffer|.
+1. [=ReadableStream/Enqueue=] |frame| in |sframeTransform|.`[[transform]]`.
## Methods ## {#sframe-transform-methods}
The setEncryptionKey(|key|, |keyID|) method steps are:
1. Let |promise| be [=a new promise=].
-2. If |keyID| is a {{bigint}} which cannot be represented as a integer between 0 and 264-1 inclusive, [=reject=] |promise| with a {{RangeError}} exception.
-3. Otherwise, [=in parallel=], run the following steps:
- 1. Set |key| with its optional |keyID| as key material to use for the SFrame transform algorithm, as defined by the SFrame specification.
- 2. If setting the key material fails, [=reject=] |promise| with an {{InvalidModificationError}} exception and abort these steps.
- 3. [=Resolve=] |promise| with undefined.
-4. Return |promise|.
+1. If |keyID| is a {{bigint}} which cannot be represented as a integer between 0 and 264-1 inclusive, [=reject=] |promise| with a {{RangeError}} exception.
+1. Otherwise, [=in parallel=], run the following steps:
+ 1. Set |key| with its optional |keyID| as key material to use for the [=SFrame transform algorithm=], as defined by the SFrame specification.
+ 1. If setting the key material fails, [=reject=] |promise| with an {{InvalidModificationError}} exception and abort these steps.
+ 1. [=Resolve=] |promise| with undefined.
+1. Return |promise|.
# RTCRtpScriptTransform # {#scriptTransform}
@@ -289,30 +303,39 @@ interface RTCRtpScriptTransform {
## Operations ## {#RTCRtpScriptTransform-operations}
-
The new RTCRtpScriptTransform(|worker|, |options|, |transfer|)
constructor steps are:
-1. Set |t1| to an [=identity transform stream=].
-2. Set |t2| to an [=identity transform stream=].
-3. Set |this|.`[[writable]]` to |t1|.`[[writable]]`.
-4. Set |this|.`[[readable]]` to |t2|.`[[readable]]`.
-5. Let |serializedOptions| be the result of [$StructuredSerializeWithTransfer$](|options|, |transfer|).
-6. Let |serializedReadable| be the result of [$StructuredSerializeWithTransfer$](|t1|.`[[readable]]`, « |t1|.`[[readable]]` »).
-7. Let |serializedWritable| be the result of [$StructuredSerializeWithTransfer$](|t2|.`[[writable]]`, « |t2|.`[[writable]]` »).
-8. [=Queue a task=] on the DOM manipulation [=task source=] |worker|'s global scope to run the following steps:
+1. Set |this|.`[[worker]]` to |worker|.
+1. Let |serializedOptions| be the result of [$StructuredSerializeWithTransfer$](|options|, |transfer|).
+1. [=Queue a task=] on the DOM manipulation [=task source=] |worker|'s global scope to run the following steps:
+ 1. Let transformer be a new {{RTCRtpScriptTransformer}}.
1. Let |transformerOptions| be the result of [$StructuredDeserialize$](|serializedOptions|, the current Realm).
- 2. Let |readable| be the result of [$StructuredDeserialize$](|serializedReadable|, the current Realm).
- 3. Let |writable| be the result of [$StructuredDeserialize$](|serializedWritable|, the current Realm).
- 4. Let |transformer| be a new {{RTCRtpScriptTransformer}}.
- 5. Set |transformer|.`[[options]]` to |transformerOptions|.
- 6. Set |transformer|.`[[readable]]` to |readable|.
- 7. Set |transformer|.`[[writable]]` to |writable|.
- 8. Let |event| be the result of [=creating an event=] with {{RTCTransformEvent}}.
- 9. Set |event|.type attribute to "rtctransform".
- 10. Set |event|.transformer to |transformer|.
- 11. Dispatch |event| on |worker|’s global scope.
+ 1. Set |transformer|.`[[options]]` to |transformerOptions|.
+ 1. Let |transformer|.`[[t1]]` to an [=identity transform stream=].
+ 1. Let |transformer|.`[[t2]]` to an [=identity transform stream=].
+ 1. Set |transformer|.`[[readable]]` to |transformer|.`[[t1]]`.`[[readable]]`.
+ 1. Set |transformer|.`[[writable]]` to |transformer|.`[[t2]]`.`[[writable]]`.
+ 1. Let |event| be the result of [=creating an event=] with {{RTCTransformEvent}}.
+ 1. Set |event|.type attribute to "rtctransform".
+ 1. Set |event|.transformer to |transformer|.
+ 1. Set |this|.`[[transformer]]` to |transformer|.
+ 1. [=Dispatch=] |event| to |worker|’s global scope.
// FIXME: Describe error handling (worker closing flag true at RTCRtpScriptTransform creation time. And worker being terminated while transform is processing data).
+A script transform stream given |transform| is created by running the following steps:
+1. Set |t1| to an [=identity transform stream=].
+1. Set |t2| to an [=identity transform stream=].
+1. Set |scriptTransform|.`[[writable]]` to |t1|.`[[writable]]`.
+1. Set |scriptTransform|.`[[readable]]` to |t2|.`[[readable]]`.
+1. Let |serializedReadable| be the result of [$StructuredSerializeWithTransfer$](|t1|.`[[readable]]`, « |t1|.`[[readable]]` »).
+1. Let |serializedWritable| be the result of [$StructuredSerializeWithTransfer$](|t2|.`[[writable]]`, « |t2|.`[[writable]]` »).
+1. [=Queue a task=] on the DOM manipulation [=task source=] |transform|.`[[worker]]`'s global scope to run the following steps:
+ 1. Let |readable| be the result of [$StructuredDeserialize$](|serializedReadable|, the current Realm).
+ 1. Let |writable| be the result of [$StructuredDeserialize$](|serializedWritable|, the current Realm).
+ 1. Call pipeTo with |readable|, |transform|.`[[transformer]]`.`[[t1]]`.`[[writable]]`.
+ 1. Call pipeTo with |transform|.`[[transformer]]`.`[[t2]]`.`[[readable]]` and |writable|.
+1. Return |scriptTransform|.
+
## Attributes ## {#RTCRtpScriptTransformer-attributes}
A RTCRtpScriptTransformer has three private slots called `[[options]]`, `[[readable]]` and `[[writable]]`.