Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix {de}serialization steps to transfer data appropriately #239

Merged
merged 1 commit into from
Feb 13, 2025
Merged

Fix {de}serialization steps to transfer data appropriately #239

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 22 additions & 19 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ The <dfn abstract-op>writeEncodedData</dfn> algorithm is given a |rtcObject| as
1. Set |rtcObject|.`[[lastReceivedFrameCounter]]` to |frame|`[[counter]]`.
1. Let |data| be |frame|.`[[data]]`.
1. Let |serializedFrame| be [$StructuredSerializeWithTransfer$](|frame|, « |data| »).
1. Let |frameCopy| be [$StructuredDeserialize$](|serializedFrame|, |frame|'s [=relevant realm=]).
1. Let |frameCopy| be [$StructuredDeserializeWithTransfer$](|serializedFrame|, |frame|'s [=relevant realm=]).
1. Enqueue |frameCopy| for processing as if it came directly from the encoded data source, by running one of the following steps:
* If |rtcObject| is a {{RTCRtpSender}}, enqueue |frameCopy| to |rtcObject|'s packetizer, to be processed [=in parallel=].
* If |rtcObject| is a {{RTCRtpReceiver}}, enqueue |frameCopy| it to |rtcObject|'s decoder, to be processed [=in parallel=].
Expand Down Expand Up @@ -371,7 +371,7 @@ dictionary RTCEncodedVideoFrameMetadata {
<dd>
<p>
An identifier for the encoded frame, monotonically increasing in decode order. Its lower
16 bits match the frame_number of the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[?AV1-RTP-SPEC]], if present.
16 bits match the frame_number of the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[AV1-RTP-SPEC]], if present.
Only present for received frames if the Dependency Descriptor Header Extension is present.
</p>
</dd>
Expand All @@ -381,7 +381,7 @@ dictionary RTCEncodedVideoFrameMetadata {
<dd>
<p>
List of frameIds of frames this frame references.
Only present for received frames if the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[?AV1-RTP-SPEC]] is present.
Only present for received frames if the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[AV1-RTP-SPEC]] is present.
</p>
</dd>
<dt>
Expand Down Expand Up @@ -460,7 +460,7 @@ interface RTCEncodedVideoFrame {
};
</pre>

### Constructor ### {#RTCEncodedVideoFrame-members}
### Constructor ### {#RTCEncodedVideoFrame-constructor}
<dl dfn-for="RTCEncodedVideoFrame" class="dictionary-members">
<dt>
<dfn for="RTCEncodedVideoFrame" method>constructor()</dfn>
Expand Down Expand Up @@ -506,11 +506,13 @@ interface RTCEncodedVideoFrame {
{{RTCEncodedVideoFrameMetadata/mimeType}}.
For <a href="https://w3c.github.io/webrtc-svc/">SVC</a>, each spatial layer
is transformed separately.
<p class="note">
</p>
<p class="note">
Since packetizers may drop certain elements, e.g. AV1 temporal delimiter OBUs,
the input to an receive-side transform may be different from the output of
a send-side transform.
</p>
</p>
<p>
The following table gives a number of examples:
</p>
<table class="simple">
Expand Down Expand Up @@ -582,19 +584,20 @@ interface RTCEncodedVideoFrame {

### Serialization ### {#RTCEncodedVideoFrame-serialization}

{{RTCEncodedVideoFrame}} objects are serializable objects [[HTML]].
{{RTCEncodedVideoFrame}} objects are [=serializable objects=].
Their [=serialization steps=], given |value|, |serialized|, and |forStorage|, are:

1. If |forStorage| is true, then throw a {{DataCloneError}}.
1. Set |serialized|.`[[type]]` to the value of |value|.{{RTCEncodedVideoFrame/type}}
1. Set |serialized|.`[[type]]` to the value of |value|.{{RTCEncodedVideoFrame/type}}.
1. Set |serialized|.`[[metadata]]` to an internal representation of |value|'s metadata.
1. Set |serialized|.`[[data]]` to |value|.`[[data]]`
1. Set |serialized|.`[[data]]` to the [=sub-serialization=] of |value|.`[[data]]`.


Their [=deserialization steps=], given |serialized|, |value| and |realm|, are:

1. Set |value|.{{RTCEncodedVideoFrame/type}} to |serialized|.`[[type]]`
1. Set |value|'s metadata to the platform object representation of |serialized|.`[[metadata]]`
1. Set |value|.`[[data]]` to |serialized|.`[[data]]`.
1. Set |value|.{{RTCEncodedVideoFrame/type}} to |serialized|.`[[type]]`.
1. Set |value|'s metadata to the platform object representation of |serialized|.`[[metadata]]`.
1. Set |value|.`[[data]]` to the [=sub-deserialization=] of |serialized|.`[[data]]`.

<p class="note">
The internal form of a serialized RTCEncodedVideoFrame is not observable;
Expand Down Expand Up @@ -689,7 +692,7 @@ interface RTCEncodedAudioFrame {
};
</pre>

### Constructor ### {#RTCEncodedAudioFrame-members}
### Constructor ### {#RTCEncodedAudioFrame-constructor}
<dl dfn-for="RTCEncodedAudioFrame" class="dictionary-members">
<dt>
<dfn for="RTCEncodedAudioFrame" method>constructor()</dfn>
Expand Down Expand Up @@ -808,17 +811,17 @@ interface RTCEncodedAudioFrame {

### Serialization ### {#RTCEncodedAudioFrame-serialization}

{{RTCEncodedAudioFrame}} objects are serializable objects [[HTML]].
{{RTCEncodedAudioFrame}} objects are [=serializable objects=].
Their [=serialization steps=], given |value|, |serialized|, and |forStorage|, are:

1. If |forStorage| is true, then throw a {{DataCloneError}}.
1. Set |serialized|.`[[metadata]]` to an internal representation of |value|'s metadata.
1. Set |serialized|.`[[data]]` to |value|.`[[data]]`
1. Set |serialized|.`[[data]]` to the [=sub-serialization=] of |value|.`[[data]]`.

Their [=deserialization steps=], given |serialized|, |value| and |realm|, are:

1. Set |value|'s metadata to the platform object representation of |serialized|.`[[metadata]]`
1. Set |value|.`[[data]]` to |serialized|.`[[data]]`.
1. Set |value|.`[[data]]` to the [=sub-deserialization=] of |serialized|.`[[data]]`.

## Interfaces ## {#RTCRtpScriptTransformer-interfaces}
<pre class="idl">
Expand Down Expand Up @@ -867,9 +870,9 @@ The <dfn constructor for="RTCRtpScriptTransform" lt="RTCRtpScriptTransform(worke
6. Let |serializedReadable| be the result of [$StructuredSerializeWithTransfer$](|t1|.`[[readable]]`, « |t1|.`[[readable]]` »).
7. Let |serializedWritable| be the result of [$StructuredSerializeWithTransfer$](|t2|.`[[writable]]`, « |t2|.`[[writable]]` »).
8. [=Queue a task=] on the DOM manipulation [=task source=] |worker|'s global scope to run the following steps:
1. Let |transformerOptions| be the result of [$StructuredDeserialize$](|serializedOptions|, the current Realm).
2. Let |readable| be the result of [$StructuredDeserialize$](|serializedReadable|, the current Realm).
3. Let |writable| be the result of [$StructuredDeserialize$](|serializedWritable|, the current Realm).
1. Let |transformerOptions| be the result of [$StructuredDeserializeWithTransfer$](|serializedOptions|, the current Realm).
2. Let |readable| be the result of [$StructuredDeserializeWithTransfer$](|serializedReadable|, the current Realm).
3. Let |writable| be the result of [$StructuredDeserializeWithTransfer$](|serializedWritable|, the current Realm).
4. Let |transformer| be a new {{RTCRtpScriptTransformer}}.
5. Set |transformer|.`[[options]]` to |transformerOptions|.
6. Set |transformer|.`[[readable]]` to |readable|.
Expand Down