From 48b6a70a2caafae3e6232038c3ef3d866a2d076d Mon Sep 17 00:00:00 2001 From: Bernard Aboba Date: Thu, 2 Feb 2023 16:23:24 -0800 Subject: [PATCH 1/7] Extend EncodedVideoChunkMetadata Fix for Issue https://github.com/w3c/webcodecs/issues/619 --- index.src.html | 48 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-) diff --git a/index.src.html b/index.src.html index a1542412..7d34f8a3 100644 --- a/index.src.html +++ b/index.src.html @@ -128,6 +128,16 @@ :: A grouping of {{EncodedVideoChunk}}s whose timestamp cadence produces a particular framerate. See {{VideoEncoderConfig/scalabilityMode}}. +: Spatial Layer +:: A grouping of {{EncodedVideoChunk}}s that produces a particular resolution. + See {{VideoEncoderConfig/scalabilityMode}}. + +: Decode Target +:: A numerical index determined by the encoder that indicates the set of frames + needed to decode a sequence of {{EncodedVideoChunk}}s at a given spatial and + temporal fidelity. Values do not necessarily correspond to a given temporal + or spatial layer. + : Progressive Image :: An image that supports decoding to multiple levels of detail, with lower levels becoming available while the encoded data is not yet fully buffered. @@ -1639,11 +1649,20 @@ |svc|.{{SvcOutputMetadata/temporalLayerId}}. 4. Assign |svc| to |chunkMetadata|.{{EncodedVideoChunkMetadata/svc}}. - 8. If |encoderConfig|.{{VideoEncoderConfig/alpha}} is set to `"keep"`: + 8. If |encoderConfig|.{{VideoEncoderConfig/scalabilityMode}} + describes multiple [=spatial layers=]: + 1. Let |svc| be a new {{SvcOutputMetadata}} instance. + 2. Let |spatial_layer_id| be the zero-based index describing the + spatial layer for |output|. + 3. Assign |spatial_layer_id| to + |svc|.{{SvcOutputMetadata/spatiallLayerId}}. + 4. Assign |svc| to + |chunkMetadata|.{{EncodedVideoChunkMetadata/svc}}. + 9. If |encoderConfig|.{{VideoEncoderConfig/alpha}} is set to `"keep"`: 1. Let |alphaSideData| be the encoded alpha data in |output|. 2. Assign |alphaSideData| to |chunkMetadata|.{{EncodedVideoChunkMetadata/alphaSideData}}. - 9. Invoke {{VideoEncoder/[[output callback]]}} with |chunk| and + 10. Invoke {{VideoEncoder/[[output callback]]}} with |chunk| and |chunkMetadata|.
Reset VideoEncoder (with |exception|)
@@ -1692,6 +1711,11 @@ dictionary SvcOutputMetadata { unsigned long temporalLayerId; + unsigned long spatiallayerId; + unsigned short frameNumber; + sequence dependsOnIds; + sequence decodeTargets; + map chainLinks; }; @@ -1711,6 +1735,26 @@ :: A number that identifies the [=temporal layer=] for the associated {{EncodedVideoChunk}}. +: spatialLayerId +:: A number that identifies the [=spatial layer=] for the associated + {{EncodedVideoChunk}}. + +: frameNumber +:: A number that identifies the frame in {{dependsOnIds}} and + {{chainLinks}} (for other {{EncodedVideoChunk}}s). + +: dependsOnIds +:: A sequence of {{frameNumber}} values that this {{EncodedVideoChunk}} + depends on. + +: decodeTargets +:: A sequence of [=Decode Target=] values that this {{EncodedVideoChunk}} + participates in. + +: chainLinks +:: A mapping of [=Decode Target=] values to the last important frame to + decode prior to this {{EncodedVideoChunk}}, for a given + [=Decode Target=] value. Configurations{#configurations} =============================== From d69d4c6bf571184a1ab83fbb995b9c4791832010 Mon Sep 17 00:00:00 2001 From: Bernard Aboba Date: Thu, 2 Feb 2023 16:33:05 -0800 Subject: [PATCH 2/7] Fix typos --- index.src.html | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/index.src.html b/index.src.html index 7d34f8a3..0c6b00a7 100644 --- a/index.src.html +++ b/index.src.html @@ -130,13 +130,13 @@ : Spatial Layer :: A grouping of {{EncodedVideoChunk}}s that produces a particular resolution. - See {{VideoEncoderConfig/scalabilityMode}}. + See {{VideoEncoderConfig/scalabilityMode}}. : Decode Target :: A numerical index determined by the encoder that indicates the set of frames - needed to decode a sequence of {{EncodedVideoChunk}}s at a given spatial and - temporal fidelity. Values do not necessarily correspond to a given temporal - or spatial layer. + needed to decode a sequence of {{EncodedVideoChunk}}s at a given spatial and + temporal fidelity. Values do not necessarily correspond to a given + [=Temporal Layer=] or [=Spatial Layer=]. : Progressive Image :: An image that supports decoding to multiple levels of detail, with lower @@ -1732,16 +1732,16 @@ channel data. : temporalLayerId -:: A number that identifies the [=temporal layer=] for the associated +:: A number that identifies the [=Temporal Layer=] for the associated {{EncodedVideoChunk}}. : spatialLayerId -:: A number that identifies the [=spatial layer=] for the associated +:: A number that identifies the [=Spatial Layer=] for the associated {{EncodedVideoChunk}}. : frameNumber :: A number that identifies the frame in {{dependsOnIds}} and - {{chainLinks}} (for other {{EncodedVideoChunk}}s). + {{chainLinks}} (for other {{EncodedVideoChunk}} values). : dependsOnIds :: A sequence of {{frameNumber}} values that this {{EncodedVideoChunk}} From e8ded9ce63d0389a29a24d0e9ff8cf17e3f3df18 Mon Sep 17 00:00:00 2001 From: Bernard Aboba Date: Wed, 1 Mar 2023 15:24:36 -0800 Subject: [PATCH 3/7] Add definitions of Chain and frame_number --- index.src.html | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/index.src.html b/index.src.html index 0c6b00a7..50235178 100644 --- a/index.src.html +++ b/index.src.html @@ -131,6 +131,14 @@ : Spatial Layer :: A grouping of {{EncodedVideoChunk}}s that produces a particular resolution. See {{VideoEncoderConfig/scalabilityMode}}. + +: Chain +:: A sequence of frames for which it can be determined instantly if a frame +from that sequence has been lost. + +: frame_number +:: A 16-bit number that increases strictly monotonically in decode order. +Defined in [[AV1-RTP]] Section A.8.3. : Decode Target :: A numerical index determined by the encoder that indicates the set of frames From 2bb690b2b3ba136f0bc37dbce0ff6ba655113d20 Mon Sep 17 00:00:00 2001 From: Bernard Aboba Date: Wed, 1 Mar 2023 15:28:38 -0800 Subject: [PATCH 4/7] Add Chain reference --- index.src.html | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/index.src.html b/index.src.html index 50235178..10136ec0 100644 --- a/index.src.html +++ b/index.src.html @@ -132,11 +132,11 @@ :: A grouping of {{EncodedVideoChunk}}s that produces a particular resolution. See {{VideoEncoderConfig/scalabilityMode}}. -: Chain +: chainLinks :: A sequence of frames for which it can be determined instantly if a frame -from that sequence has been lost. +from that sequence has been lost. Defined in [[AV1-RTP]] Section A.5. -: frame_number +: frameNumber :: A 16-bit number that increases strictly monotonically in decode order. Defined in [[AV1-RTP]] Section A.8.3. From ee286110077dd1cf87675f43676f1de48cba5c6c Mon Sep 17 00:00:00 2001 From: Bernard Aboba Date: Wed, 1 Mar 2023 15:34:01 -0800 Subject: [PATCH 5/7] Update definitions --- index.src.html | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/index.src.html b/index.src.html index 10136ec0..8eaa75fd 100644 --- a/index.src.html +++ b/index.src.html @@ -132,13 +132,14 @@ :: A grouping of {{EncodedVideoChunk}}s that produces a particular resolution. See {{VideoEncoderConfig/scalabilityMode}}. -: chainLinks -:: A sequence of frames for which it can be determined instantly if a frame -from that sequence has been lost. Defined in [[AV1-RTP]] Section A.5. +: Chains +:: {{chainLinks}} represents a sequence of frames for which it can be determined + instantly if a frame from that sequence has been lost. Defined in [[AV1-RTP]] + Section A.5. -: frameNumber -:: A 16-bit number that increases strictly monotonically in decode order. -Defined in [[AV1-RTP]] Section A.8.3. +: Frame Number +:: {{frameNumber}} is a 16-bit number that increases strictly monotonically in + decode order. Defined in [[AV1-RTP]] Section A.8.3. : Decode Target :: A numerical index determined by the encoder that indicates the set of frames @@ -1760,8 +1761,8 @@ participates in. : chainLinks -:: A mapping of [=Decode Target=] values to the last important frame to - decode prior to this {{EncodedVideoChunk}}, for a given +:: A mapping of [=Decode Target=] values to the last important frame + to decode prior to this {{EncodedVideoChunk}}, for a given [=Decode Target=] value. Configurations{#configurations} From 493144db9df32b18779768e0ce0355ea19affad1 Mon Sep 17 00:00:00 2001 From: Bernard Aboba Date: Wed, 15 Mar 2023 23:16:13 -0400 Subject: [PATCH 6/7] Fix AV1-RTP references --- index.src.html | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/index.src.html b/index.src.html index 8eaa75fd..03f5b18b 100644 --- a/index.src.html +++ b/index.src.html @@ -134,12 +134,12 @@ : Chains :: {{chainLinks}} represents a sequence of frames for which it can be determined - instantly if a frame from that sequence has been lost. Defined in [[AV1-RTP]] - Section A.5. + instantly if a frame from that sequence has been lost. Defined in Section A.5 + of [[AV1-RTP]]. : Frame Number :: {{frameNumber}} is a 16-bit number that increases strictly monotonically in - decode order. Defined in [[AV1-RTP]] Section A.8.3. + decode order. Defined in Section A.8.3 of [[AV1-RTP]]. : Decode Target :: A numerical index determined by the encoder that indicates the set of frames From 0a2ac63b4136285967fe03e4d4a20ba86e5bd037 Mon Sep 17 00:00:00 2001 From: Bernard Aboba Date: Wed, 15 Mar 2023 23:21:58 -0400 Subject: [PATCH 7/7] Add AV1-RTP reference --- index.src.html | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/index.src.html b/index.src.html index 03f5b18b..4eecb438 100644 --- a/index.src.html +++ b/index.src.html @@ -54,6 +54,15 @@ 
 {
+  "AV1-RTP": {
+    "title": "RTP Payload Format for AV1",
+    "href": "https://aomediacodec.github.io/av1-rtp-spec/",
+    "authors": [
+      "AV1 RTC SG"
+    ],
+    "status": "Standard",
+    "publisher": "Alliance for Open Media"
+  },
   "H.273": {
     "href": "https://www.itu.int/rec/T-REC-H.273/en",
     "title": "Coding-independent code points for video signal type identification",