Skip to content

Allow other specifications to create readable byte streams #1130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
domenic merged 9 commits into from
Jul 15, 2021
Merged

Allow other specifications to create readable byte streams #1130

Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
195d45a
Allow other specifications to create readable byte streams
domenic Jun 2, 2021
60c37bb
Enqueue a view
domenic Jul 12, 2021
80c1824
Update index.bs
domenic Jul 12, 2021
3efb6a2
Update index.bs
domenic Jul 12, 2021
ca10093
Update index.bs
domenic Jul 12, 2021
8c892e5
Add assert
domenic Jul 12, 2021
df3493d
Update index.bs
domenic Jul 14, 2021
a5a82b6
Consolidate the algorithms
domenic Jul 14, 2021
b6e8404
Disallow transferring/detaching
domenic Jul 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
173 changes: 142 additions & 31 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ urlPrefix: https://tc39.es/ecma262/; spec: ECMASCRIPT
text: the typed array constructors table; url: #table-49
text: typed array; url: #sec-typedarray-objects
text: Number type; url: #sec-ecmascript-language-types-number-type
text: Data Block; url: #sec-data-blocks
type: abstract-op
text: CloneArrayBuffer; url: #sec-clonearraybuffer
text: CopyDataBlockBytes; url: #sec-copydatablockbytes
Expand Down Expand Up @@ -165,9 +166,9 @@ readable stream tee">branches</dfn>, which can be consumed independently.
For streams representing bytes, an extended version of the [=readable stream=] is provided to handle
bytes efficiently, in particular by minimizing copies. The [=underlying source=] for such a readable
stream is called an <dfn>underlying byte source</dfn>. A readable stream whose underlying source is
an underlying byte source is sometimes called a <dfn>readable byte stream</dfn>. Consumers of a
readable byte stream can acquire a [=BYOB reader=] using the stream's {{ReadableStream/getReader()}}
method.
an underlying byte source is sometimes called a <dfn export>readable byte stream</dfn>. Consumers of
a readable byte stream can acquire a [=BYOB reader=] using the stream's
{{ReadableStream/getReader()}} method.

<h3 id="ws-model">Writable streams</h3>

Expand Down Expand Up @@ -309,11 +310,12 @@ object that allows direct reading of [=chunks=] from a [=readable stream=]. With
stream|canceling=] the stream, or [=piping=] the readable stream to a writable stream. A reader is
acquired via the stream's {{ReadableStream/getReader()}} method.

A [=readable byte stream=] has the ability to vend two types of readers: <dfn>default readers</dfn>
and <dfn>BYOB readers</dfn>. BYOB ("bring your own buffer") readers allow reading into a
developer-supplied buffer, thus minimizing copies. A non-byte readable stream can only vend default
readers. Default readers are instances of the {{ReadableStreamDefaultReader}} class, while BYOB
readers are instances of {{ReadableStreamBYOBReader}}.
A [=readable byte stream=] has the ability to vend two types of readers: <dfn export lt="default
reader">default readers</dfn> and <dfn export lt="BYOB reader">BYOB readers</dfn>. BYOB ("bring your
own buffer") readers allow reading into a developer-supplied buffer, thus minimizing copies. A
non-byte readable stream can only vend default readers. Default readers are instances of the
{{ReadableStreamDefaultReader}} class, while BYOB readers are instances of
{{ReadableStreamBYOBReader}}.

Similarly, a <dfn lt="writer|writable stream writer">writable stream writer</dfn>, or simply
writer, is an object that allows direct writing of [=chunks=] to a [=writable stream=]. Without a
Expand Down Expand Up @@ -3244,8 +3246,9 @@ The following abstract operations support the implementation of the
</div>

<div algorithm>
<dfn abstract-op lt="ReadableByteStreamControllerGetBYOBRequest">ReadableByteStreamControllerGetBYOBRequest(|controller|)</dfn>
performs the following steps:
<dfn abstract-op
lt="ReadableByteStreamControllerGetBYOBRequest">ReadableByteStreamControllerGetBYOBRequest(|controller|)</dfn> performs
the following steps:

1. If |controller|.[=ReadableByteStreamController/[[byobRequest]]=] is null and
|controller|.[=ReadableByteStreamController/[[pendingPullIntos]]=] is not [=list/is empty|empty=],
Expand Down Expand Up @@ -6463,7 +6466,7 @@ to grow organically as needed.
{{ReadableStream}} object |stream|, given an optional algorithm <dfn export for="ReadableStream/set
up"><var>pullAlgorithm</var></dfn>, an optional algorithm <dfn export for="ReadableStream/set
up"><var>cancelAlgorithm</var></dfn>, an optional number <dfn export for="ReadableStream/set
up"><var>highWaterMark</var></dfn> (default 1), an optional algorithm <dfn export
up"><var>highWaterMark</var></dfn> (default 1), and an optional algorithm <dfn export
for="ReadableStream/set up"><var>sizeAlgorithm</var></dfn>, perform the following steps. If
given, |pullAlgorithm| and |cancelAlgorithm| may return a promise. If given, |sizeAlgorithm| must
be an algorithm accepting [=chunk=] objects and returning a number; and if given, |highWaterMark|
Expand All @@ -6481,42 +6484,149 @@ to grow organically as needed.
1. If |result| is a {{Promise}}, then return |result|.
1. Return [=a promise resolved with=] undefined.
1. If |sizeAlgorithm| was not given, then set it to an algorithm that returns 1.
1. Perform [$InitializeReadableStream$](|stream|).
1. Perform ! [$InitializeReadableStream$](|stream|).
1. Let |controller| be a [=new=] {{ReadableStreamDefaultController}}.
1. Perform ! [$SetUpReadableStreamDefaultController$](|stream|, |controller|, |startAlgorithm|,
|pullAlgorithmWrapper|, |cancelAlgorithmWrapper|, |highWaterMark|, |sizeAlgorithm|).
</div>

<div class="example" id="example-set-up-rs">
Creating a {{ReadableStream}} from other specifications is thus a two-step process, like so:
<div algorithm="set up a byte-source ReadableStream">
To <dfn export for="ReadableStream">set up with byte reading support</dfn> a
newly-[=new|created-via-Web IDL=] {{ReadableStream}} object |stream|, given an optional algorithm
<dfn export for="ReadableStream/set up with byte reading support"><var>pullAlgorithm</var></dfn>,
an optional algorithm <dfn export for="ReadableStream/set up with byte reading
support"><var>cancelAlgorithm</var></dfn>, and an optional number <dfn export
for="ReadableStream/set up with byte reading support"><var>highWaterMark</var></dfn> (default 0),
perform the following steps. If given, |pullAlgorithm| and |cancelAlgorithm| may return a promise.
If given, |highWaterMark| must be a non-negative, non-NaN number.

1. Let |readableStream| be a [=new=] {{ReadableStream}}.
1. [=ReadableStream/Set up=] |readableStream| given….
</div>
1. Let |startAlgorithm| be an algorithm that returns undefined.
1. Let |pullAlgorithmWrapper| be an algorithm that runs these steps:
1. Let |result| be the result of running |pullAlgorithm|, if |pullAlgorithm| was given, or null
otherwise. If this throws an exception |e|, return [=a promise rejected with=] |e|.
1. If |result| is a {{Promise}}, then return |result|.
1. Return [=a promise resolved with=] undefined.
Comment thread
domenic marked this conversation as resolved.
1. Let |cancelAlgorithmWrapper| be an algorithm that runs these steps:
1. Let |result| be the result of running |cancelAlgorithm|, if |cancelAlgorithm| was given, or
null otherwise. If this throws an exception |e|, return [=a promise rejected with=] |e|.
1. If |result| is a {{Promise}}, then return |result|.
1. Return [=a promise resolved with=] undefined.
1. Perform ! [$InitializeReadableStream$](|stream|).
1. Let |controller| be a [=new=] {{ReadableByteStreamController}}.
1. Perform ! [$SetUpReadableByteStreamController$](|stream|, |controller|, |startAlgorithm|,
|pullAlgorithmWrapper|, |cancelAlgorithmWrapper|, |highWaterMark|, undefined).
</div>

<p class="note">Subclasses of {{ReadableStream}} will use the [=ReadableStream/set up=] operation
directly on the [=this=] value inside their constructor steps.</p>
<div class="example" id="example-set-up-rs">
Creating a {{ReadableStream}} from other specifications is thus a two-step process, like so:

1. Let |readableStream| be a [=new=] {{ReadableStream}}.
1. [=ReadableStream/Set up=] |readableStream| given….
</div>

<p class="note">Subclasses of {{ReadableStream}} will use the [=ReadableStream/set up=] or
[=ReadableStream/set up with byte reading support=] operations directly on the [=this=] value inside
their constructor steps.

<hr>

The following algorithms must only be used on {{ReadableStream}} instances initialized via the above
[=ReadableStream/set up=] algorithm:
[=ReadableStream/set up=] or [=ReadableStream/set up with byte reading support=] algorithms (not,
e.g., on web-developer-created instances):

<div algorithm>
A {{ReadableStream}} |stream|'s <dfn export for="ReadableStream">desired size to fill up to the
high water mark</dfn> is the result of running the following steps:

1. If |stream| is not [=ReadableStream/readable=], then return 0.
1. If |stream|.[=ReadableStream/[[controller]]=] [=implements=] {{ReadableByteStreamController}},
then return !
[$ReadableByteStreamControllerGetDesiredSize$](|stream|.[=ReadableStream/[[controller]]=]).
1. Return !
[$ReadableStreamDefaultControllerGetDesiredSize$](|stream|.[=ReadableStream/[[controller]]=]).
</div>

<p algorithm>A {{ReadableStream}} <dfn export for="ReadableStream" lt="need more data|needs
more data">needs more data</dfn> if its [=ReadableStream/desired size to fill up to the high water
mark=] is greater than zero.

<p algorithm>A {{ReadableStream}} |stream| <dfn export for="ReadableStream" lt="need more
data|needs more data">needs more data</dfn> if |stream| is [=ReadableStream/readable=] and !
[$ReadableStreamDefaultControllerGetDesiredSize$](|stream|.[=ReadableStream/[[controller]]=])
returns a positive number.
<div algorithm>
To <dfn export for="ReadableStream">close</dfn> a {{ReadableStream}} |stream|:

1. If |stream|.[=ReadableStream/[[controller]]=] [=implements=] {{ReadableByteStreamController}},
then perform !
[$ReadableByteStreamControllerClose$](|stream|.[=ReadableStream/[[controller]]=]).
Comment thread
domenic marked this conversation as resolved.
Outdated
1. Perform ! [$ReadableStreamDefaultControllerClose$](|stream|.[=ReadableStream/[[controller]]=]).
Comment thread
domenic marked this conversation as resolved.
Outdated
</div>

<div algorithm>
To <dfn export for="ReadableStream">error</dfn> a {{ReadableStream}} |stream| given a JavaScript
value |e|:

1. If |stream|.[=ReadableStream/[[controller]]=] [=implements=] {{ReadableByteStreamController}},
Copy link
Copy Markdown
Collaborator

@ricea ricea Jul 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this switching on controller type is unfortunate. If we had more types we might add some polymorphism to avoid it. But with only two types this is probably simpler.

then perform ! [$ReadableByteStreamControllerError$](|stream|.[=ReadableStream/[[controller]]=],
|e|).
1. Perform ! [$ReadableStreamDefaultControllerError$](|stream|.[=ReadableStream/[[controller]]=],
Comment thread
domenic marked this conversation as resolved.
Outdated
|e|).
</div>

<hr>

The following algorithm must only be used on {{ReadableStream}} instances initialized via the above
[=ReadableStream/set up=] algorithm:

<p algorithm>To <dfn export for="ReadableStream">enqueue</dfn> the JavaScript value |chunk| into a
{{ReadableStream}} |stream|, perform !
[$ReadableStreamDefaultControllerEnqueue$](|stream|.[=ReadableStream/[[controller]]=], |chunk|).
Comment thread
domenic marked this conversation as resolved.
Outdated

<p algorithm>To <dfn export for="ReadableStream">close</dfn> a {{ReadableStream}} |stream|, perform
! [$ReadableStreamDefaultControllerClose$](|stream|.[=ReadableStream/[[controller]]=]).
<hr>

The following algorithms must only be used on {{ReadableStream}} instances initialized via the above
[=ReadableStream/set up with byte reading support=] algorithm:

<div algorithm>
The <dfn export for="ReadableStream">current BYOB request view</dfn> for a
{{ReadableStream}} |stream| is either an {{ArrayBufferView}} or null, determined by the following
steps:

1. Assert: |stream|.[=ReadableStream/[[controller]]=] [=implements=]
{{ReadableByteStreamController}}.
1. Let |byobRequest| be !
[$ReadableByteStreamControllerGetBYOBRequest$](|stream|.[=ReadableStream/[[controller]]=]).
1. If |byobRequest| is null, then return null.
1. Return |byobRequest|.[=ReadableStreamBYOBRequest/[[view]]=].
</div>

<div algorithm>
To <dfn export for="ReadableStream">enqueue a view</dfn> into a {{ReadableStream}} |stream| given
an {{ArrayBufferView}} |view|:

1. Assert: |stream|.[=ReadableStream/[[controller]]=] [=implements=]
{{ReadableByteStreamController}}.
1. Let |byobRequest| be !
[$ReadableByteStreamControllerGetBYOBRequest$](|stream|.[=ReadableStream/[[controller]]=]).
1. If |byobRequest| is non-null, and |view|.\[[ViewedArrayBuffer]].\[[ArrayBufferData]] is the same
[=Data Block=] as |byobRequest|.\[[view]].\[[ViewedArrayBuffer]].\[[ArrayBufferData]], then:
1. Assert: |view|.\[[ByteOffset]] is |byobRequest|.\[[view]].\[[ByteOffset]].
1. Assert: |view|.\[[ByteLength]] ≤ |byobRequest|.\[[view]].\[[ByteLength]].
Copy link
Copy Markdown
Member Author

@domenic domenic Jul 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made these asserts instead of if tests (compared to w3c/webtransport#297 (comment)) since it seems like it would be bad if the caller used the view but went past the requested bounds.

<p class="note">These asserts ensure that the caller does not write outside the requested
range in the [=ReadableStream/current BYOB request view=].
1. Perform ?
[$ReadableByteStreamControllerRespondWithNewView$](|stream|.[=ReadableStream/[[controller]]=],
|view|).
Copy link
Copy Markdown
Member Author

@domenic domenic Jul 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I handled the case where you transfer the backing Data Block here, which goes a bit beyond w3c/webtransport#297 (comment) but seems like a good idea.

<p class="note">This works fine even in the case where |view|.\[[ViewedArrayBuffer]] equals
|byobRequest|.\[[view]].\[[ViewedArrayBuffer]].
1. Otherwise, perform ?
[$ReadableByteStreamControllerEnqueue$](|stream|.[=ReadableStream/[[controller]]=], |view|).
</div>

Specifications should, when possible, [=ArrayBufferView/write=] into the [=ReadableStream/current
BYOB request view=] when it is non-null, and then call [=enqueue bytes=] with that view. They should
only [=ArrayBufferView/create=] a new {{ArrayBufferView}} to pass to [=enqueue bytes=] when the
Comment thread
domenic marked this conversation as resolved.
Outdated
[=ReadableStream/current BYOB request view=] is null, or when they have more bytes on hand than the
[=ReadableStream/current BYOB request view=]'s [=BufferSource/byte length=]. This avoids unnecessary
copies and better respects the wishes of the stream's [=consumer=].

<p algorithm>To <dfn export for="ReadableStream">error</dfn> a {{ReadableStream}} |stream| given a
JavaScript value |e|, perform !
[$ReadableStreamDefaultControllerError$](|stream|.[=ReadableStream/[[controller]]=], |e|).

<h4 id="other-specs-rs-reading">Reading</h4>

Expand Down Expand Up @@ -6573,7 +6683,8 @@ a chunk</dfn> from a {{ReadableStreamDefaultReader}} |reader|, given a [=read re

<p class="note">Because |reader| grants exclusive access to its corresponding {{ReadableStream}},
the actual mechanism of how to read cannot be observed. Implementations could use a more direct
mechanism if convenient.
mechanism if convenient, such as acquiring and using a {{ReadableStreamBYOBReader}} instead of a
{{ReadableStreamDefaultReader}}, or accessing the chunks directly.
</div>

<p algorithm>To <dfn export for="ReadableStreamDefaultReader">release</dfn> a
Expand Down Expand Up @@ -6659,7 +6770,7 @@ for="ReadableStream">locked</dfn> if ! [$IsReadableStreamLocked$](|stream|) retu
1. If |result| is a {{Promise}}, then return |result|.
1. Return [=a promise resolved with=] undefined.
1. If |sizeAlgorithm| was not given, then set it to an algorithm that returns 1.
1. Perform [$InitializeWritableStream$](|stream|).
1. Perform ! [$InitializeWritableStream$](|stream|).
1. Let |controller| be a [=new=] {{WritableStreamDefaultController}}.
1. Perform ! [$SetUpWritableStreamDefaultController$](|stream|, |controller|, |startAlgorithm|,
|writeAlgorithm|, |closeAlgorithmWrapper|, |abortAlgorithmWrapper|, |highWaterMark|,
Expand Down