-
Notifications
You must be signed in to change notification settings - Fork 163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add async iterable<T>
type to WebIDL
#1397
base: main
Are you sure you want to change the base?
Conversation
This commit lifts the async iterator processing logic from the `ReadableStreamFromIterable` operation in the Streams spec, to a new WebIDL type `async iterable<T>`. This will clean up the streams spec, and enable a change in the Fetch spec to directly allow taking async iterable as request/response bodies, without having to pass them to `ReadableStream.from` first.
Has there been discussion somewhere publicly? 👀 |
@saschanaz No, not of any specifics. The background here is this issue: whatwg/fetch#1291 (comment). @annevk mentioned we'd need to figure out how to make this work IDL wise. This is my attempt at that (it enables updating the typedef (Blob or BufferSource or FormData or URLSearchParams or USVString) XMLHttpRequestBodyInit;
typedef (ReadableStream or XMLHttpRequestBodyInit) BodyInit; to typedef (Blob or BufferSource or FormData or URLSearchParams or USVString) XMLHttpRequestBodyInit;
typedef (ReadableStream or async iterable<BufferSource> or XMLHttpRequestBodyInit) BodyInit; |
Hmm, I see. BTW please include the full PR template: https://github.com/whatwg/webidl/blob/main/PULL_REQUEST_TEMPLATE.md |
I wonder how this ends up working in practice. Say you have |
What is being proposed exactly, to what WebIDL type would it be converted? A reference to an |
@annevk I have specified this behaviour in an update to the overload resolution algorithm in this PR already. The gist is: an I have updated the graphic in the spec that explains this too: In practice:
@petervanderbeken I am proposing the addition of an
If you imagine a matrix of JS stream primitives, with asynchronicity on one axis, and eagerness on the other, this fills out the square opposite of
The use case for this type are Web APIs that take "streams" of values (of unknown length), that they then process in some way. A real Web API that does this right now is
Introduction of this |
Per https://whatpr.org/webidl/1397/402a886...14f1bdb.html#js-async-iterable the result indeed seem to be iterator record. (Adding because I don't think the comment above actually answers this.) |
This design sounds good to me. It seems the Build is currently failing though. Could you write a corresponding PR for Streams to show it in action so we have some assurance it can be adopted? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking pretty good!
This would technically be a breaking change for That said, I wrote that test more for the sake of completeness, and I don't expect much actual usage in the wild. We can consider making |
@MattiasBuelens I was not aware of this behaviour in Streams. I think we should stick with the behaviour where Let's discuss whether to accept or error on |
@annevk I had already opened a draft PR to the streams spec, showing the integration yesterday. Here is a link: whatwg/streams#1310. |
Do we have Web IDL tests in WPT? I don't think so, right? I guess no tests specifically for this are relevant. Considering this is mostly an editorial refactor, do we even need "implementer interest"? There is nothing to implement until an upstream spec uses this. I'll ask for support anyway though:
Also @jasnell, any thoughts? |
Got some feedback from @saschanaz elsewhere that the constraint that |
We do, but they're weird since they test things via APIs that use those features. https://github.com/web-platform-tests/wpt/tree/master/webidl/ecmascript-binding
I think linking to the tests for streams using this feature would be good. Especially if you can verify that everything you introduce in this PR is covered by some aspect of the streams tests. (Which is probably the case since the streams tests are quite thorough, IIRC.)
Implementer support for Web IDL stuff like this is tricky and requires a judgement call. I think we should CC the bindings teams and give them a chance to weigh in, so @japhet and @yuki3 for Chromium. For example if they were concerned about how to implement this in the bindings layer, or if they were concerned that introducing this feature would be bad because then too many specs might use it and actually it should be rare. However, since this doesn't require any immediate changes to any implementations, I think we shouldn't block on requiring such support, if we don't hear back. |
Regarding strings, I think we should follow the behavior of async iterator helpers's |
The way And since it will not specifically reject strings, and strings have a (Sorry the spec for |
Thanks for the quick response. I'm unsure whether we should think of |
Personally I would specify this by making Plus if you take a string explicitly then you can use it directly instead of having to look up and invoke |
Thanks for the comments @domenic and @bakkot. I agree strongly with @bakkot that in the Web IDL type, we should only allow objects implementing the async iterable protocol for the For the streams specific discussion, lets move it right here: whatwg/streams#1310 (comment) |
Folks, could you take another look? I have clarified the allowed positions for async iterables (namely not in attributes), and have added guidance that: @annevk has confirmed implementer interest for WebKit on Matrix - is anyone else in favor, or opposed? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this! I think the theoretically-language-agnostic part is not quite separated nicely from the JS language part, and we can make the latter more rigorous. But the feedback is largely editorial.
index.bs
Outdated
<h4 id="idl-async-iterable-type" lt="async iterable" dfn export>Async iterable types — async iterable<|T|></h4> | ||
|
||
The <dfn lt="async iterable type" export>async iterable<|T|></dfn> type is a parameterized | ||
type whose values are references to objects that can produce an [=async iterator=]. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem to match how other parameterizable types are defined. I'd expect just "async iterable" as the dfn text.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That dfn
is on the heading. I copied this from sequence. It has two dfn:
sequence
in the headingsequence type
in the body
index.bs
Outdated
Async iterators, unlike sequences, do not have a fixed length and can be infinite. Values are | ||
asynchronously produced as the async iterator is iterated over. | ||
|
||
Async iterators are values, not types, and thus cannot be used as a type in IDL. They can only be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems pretty confusing. What type to the created values have, then?
I think this whole section is JS language binding specific, since it relates to algorithms like "open" and "get the next value" which are only defined for the JS language binding. So maybe we can remove this confusing not-a-type from the Web IDL "Types" section (2.13), and make it something produced and consumed by those algorithms?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I moved it down to the JS language section, and added some other wording up here to express that IDL async iterables are themselves not being iterated over, but produce a value that is being iterated over (without referencing [=async iterator=]
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks really nice now! Just a few more editorial tidbits. Also be sure to fill out the template in the OP.
@domenic What tests should I write for this? I don't think we have specific webidl tests in WPT, right? Only tests for features that use the webidl types. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What tests should I write for this? I don't think we have specific webidl tests in WPT, right? Only tests for features that use the webidl types.
This was discussed in #1397 (comment).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM (aside from one last nit)
Implements whatwg/webidl#1397 Fixes #21454 Closes #24849
Just to make sure I understand correctly, this is blocked on tests and this is best tested through the Streams change whatwg/streams#1310 which is currently still in draft. So once the Streams change is ready and has test coverage (which should include coverage for the IDL bits) this is all good to go? |
So during an experiment we ran for We have two options here:
The big question is whether, assuming Any thoughts? |
In TC39 we went the other direction, and said that String objects are just another kind of object. Having the check conceptually be "is a primitive" is the simplest mental model.
|
That also seems reasonable, but the problem is that any API currently accepting a string would need to juggle a lot of complexity to safely adopt Perhaps there's a third solution whereby |
I like the option where we just handle this in the union. |
an [=async iterator=]. The [=async iterator=] can be asynchronously iterated over to produce values. | ||
|
||
<dfn lt="async iterator" export>Async iterators</dfn> are [=structs=] with the following [=struct/items=]: | ||
* <dfn for="JS async iterator">underlying record</dfn>, an [=Iterator=] record |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ECMAScript routinely refers to these as the type "Iterator Record". The term is often linkable as "Iterator Record" too, so could we maybe follow suit and make this [=Iterator Record=]
instead?
|
||
1. Let |nextResult| be | ||
<a abstract-op>IteratorNext</a>(|iterator|'s [=JS async iterator/underlying record=]). | ||
1. If |nextResult| is an abrupt completion, return [=a promise rejected with=] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All references of "abrupt completion" in the current standard link to the ES term. Could we do that too here?
|
||
To <dfn id="async-iterator-close" export lt="close an async iterator">close</dfn> an | ||
<code><a lt="async iterator">async iterator<<var ignore>T</var>></a></code> |iterator|, | ||
with a reason |reason|: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe clarify that |reason|
is a JavaScript value (i.e., instead of a Web IDL value that would need to be converted), like we do in https://webidl.spec.whatwg.org/#a-promise-rejected-with?
We ran another experiment with changed behaviour where for unions of |
WebIDL `async iterable<T>` type rejects `string` Ref whatwg/webidl#1397, #24623
WebIDL `async iterable<T>` type rejects `string` Ref whatwg/webidl#1397, #24623
This commit lifts the async iterator processing logic from the
ReadableStreamFromIterable
operation in the Streams spec, to a new WebIDL typeasync iterable<T>
.This will clean up the streams spec, and enable a change in the Fetch spec to
directly allow taking async iterable as request/response bodies, without users
having to pass them to
ReadableStream.from
first.async iterable<T>
type to WebIDL #1397 (comment)async iterable<T>
WebIDL type denoland/deno#24849async iterable<T>
WebIDL type nodejs/node#54175async iterable<T>
type w3c/webidl2.js#775async iterable<T>
type plinss/widlparser#85(See WHATWG Working Mode: Changes for more details.)
Preview | Diff