Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: writable streams from async iterators example issue #31365

Closed
ronag opened this issue Jan 15, 2020 · 7 comments
Closed

docs: writable streams from async iterators example issue #31365

ronag opened this issue Jan 15, 2020 · 7 comments

Comments

@ronag
Copy link
Member

ronag commented Jan 15, 2020

Continuation of #31222

Further issues with the example. See comment:

async function pump(iterable, writable) {
  for await (const chunk of iterable) {
    // Handle backpressure on write().
    if (!writable.write(chunk)) {
      if (writable.destroyed) return;
      await once(writable, 'drain'); // BUG? This will never complete if writable is destroyed with `.destroy()`.
    }
  }
  writable.end();
}

The problem here is that it assumes that either 'drain' or 'error' will be emitted, however this is not always the case.

@ronag
Copy link
Member Author

ronag commented Jan 15, 2020

ping @mcollina

@ronag
Copy link
Member Author

ronag commented Jan 15, 2020

These are the solutions I can think of at the moment:

  1. Have destroy() emit a 'drain' before 'close' if needDrain === true.

await Promise.all([
  await once(writable, 'drain')
  await once(writable, 'close')
]);
  1. Have Writable emit premature close error if .destroy() is called before 'finish'

@mcollina
Copy link
Member

See openjs-foundation/summit#216 (comment) and the rest of the thread.

I think we should add a promisified write.

cc @benjamingr @apapirovski @davidmarkclements

@ronag
Copy link
Member Author

ronag commented Jan 15, 2020

I think we should add a promisified write.

Maybe remove this await once(writable, 'drain') example then? Since it is possibly broken.

@ronag
Copy link
Member Author

ronag commented Jan 15, 2020

See openjs-foundation/summit#216 (comment) and the rest of the thread.

This example has the same problem though... 'drain' might not be emitted.

EDIT: I think most of the suggestions in that thread have the same problem in assuming that either 'error' or 'drain' will be emitted if needDrain.

@mcollina
Copy link
Member

We might have to listen for 'finish' as well. Would you mind to send a PR to add that? I think it would fix the issue.

@ronag
Copy link
Member Author

ronag commented Jan 15, 2020

Do we and/or should we emit 'finish' when abruptly closing a writable with .destroy()? i.e. should end() and destroy() mean the same thing for a Writable?

I guess we should in order to not break... since we don't error it in this case. Just seems a little weird.

I think I would have preferred the "Have Writable emit premature close error if .destroy() is called before 'finish'" option. Though that's probably to breaking.

ronag added a commit to nxtedition/node that referenced this issue Jan 15, 2020
Further fixes an issue with the async iterator example where an
incorrect assumption was made in regards that drain or error
is always invoked after !write().

Fixes: nodejs#31365
@ronag ronag closed this as completed in 4fefd18 Jan 24, 2020
codebytere pushed a commit that referenced this issue Feb 17, 2020
Further fixes an issue with the async iterator example where an
incorrect assumption was made in regards that drain or error
is always invoked after !write().

Fixes: #31365

PR-URL: #31367
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
codebytere pushed a commit that referenced this issue Mar 15, 2020
Further fixes an issue with the async iterator example where an
incorrect assumption was made in regards that drain or error
is always invoked after !write().

Fixes: #31365

PR-URL: #31367
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
codebytere pushed a commit that referenced this issue Mar 17, 2020
Further fixes an issue with the async iterator example where an
incorrect assumption was made in regards that drain or error
is always invoked after !write().

Fixes: #31365

PR-URL: #31367
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants