stream: utility consumers for web and node.js streams

[jasnell]

@mcollina @ronag ... here's an alternative approach to the Body mixin stuff. Rather than providing the Body mixin directly, these introduce utility functions that can be used by the ecosystem to provide those basic methods, at least in part.

For a very rudimentary example...

const {
  arrayBuffer,
  blob,
  json,
  text,
} = require('stream/consumers');

const kStream = Symbol('kStream');

const MyObjectWithBodyMixin {

  arrayBuffer() { return arrayBuffer(this[kStream]); }

  blob() { return blob(this[kStream]); }

  json() { return json(this[kStream]); }

  text() { return text(this[kStream]); }

}

These work with ReadableStream, stream.Readable, and async interables.

There's likely a bit more error handling that could be added but I wanted to at least open the PR to give a basic idea.

[Mesteery]

Could a buffer(stream) function also be added?

[ronag]

LGTM. I think error and type checking is done indirectly.

[mcollina]

lgtm

[targos]

What happens if the functions are called on object streams?

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/39478/ 💚

[jasnell]

@targos:

What happens if the functions are called on object streams?

For blob(), buffer() and arrayBuffer(), the object is coerced using toString() rules. For text() and json() the promises reject because the objects cannot be decoded as utf8 byte sequences. I've extended the tests accordingly.

[jasnell]

Landed in c524107

[jimmywarting]

I wished that buffer() wasn't added
If one would want to make a user-land package out of consumers then you are depending on Buffer that don't fit well into Deno or Browsers.
If someone would have really wanted a Buffer then they could just have done: arrayBuffer(stream).then(Buffer.from)

I'm a bit biased towards Buffer in general cuz it isn't cross env friendly. and it's bloated with stuff TextEncoder and DataView is suppose to solve for you when working with typed arrays

[jimmywarting]

Would it be optimizable if arrayBuffer() had a totalLength option too?

There would not be pkg like this otherwise that don't need to take up twice the size when it's time to concatinate:
https://github.com/feross/stream-with-known-length-to-buffer

[targos]

Like #39134, this needs a volunteer to backport to v16.x-staging.

[Mesteery]

I'm willing to take care of it.

[Mesteery]

Isn't it already good?
https://github.com/nodejs/node/blob/v16.x-staging/lib/stream/consumers.js
@targos

[targos]

@Mesteery you're right, sorry. It looks like the backport-requested label was added by mistake