Use async iterators to get response body

[szmarczak]

Just to utilize newest Node.js features :)

Checklist

• I have read the documentation.
• I have included a pull request description of my changes.
• I have included some tests.
• If it's a new feature, I have included documentation updates.

[sindresorhus]

Why not just do this directly in get-stream? We're planning to do the change there anyway.

[szmarczak]

Why not just do this directly in get-stream? We're planning to do the change there anyway.

Sure, I just want to test things before we make an actual release :)

[szmarczak]

I'll merge this for now. I think this may fix #1187.

[arcanis]

@szmarczak We upgraded got in Yarn, and I noticed it caused a breakage with all requests being returned with no content (it doesn't happen to other contributors, weirdly, even on the same Node versions; I'm on OSX, but our CI tests are passing there too). After investigating, the problem I experience appeared between the 11.3 and the 11.4 releases, and more specifically by this PR.

I can patch it locally by replacing the following:

for await (const chunk of stream) {
    chunks.push(chunk);
    length += Buffer.byteLength(chunk);
}

By this:

const {PassThrough} = require(`stream`);

const copy = new PassThrough();
stream.pipe(copy);

for await (const chunk of copy) {
    chunks.push(chunk);
    length += Buffer.byteLength(chunk);
}

I suspect this is because the various Node stream interfaces aren't meant to be used together, and weird effects are possible should that happen. I don't know if that's the case for sure here 🙁

[szmarczak]

@arcanis That seems weird. What Node.js version are you using? It'd be best to open a new issue so we don't lose the track here.

This looks like a Node.js bug.