stream: support typed arrays #51866

IlyasShabi · 2024-02-24T23:56:49Z

This PR aims to add support for typed arrays in streams, addressing this issue #51728
cc: @nodejs/streams

nodejs-github-bot · 2024-02-24T23:56:53Z

Review requested:

@nodejs/streams

mcollina

Can you also update the docs?

lib/stream.js

nodejs-github-bot · 2024-02-25T11:45:17Z

CI: https://ci.nodejs.org/job/node-test-pull-request/57401/

IlyasShabi · 2024-02-26T21:09:14Z

@mcollina could you please check again ?

nodejs-github-bot · 2024-02-27T08:38:20Z

CI: https://ci.nodejs.org/job/node-test-pull-request/57448/

mcollina

lgtm

rluvaton

Thank you for your contribution, left minor comment

lib/internal/streams/readable.js

lib/internal/streams/writable.js

addaleax · 2024-02-27T22:52:24Z

doc/api/stream.md

-to work with other types of JavaScript values (with the exception of `null`,
-which serves a special purpose within streams). Such streams are considered to
-operate in "object mode".
+(or {TypedArray} and {DataView}) objects. It is possible, however, for stream


You'll probably want to write down what writing a TypedArray actually does, because there are at least two obvious potential options (values or underlying bytes) and the docs currently don't say which it is. :)

ShogunPanda

LGTM!

benjamingr

Generally LGTM, please address Anna's comment and run benchmarks to see if this causes a regression in Uint8Array usage (let me know if you need instructions on how to do those) :)

IlyasShabi · 2024-03-02T21:38:04Z

I have made some updates to the PR. Could you please review it again? 🙏
@addaleax @benjamingr I also changed the doc and added benchmarks

IlyasShabi · 2024-03-02T21:41:53Z

local benchmark results:

                                                              confidence improvement accuracy (*)   (**)  (***)
streams/readable-uint8array.js kind='encoding' n=1000000                    -1.45 %       ±2.62% ±3.49% ±4.54%
streams/readable-uint8array.js kind='read' n=1000000                        -0.70 %       ±2.46% ±3.28% ±4.27%
streams/writable-uint8array.js kind='object-mode' n=50000000                -0.36 %       ±0.67% ±0.90% ±1.17%
streams/writable-uint8array.js kind='write' n=50000000                       1.18 %       ±1.27% ±1.70% ±2.22%
streams/writable-uint8array.js kind='writev' n=50000000               *      1.26 %       ±1.21% ±1.61% ±2.10%

nodejs-github-bot · 2024-03-20T17:27:30Z

Landed in 7d258db

github-actions · 2024-03-21T11:17:03Z

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @mcollina.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

mcollina · 2024-03-21T11:22:36Z

Note that this broke undici extensive test suite. Out of caution, I'm going to add the backing--for-lts label

PR-URL: nodejs#51866 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Raz Luvaton <rluvaton@gmail.com> Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

PR-URL: #51866 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Raz Luvaton <rluvaton@gmail.com> Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

Notable changes: benchmark: * add AbortSignal.abort benchmarks (Raz Luvaton) #52408 buffer: * improve `base64` and `base64url` performance (Yagiz Nizipli) #52428 crypto: * deprecate implicitly shortened GCM tags (Tobias Nießen) #52345 deps: * (SEMVER-MINOR) update simdutf to 5.0.0 (Daniel Lemire) #52138 * (SEMVER-MINOR) update undici to 6.3.0 (Node.js GitHub Bot) #51462 * (SEMVER-MINOR) update undici to 6.2.1 (Node.js GitHub Bot) #51278 dns: * (SEMVER-MINOR) add order option and support ipv6first (Paolo Insogna) #52492 doc: * update release gpg keyserver (marco-ippolito) #52257 * add release key for marco-ippolito (marco-ippolito) #52257 * add UlisesGascon as a collaborator (Ulises Gascón) #51991 * (SEMVER-MINOR) deprecate fs.Stats public constructor (Marco Ippolito) #51879 events,doc: * mark CustomEvent as stable (Daeyeon Jeong) #52618 fs: * add stacktrace to fs/promises (翠 / green) #49849 lib, url: * (SEMVER-MINOR) add a `windows` option to path parsing (Aviv Keller) #52509 net: * (SEMVER-MINOR) add CLI option for autoSelectFamilyAttemptTimeout (Paolo Insogna) #52474 report: * (SEMVER-MINOR) add `--report-exclude-network` option (Ethan Arrowood) #51645 src: * (SEMVER-MINOR) add `string_view` overload to snapshot FromBlob (Anna Henningsen) #52595 * (SEMVER-MINOR) add C++ ProcessEmitWarningSync() (Joyee Cheung) #51977 * (SEMVER-MINOR) add uv_get_available_memory to report and process (theanarkh) #52023 * (SEMVER-MINOR) preload function for Environment (Cheng Zhao) #51539 stream: * (SEMVER-MINOR) support typed arrays (IlyasShabi) #51866 test_runner: * (SEMVER-MINOR) add suite() (Colin Ihrig) #52127 * (SEMVER-MINOR) support forced exit (Colin Ihrig) #52038 * (SEMVER-MINOR) add `test:complete` event to reflect execution order (Moshe Atlow) #51909 util: * (SEMVER-MINOR) support array of formats in util.styleText (Marco Ippolito) #52040 v8: * (SEMVER-MINOR) implement v8.queryObjects() for memory leak regression testing (Joyee Cheung) #51927 watch: * mark as stable (Moshe Atlow) #52074 PR-URL: TODO

Notable changes: benchmark: * add AbortSignal.abort benchmarks (Raz Luvaton) #52408 buffer: * improve `base64` and `base64url` performance (Yagiz Nizipli) #52428 crypto: * deprecate implicitly shortened GCM tags (Tobias Nießen) #52345 deps: * (SEMVER-MINOR) update simdutf to 5.0.0 (Daniel Lemire) #52138 * (SEMVER-MINOR) update undici to 6.3.0 (Node.js GitHub Bot) #51462 * (SEMVER-MINOR) update undici to 6.2.1 (Node.js GitHub Bot) #51278 dns: * (SEMVER-MINOR) add order option and support ipv6first (Paolo Insogna) #52492 doc: * update release gpg keyserver (marco-ippolito) #52257 * add release key for marco-ippolito (marco-ippolito) #52257 * add UlisesGascon as a collaborator (Ulises Gascón) #51991 * (SEMVER-MINOR) deprecate fs.Stats public constructor (Marco Ippolito) #51879 events,doc: * mark CustomEvent as stable (Daeyeon Jeong) #52618 fs: * add stacktrace to fs/promises (翠 / green) #49849 lib, url: * (SEMVER-MINOR) add a `windows` option to path parsing (Aviv Keller) #52509 net: * (SEMVER-MINOR) add CLI option for autoSelectFamilyAttemptTimeout (Paolo Insogna) #52474 report: * (SEMVER-MINOR) add `--report-exclude-network` option (Ethan Arrowood) #51645 src: * (SEMVER-MINOR) add `string_view` overload to snapshot FromBlob (Anna Henningsen) #52595 * (SEMVER-MINOR) add C++ ProcessEmitWarningSync() (Joyee Cheung) #51977 * (SEMVER-MINOR) add uv_get_available_memory to report and process (theanarkh) #52023 * (SEMVER-MINOR) preload function for Environment (Cheng Zhao) #51539 stream: * (SEMVER-MINOR) support typed arrays (IlyasShabi) #51866 test_runner: * (SEMVER-MINOR) add suite() (Colin Ihrig) #52127 * (SEMVER-MINOR) support forced exit (Colin Ihrig) #52038 * (SEMVER-MINOR) add `test:complete` event to reflect execution order (Moshe Atlow) #51909 util: * (SEMVER-MINOR) support array of formats in util.styleText (Marco Ippolito) #52040 v8: * (SEMVER-MINOR) implement v8.queryObjects() for memory leak regression testing (Joyee Cheung) #51927 watch: * mark as stable (Moshe Atlow) #52074 PR-URL: #52793

PR-URL: #51866 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Raz Luvaton <rluvaton@gmail.com> Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

Notable changes: benchmark: * add AbortSignal.abort benchmarks (Raz Luvaton) #52408 buffer: * improve `base64` and `base64url` performance (Yagiz Nizipli) #52428 crypto: * deprecate implicitly shortened GCM tags (Tobias Nießen) #52345 deps: * (SEMVER-MINOR) update simdutf to 5.0.0 (Daniel Lemire) #52138 * (SEMVER-MINOR) update undici to 6.3.0 (Node.js GitHub Bot) #51462 * (SEMVER-MINOR) update undici to 6.2.1 (Node.js GitHub Bot) #51278 dns: * (SEMVER-MINOR) add order option and support ipv6first (Paolo Insogna) #52492 doc: * update release gpg keyserver (marco-ippolito) #52257 * add release key for marco-ippolito (marco-ippolito) #52257 * add UlisesGascon as a collaborator (Ulises Gascón) #51991 * (SEMVER-MINOR) deprecate fs.Stats public constructor (Marco Ippolito) #51879 events,doc: * mark CustomEvent as stable (Daeyeon Jeong) #52618 fs: * add stacktrace to fs/promises (翠 / green) #49849 lib, url: * (SEMVER-MINOR) add a `windows` option to path parsing (Aviv Keller) #52509 net: * (SEMVER-MINOR) add CLI option for autoSelectFamilyAttemptTimeout (Paolo Insogna) #52474 report: * (SEMVER-MINOR) add `--report-exclude-network` option (Ethan Arrowood) #51645 src: * (SEMVER-MINOR) add `string_view` overload to snapshot FromBlob (Anna Henningsen) #52595 * (SEMVER-MINOR) add C++ ProcessEmitWarningSync() (Joyee Cheung) #51977 * (SEMVER-MINOR) add uv_get_available_memory to report and process (theanarkh) #52023 * (SEMVER-MINOR) preload function for Environment (Cheng Zhao) #51539 stream: * (SEMVER-MINOR) support typed arrays (IlyasShabi) #51866 test_runner: * (SEMVER-MINOR) add suite() (Colin Ihrig) #52127 * (SEMVER-MINOR) add `test:complete` event to reflect execution order (Moshe Atlow) #51909 util: * (SEMVER-MINOR) support array of formats in util.styleText (Marco Ippolito) #52040 v8: * (SEMVER-MINOR) implement v8.queryObjects() for memory leak regression testing (Joyee Cheung) #51927 watch: * mark as stable (Moshe Atlow) #52074 PR-URL: #52793

Notable changes: benchmark: * add AbortSignal.abort benchmarks (Raz Luvaton) nodejs#52408 buffer: * improve `base64` and `base64url` performance (Yagiz Nizipli) nodejs#52428 crypto: * deprecate implicitly shortened GCM tags (Tobias Nießen) nodejs#52345 deps: * (SEMVER-MINOR) update simdutf to 5.0.0 (Daniel Lemire) nodejs#52138 * (SEMVER-MINOR) update undici to 6.3.0 (Node.js GitHub Bot) nodejs#51462 * (SEMVER-MINOR) update undici to 6.2.1 (Node.js GitHub Bot) nodejs#51278 dns: * (SEMVER-MINOR) add order option and support ipv6first (Paolo Insogna) nodejs#52492 doc: * update release gpg keyserver (marco-ippolito) nodejs#52257 * add release key for marco-ippolito (marco-ippolito) nodejs#52257 * add UlisesGascon as a collaborator (Ulises Gascón) nodejs#51991 * (SEMVER-MINOR) deprecate fs.Stats public constructor (Marco Ippolito) nodejs#51879 events,doc: * mark CustomEvent as stable (Daeyeon Jeong) nodejs#52618 fs: * add stacktrace to fs/promises (翠 / green) nodejs#49849 lib, url: * (SEMVER-MINOR) add a `windows` option to path parsing (Aviv Keller) nodejs#52509 net: * (SEMVER-MINOR) add CLI option for autoSelectFamilyAttemptTimeout (Paolo Insogna) nodejs#52474 report: * (SEMVER-MINOR) add `--report-exclude-network` option (Ethan Arrowood) nodejs#51645 src: * (SEMVER-MINOR) add `string_view` overload to snapshot FromBlob (Anna Henningsen) nodejs#52595 * (SEMVER-MINOR) add C++ ProcessEmitWarningSync() (Joyee Cheung) nodejs#51977 * (SEMVER-MINOR) add uv_get_available_memory to report and process (theanarkh) nodejs#52023 * (SEMVER-MINOR) preload function for Environment (Cheng Zhao) nodejs#51539 stream: * (SEMVER-MINOR) support typed arrays (IlyasShabi) nodejs#51866 test_runner: * (SEMVER-MINOR) add suite() (Colin Ihrig) nodejs#52127 * (SEMVER-MINOR) add `test:complete` event to reflect execution order (Moshe Atlow) nodejs#51909 util: * (SEMVER-MINOR) support array of formats in util.styleText (Marco Ippolito) nodejs#52040 v8: * (SEMVER-MINOR) implement v8.queryObjects() for memory leak regression testing (Joyee Cheung) nodejs#51927 watch: * mark as stable (Moshe Atlow) nodejs#52074 PR-URL: nodejs#52793

targos · 2024-05-08T14:40:59Z

@mcollina This landed on v20.13.0 by mistake. Should we revert it?

mcollina · 2024-05-08T15:23:40Z

I would not revert now, but wait if there are bugs coming out of it. If there are, we revert.

(If you are assembling a revert release it might be good to be cautious, but don’t do one for this).

Notable changes: benchmark: * add AbortSignal.abort benchmarks (Raz Luvaton) nodejs#52408 buffer: * improve `base64` and `base64url` performance (Yagiz Nizipli) nodejs#52428 crypto: * deprecate implicitly shortened GCM tags (Tobias Nießen) nodejs#52345 deps: * (SEMVER-MINOR) update simdutf to 5.0.0 (Daniel Lemire) nodejs#52138 * (SEMVER-MINOR) update undici to 6.3.0 (Node.js GitHub Bot) nodejs#51462 * (SEMVER-MINOR) update undici to 6.2.1 (Node.js GitHub Bot) nodejs#51278 dns: * (SEMVER-MINOR) add order option and support ipv6first (Paolo Insogna) nodejs#52492 doc: * update release gpg keyserver (marco-ippolito) nodejs#52257 * add release key for marco-ippolito (marco-ippolito) nodejs#52257 * add UlisesGascon as a collaborator (Ulises Gascón) nodejs#51991 * (SEMVER-MINOR) deprecate fs.Stats public constructor (Marco Ippolito) nodejs#51879 events,doc: * mark CustomEvent as stable (Daeyeon Jeong) nodejs#52618 fs: * add stacktrace to fs/promises (翠 / green) nodejs#49849 lib, url: * (SEMVER-MINOR) add a `windows` option to path parsing (Aviv Keller) nodejs#52509 net: * (SEMVER-MINOR) add CLI option for autoSelectFamilyAttemptTimeout (Paolo Insogna) nodejs#52474 report: * (SEMVER-MINOR) add `--report-exclude-network` option (Ethan Arrowood) nodejs#51645 src: * (SEMVER-MINOR) add `string_view` overload to snapshot FromBlob (Anna Henningsen) nodejs#52595 * (SEMVER-MINOR) add C++ ProcessEmitWarningSync() (Joyee Cheung) nodejs#51977 * (SEMVER-MINOR) add uv_get_available_memory to report and process (theanarkh) nodejs#52023 * (SEMVER-MINOR) preload function for Environment (Cheng Zhao) nodejs#51539 stream: * (SEMVER-MINOR) support typed arrays (IlyasShabi) nodejs#51866 test_runner: * (SEMVER-MINOR) add suite() (Colin Ihrig) nodejs#52127 * (SEMVER-MINOR) add `test:complete` event to reflect execution order (Moshe Atlow) nodejs#51909 util: * (SEMVER-MINOR) support array of formats in util.styleText (Marco Ippolito) nodejs#52040 v8: * (SEMVER-MINOR) implement v8.queryObjects() for memory leak regression testing (Joyee Cheung) nodejs#51927 watch: * mark as stable (Moshe Atlow) nodejs#52074 PR-URL: nodejs#52793

nodejs-github-bot added needs-ci PRs that need a full CI run. stream Issues and PRs related to the stream subsystem. labels Feb 24, 2024

IlyasShabi force-pushed the streams-typed-array branch from bd009a2 to 00931bd Compare February 25, 2024 08:35

mcollina reviewed Feb 25, 2024

View reviewed changes

lib/stream.js Outdated Show resolved Hide resolved

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 25, 2024

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 25, 2024

IlyasShabi force-pushed the streams-typed-array branch 2 times, most recently from 9a4bb10 to 16dddd2 Compare February 26, 2024 15:16

marco-ippolito added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 27, 2024

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 27, 2024

mcollina approved these changes Feb 27, 2024

View reviewed changes

mcollina requested review from ShogunPanda, ronag and rluvaton February 27, 2024 10:45

rluvaton reviewed Feb 27, 2024

View reviewed changes

lib/internal/streams/readable.js Outdated Show resolved Hide resolved

lib/internal/streams/writable.js Outdated Show resolved Hide resolved

IlyasShabi force-pushed the streams-typed-array branch 2 times, most recently from 76463b6 to 3fb394f Compare February 27, 2024 20:49

addaleax reviewed Feb 27, 2024

View reviewed changes

rluvaton approved these changes Feb 28, 2024

View reviewed changes

ShogunPanda approved these changes Feb 28, 2024

View reviewed changes

benjamingr requested changes Mar 1, 2024

View reviewed changes

IlyasShabi force-pushed the streams-typed-array branch from 3fb394f to bb7eaf6 Compare March 2, 2024 21:27

IlyasShabi force-pushed the streams-typed-array branch from bb7eaf6 to 0db88ac Compare March 2, 2024 21:40

benjamingr approved these changes Mar 9, 2024

View reviewed changes

mcollina added commit-queue Add this label to land a pull request using GitHub Actions. and removed commit-queue-failed An error occurred while landing this pull request using GitHub Actions. labels Mar 20, 2024

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Mar 20, 2024

nodejs-github-bot merged commit 7d258db into nodejs:main Mar 20, 2024
62 checks passed

mcollina added the notable-change PRs with changes that should be highlighted in changelogs. label Mar 21, 2024

mcollina added the baking-for-lts PRs that need to wait before landing in a LTS release. label Mar 21, 2024

mcollina added semver-minor PRs that contain new features and should be released in the next minor version. and removed needs-ci PRs that need a full CI run. labels Mar 21, 2024

RafaelGSS mentioned this pull request Apr 15, 2024

2024-04-24, Version 22.0.0 (Current) #52505

Merged

marco-ippolito mentioned this pull request May 2, 2024

v20.13.0 proposal #52793

Merged

marco-ippolito removed the baking-for-lts PRs that need to wait before landing in a LTS release. label Jun 17, 2024

IlyasShabi deleted the streams-typed-array branch October 16, 2024 18:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stream: support typed arrays #51866

stream: support typed arrays #51866

IlyasShabi commented Feb 24, 2024

nodejs-github-bot commented Feb 24, 2024

mcollina left a comment

nodejs-github-bot commented Feb 25, 2024

IlyasShabi commented Feb 26, 2024

nodejs-github-bot commented Feb 27, 2024

mcollina left a comment

rluvaton left a comment

addaleax Feb 27, 2024

ShogunPanda left a comment

benjamingr left a comment

IlyasShabi commented Mar 2, 2024

IlyasShabi commented Mar 2, 2024

nodejs-github-bot commented Mar 20, 2024

github-actions bot commented Mar 21, 2024

mcollina commented Mar 21, 2024

targos commented May 8, 2024

mcollina commented May 8, 2024

stream: support typed arrays #51866

stream: support typed arrays #51866

Conversation

IlyasShabi commented Feb 24, 2024

nodejs-github-bot commented Feb 24, 2024

mcollina left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Feb 25, 2024

IlyasShabi commented Feb 26, 2024

nodejs-github-bot commented Feb 27, 2024

mcollina left a comment

Choose a reason for hiding this comment

rluvaton left a comment

Choose a reason for hiding this comment

addaleax Feb 27, 2024

Choose a reason for hiding this comment

ShogunPanda left a comment

Choose a reason for hiding this comment

benjamingr left a comment

Choose a reason for hiding this comment

IlyasShabi commented Mar 2, 2024

IlyasShabi commented Mar 2, 2024

nodejs-github-bot commented Mar 20, 2024

github-actions bot commented Mar 21, 2024

mcollina commented Mar 21, 2024

targos commented May 8, 2024

mcollina commented May 8, 2024