Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

stream: improve writable.write() performance #31624

Merged
merged 2 commits into from
Feb 9, 2020
Merged

stream: improve writable.write() performance #31624

merged 2 commits into from
Feb 9, 2020

Conversation

mscdex
Copy link
Contributor

@mscdex mscdex commented Feb 3, 2020 •
edited
Loading

Benchmark results:

                                                                                   confidence improvement accuracy (*)   (**)  (***)
 streams/writable-manywrites.js callback='no' writev='no' sync='no' n=40000000                    0.72 %       ±1.42% ±1.89% ±2.46%
 streams/writable-manywrites.js callback='no' writev='no' sync='yes' n=40000000          ***      2.82 %       ±0.54% ±0.72% ±0.94%
 streams/writable-manywrites.js callback='no' writev='yes' sync='no' n=40000000                   1.04 %       ±1.11% ±1.47% ±1.92%
 streams/writable-manywrites.js callback='no' writev='yes' sync='yes' n=40000000         ***      3.18 %       ±0.69% ±0.92% ±1.20%
 streams/writable-manywrites.js callback='yes' writev='no' sync='no' n=40000000                  -0.39 %       ±1.56% ±2.09% ±2.73%
 streams/writable-manywrites.js callback='yes' writev='no' sync='yes' n=40000000         ***      7.03 %       ±0.83% ±1.11% ±1.45%
 streams/writable-manywrites.js callback='yes' writev='yes' sync='no' n=40000000         ***      2.28 %       ±1.06% ±1.41% ±1.83%
 streams/writable-manywrites.js callback='yes' writev='yes' sync='yes' n=40000000        ***      8.46 %       ±0.89% ±1.18% ±1.55%

There is also a change to the benchmark runner to ensure that end() is not called multiple times. This was happening in the Writable benchmark and was at least causing weird display issues (e.g. improper fraction for # of configs, completion percentage bouncing back and forth, etc.) when using benchmark/compare.js and showing progress.

The reason why that was happening was that process.send() is used to send the result, which is async and gives individual benchmarks time to call .end() again before the process actually exits.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

@mscdex mscdex added stream Issues and PRs related to the stream subsystem. benchmark Issues and PRs related to the benchmark subsystem. performance Issues and PRs related to the performance of Node.js. labels Feb 3, 2020
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/28901/

@jasnell jasnell requested a review from mcollina February 4, 2020 20:14
mcollina
mcollina approved these changes Feb 5, 2020
View reviewed changes
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@addaleax addaleax added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Feb 7, 2020
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/28977/

@mscdex mscdex force-pushed the stream-writable-write-perf branch 2 times, most recently from 9584701 to dea2b50 Compare February 9, 2020 02:35
mscdex added 2 commits February 8, 2020 21:36
@mscdex
benchmark: check for and fix multiple end()
b862a0c 
PR-URL: nodejs#31624
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
@mscdex
stream: improve writable.write() performance
43783b5 
PR-URL: nodejs#31624
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
@mscdex mscdex force-pushed the stream-writable-write-perf branch from 568dcf0 to 43783b5 Compare February 9, 2020 02:37
@mscdex mscdex merged commit 43783b5 into nodejs:master Feb 9, 2020
@mscdex mscdex deleted the stream-writable-write-perf branch February 9, 2020 02:38
codebytere pushed a commit that referenced this pull request Feb 17, 2020
@mscdex @codebytere
benchmark: check for and fix multiple end()
161db60 
PR-URL: #31624
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
@codebytere codebytere mentioned this pull request Feb 17, 2020
Merged
@codebytere
Copy link
Member

@mscdex this needs a manual backport to v13.x - if it shouldn't be feel free to just swap that label out for dont-land!

@MylesBorins MylesBorins mentioned this pull request Mar 9, 2020
Closed
7 tasks
@ronag ronag mentioned this pull request Mar 9, 2020
Closed
4 tasks
@ronag ronag mentioned this pull request Mar 10, 2020
Closed
4 tasks
ronag pushed a commit to nxtedition/node that referenced this pull request Mar 10, 2020
@mscdex @ronag
stream: improve writable.write() performance
1d88182 
PR-URL: nodejs#31624
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Backport-PR-URL: nodejs#32169
MylesBorins pushed a commit that referenced this pull request Mar 10, 2020
@mscdex @MylesBorins
stream: improve writable.write() performance
1e05ddf 
Backport-PR-URL: #32169
PR-URL: #31624
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
@MylesBorins MylesBorins mentioned this pull request Mar 10, 2020
Merged
codebytere pushed a commit that referenced this pull request Mar 15, 2020
@mscdex @codebytere
benchmark: check for and fix multiple end()
00f8569 
PR-URL: #31624
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
codebytere pushed a commit that referenced this pull request Mar 17, 2020
@mscdex @codebytere
benchmark: check for and fix multiple end()
c21fb89 
PR-URL: #31624
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
@codebytere codebytere mentioned this pull request Mar 17, 2020
Merged
codebytere pushed a commit that referenced this pull request Mar 30, 2020
@mscdex @codebytere
benchmark: check for and fix multiple end()
a305ae2 
PR-URL: #31624
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lpinca lpinca lpinca approved these changes

@jasnell jasnell jasnell approved these changes

@mcollina mcollina mcollina approved these changes

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. benchmark Issues and PRs related to the benchmark subsystem. performance Issues and PRs related to the performance of Node.js. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@mscdex @nodejs-github-bot @codebytere @mcollina @jasnell @lpinca @MylesBorins @addaleax @targos