Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v14.x backport] fs: improve fsPromises readFile performance #37703

Closed

Conversation

Linkgoron
Copy link
Member

@Linkgoron Linkgoron commented Mar 10, 2021

Improve the fsPromises readFile performance
by allocating only one buffer, when size is known,
increase the size of the readbuffer chunks,
and dont read more data if size bytes have been read

refs: #37583

PR-URL: #37608

I only cherry-picked the changes to fs/promises.js (first commit) but didn't backport the benchmark (second commit). If that's needed, I'll be happy to do it as well. This PR also incorporates the changes of #37127, so I'm not sure if that PR should be backported before this one.

@nodejs-github-bot nodejs-github-bot added fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. v14.x labels Mar 10, 2021
@Linkgoron
Copy link
Member Author

Benchmark numbers:

                                                               confidence improvement accuracy (*)   (**)  (***)
 fs/readfile-promises.js concurrent=1 len=1024 duration=5             ***     24.56 %       ±0.52% ±0.69% ±0.90%
 fs/readfile-promises.js concurrent=1 len=16777216 duration=5         ***    353.44 %       ±4.88% ±6.56% ±8.68%
 fs/readfile-promises.js concurrent=10 len=1024 duration=5            ***     30.64 %       ±0.58% ±0.78% ±1.02%
 fs/readfile-promises.js concurrent=10 len=16777216 duration=5        ***    573.13 %       ±4.38% ±5.89% ±7.78%

Linkgoron added a commit to Linkgoron/node that referenced this pull request Mar 11, 2021
Improve the fsPromises readFile performance
by allocating only one buffer, when size is known,
increase the size of the readbuffer chunks,
and dont read more data if size bytes have been read

refs: nodejs#37583
Backport-PR-URL: nodejs#37703
PR-URL: nodejs#37608
@Linkgoron
Copy link
Member Author

Linkgoron commented Mar 11, 2021

Noticed that this needs to wait for two weeks.

@nodejs-github-bot
Copy link
Collaborator

Improve the fsPromises readFile performance
by allocating only one buffer, when size is known,
increase the size of the readbuffer chunks,
and dont read more data if size bytes have been read

refs: nodejs#37583
Backport-PR-URL: nodejs#37703
PR-URL: nodejs#37608
@Linkgoron
Copy link
Member Author

Linkgoron commented Apr 24, 2021

@targos given the AbortController PR, this (the original PR) might land cleanly without any modifications after #35911 is backported.

@targos
Copy link
Member

targos commented Apr 25, 2021

This PR is actually what started my quest of backporting AbortController 😄

@Linkgoron Linkgoron closed this May 3, 2021
MoritzLoewenstein added a commit to MoritzLoewenstein/node that referenced this pull request Aug 21, 2021
Improve the fsPromises readFile performance
by allocating only one buffer, when size is known,
increase the size of the readbuffer chunks,
and dont read more data if size bytes have been read.
Also moves constants to internal/fs/utils.

refs: nodejs#37583
(Old) Backport-PR-URL: nodejs#37703
PR-URL: nodejs#37608
MoritzLoewenstein added a commit to MoritzLoewenstein/node that referenced this pull request Aug 21, 2021
Improve the fsPromises readFile performance
by allocating only one buffer, when size is known,
increase the size of the readbuffer chunks,
and dont read more data if size bytes have been read.
Also moves constants to internal/fs/utils.

refs: nodejs#37583
(Old) Backport-PR-URL: nodejs#37703
PR-URL: nodejs#37608
@targos targos removed their assignment Oct 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants