Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer: remove MAX_SAFE_INTEGER check on length #14131

Closed
wants to merge 1 commit into from

Conversation

Trott
Copy link
Member

@Trott Trott commented Jul 8, 2017

MAX_SAFE_INTEGER is millions of times larger than the largest buffer
allowed in Node.js. There is no need to squash the length down to
MAX_SAFE_INTEGER. Removing that check results in a small but
statistically significant increase for Buffer.from() operating on
ArrayBuffers in some situations.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines
Affected core subsystem(s)

buffer

MAX_SAFE_INTEGER is millions of times larger than the largest buffer
allowed in Node.js. There is no need to squash the length down to
MAX_SAFE_INTEGER. Removing that check results in a small but
statistically significant increase for Buffer.from() operating on
ArrayBuffers in some situations.
@Trott Trott added the buffer Issues and PRs related to the buffer subsystem. label Jul 8, 2017
@Trott
Copy link
Member Author

Trott commented Jul 8, 2017

                                                                    improvement confidence     p.value
 buffers/buffer-from.js n=2048 len=10 source="arraybuffer-middle"        1.54 %         ** 0.001666215
 buffers/buffer-from.js n=2048 len=2048 source="arraybuffer-middle"      1.76 %         ** 0.003719416

@Trott
Copy link
Member Author

Trott commented Jul 8, 2017

If you do somehow manage to get Number.MAX_SAFE_INTEGER in there, the new FastBuffer() at the end of the function results in:

RangeError: Invalid typed array length: 9007199254740991

Before, if you manage to go here, say with Number.MAX_VALUE, you'd still get the error above because it got squashed down to Number.MAX_SAFE_INTEGER. Now you get:

RangeError: Invalid typed array length: 1.7976931348623157e+308

This seems more correct to me. You're getting the actual length that is problematic rather than some other length that you didn't specify.

@Trott
Copy link
Member Author

Trott commented Jul 8, 2017

@mscdex
Copy link
Contributor

mscdex commented Jul 8, 2017

Is this technically semver-major due to the possible error change for large values?

@bnoordhuis
Copy link
Member

Is this technically semver-major due to the possible error change for large values?

No, IMO.

@Trott
Copy link
Member Author

Trott commented Jul 8, 2017

The errors in buffer module is migrating to use internal/errors, so this PR may have conflicts with #13976 , which has a new code ERR_BUFFER_OUT_OF_BOUNDS for the RangerError here.

This PR doesn't change the RangeError code. It deletes two lines untouched by #13976.

@refack
Copy link
Contributor

refack commented Jul 8, 2017

Is this technically semver-major due to the possible error change for large values?

Depends on resolution of #13937

@refack
Copy link
Contributor

refack commented Jul 8, 2017

Removing that check results in a small but
statistically significant increase for Buffer.from() operating on
ArrayBuffers in some situations.

A ~1.5% of the time was used for that if... Very unintuitive. Makes me put the meaning of our micro-benchmarks into new proportions, they seem to be very node core specific with limited real world correlation.
Ref: nodejs/CTC#146

@Trott
Copy link
Member Author

Trott commented Jul 11, 2017

Is this technically semver-major due to the possible error change for large values?
No, IMO.

I agree. If the wording changed, then yes. But since it's just correcting the (IMO) wrong value repeated back to the user, I would say no.

Depends on resolution of #13937

I don't think so because that's about changes after we move to the new internal error system. This one is not part of the new internal error system.

@Trott
Copy link
Member Author

Trott commented Jul 12, 2017

Landed in e6e6b07

@Trott Trott closed this Jul 12, 2017
Trott added a commit that referenced this pull request Jul 12, 2017
MAX_SAFE_INTEGER is millions of times larger than the largest buffer
allowed in Node.js. There is no need to squash the length down to
MAX_SAFE_INTEGER. Removing that check results in a small but
statistically significant increase for Buffer.from() operating on
ArrayBuffers in some situations.

PR-URL: #14131
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
addaleax pushed a commit that referenced this pull request Jul 12, 2017
MAX_SAFE_INTEGER is millions of times larger than the largest buffer
allowed in Node.js. There is no need to squash the length down to
MAX_SAFE_INTEGER. Removing that check results in a small but
statistically significant increase for Buffer.from() operating on
ArrayBuffers in some situations.

PR-URL: #14131
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
@addaleax addaleax mentioned this pull request Jul 18, 2017
addaleax pushed a commit that referenced this pull request Jul 18, 2017
MAX_SAFE_INTEGER is millions of times larger than the largest buffer
allowed in Node.js. There is no need to squash the length down to
MAX_SAFE_INTEGER. Removing that check results in a small but
statistically significant increase for Buffer.from() operating on
ArrayBuffers in some situations.

PR-URL: #14131
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Fishrock123 pushed a commit that referenced this pull request Jul 19, 2017
MAX_SAFE_INTEGER is millions of times larger than the largest buffer
allowed in Node.js. There is no need to squash the length down to
MAX_SAFE_INTEGER. Removing that check results in a small but
statistically significant increase for Buffer.from() operating on
ArrayBuffers in some situations.

PR-URL: #14131
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
@MylesBorins
Copy link
Contributor

Should this be backported to v6.x-staging? If yes please follow the guide and raise a backport PR, if no let me know or add the dont-land-on label.

@Trott
Copy link
Member Author

Trott commented Aug 16, 2017

@MylesBorins Did you mean to label this either backport-requested or dont-land-on instead of backported?

@Trott
Copy link
Member Author

Trott commented Aug 16, 2017

(Changed to backport requested..)

@Trott
Copy link
Member Author

Trott commented Aug 16, 2017

@MylesBorins If #12361 lands in v6.x-staging, then this should land and will land cleanly. If #12361 does not land in v6.x-staging, then neither should this. #12361 is currently marked for lts-watch. I'm going to remove the backport-requested and add an lts-watch label for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
buffer Issues and PRs related to the buffer subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.