worker: keep allocators for transferred SAB instances alive longer #29637
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Keep the `ArrayBuffer::Allocator` behind a `SharedArrayBuffer` instance alive for at least as long as the receiving Isolate lives, if the `SharedArrayBuffer` instance isn’t already destroyed through GC. This is to work around the fact that V8 7.9 started refactoring how backing stores for `SharedArrayBuffer` instances work, changing the timing of the call that releases the backing store to be during Isolate disposal. The flag added to the test is optional but helps verify that the backing store is actually free’d at the end of the test and does not leak memory. Fixes: nodejs/node-v8#115
nodejs-github-bot
added
the
c++
addaleax
added
dont-land-on-v10.x
v8 engine
Trott
approved these changes
Sep 24, 2019
Trott
added
the
author ready
Trott
reviewed
Sep 24, 2019
| // | ||
| // This is a workaround for https://github.com/nodejs/node-v8/issues/115 | ||
| // (introduced in V8 7.9) and we should be able to remove it once V8 | ||
| // ArrayBuffer::Allocator refactoring/removal is complete. |
There was a problem hiding this comment.
Shout out to long and helpful comments!
targos
approved these changes
Sep 24, 2019
bnoordhuis
approved these changes
Sep 24, 2019
| @@ -1036,6 +1036,21 @@ char* Environment::Reallocate(char* data, size_t old_size, size_t size) { | |||
| return new_data; | |||
| } | |||
|
|
|||
| void Environment::AddArrayBufferAllocatorToKeepAliveUntilIsolateDispose( | |||
There was a problem hiding this comment.
Suggested change
| void Environment::AddArrayBufferAllocatorToKeepAliveUntilIsolateDispose( | |
| void Environment::ThisMethodNameIsSoLongItMakesMyEyeballsBleedIDontEven( |
There was a problem hiding this comment.
I mean, I’m happy to take a more concise suggestion, but given the very specific use case I’m fine with a long name.
|
Landed in 7899a96 |
Trott
pushed a commit
that referenced
this pull request
Sep 25, 2019
Keep the `ArrayBuffer::Allocator` behind a `SharedArrayBuffer` instance alive for at least as long as the receiving Isolate lives, if the `SharedArrayBuffer` instance isn’t already destroyed through GC. This is to work around the fact that V8 7.9 started refactoring how backing stores for `SharedArrayBuffer` instances work, changing the timing of the call that releases the backing store to be during Isolate disposal. The flag added to the test is optional but helps verify that the backing store is actually free’d at the end of the test and does not leak memory. Fixes: nodejs/node-v8#115 PR-URL: #29637 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
targos
pushed a commit
that referenced
this pull request
Oct 1, 2019
Keep the `ArrayBuffer::Allocator` behind a `SharedArrayBuffer` instance alive for at least as long as the receiving Isolate lives, if the `SharedArrayBuffer` instance isn’t already destroyed through GC. This is to work around the fact that V8 7.9 started refactoring how backing stores for `SharedArrayBuffer` instances work, changing the timing of the call that releases the backing store to be during Isolate disposal. The flag added to the test is optional but helps verify that the backing store is actually free’d at the end of the test and does not leak memory. Fixes: nodejs/node-v8#115 PR-URL: #29637 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
pull bot
pushed a commit
to ksti/v8
that referenced
this pull request
Oct 24, 2019
Add an `array_buffer_allocator_shared` field to the `Isolate::CreateParams` struct that allows embedders to share ownership of the ArrayBuffer::Allocator with V8, and which in particular means that when this method is used that the BackingStore deleter will not perform an use-after-free access to the Allocator under certain circumstances. For Background: tl;dr: This is necessary for Node.js to perform the transition to V8 7.9, because of the way that ArrayBuffer::Allocators and their lifetimes currently work there. In Node.js, each Worker thread has its own ArrayBuffer::Allocator. Changing that would currently be impractical, as each allocator depends on per-Isolate state. However, now that backing stores are managed globally and keep a pointer to the original ArrayBuffer::Allocator, this means that when transferring an ArrayBuffer (e.g. from one Worker to another through postMessage()), the original Allocator has to be kept alive until the ArrayBuffer no longer exists in the receiving Isolate (or until that Isolate is disposed). See [1] for an example Node.js test that fails with V8 7.9. This problem also existed for SharedArrayBuffers, where Node.js was broken by V8 earlier for the same reasons (see [2] for the bug report on that and [3] for the resolution in Node.js). For SharedArrayBuffers, we already had extensive tracking logic, so adding a shared_ptr to keep alive the ArrayBuffer::Allocator was not a significant amount of work. However, the mechanism for transferring non-shared ArrayBuffers is quite different, and it seems both easier for us and better for V8 from an API standpoint to keep the Allocator alive from where it is being referenced. By sharing memory with the custom deleter function/data pair, this comes at no memory overhead. [1]: nodejs/node#30044 [2]: nodejs/node-v8#115 [3]: nodejs/node#29637 Bug: v8:9380 Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347 Commit-Queue: Ulan Degenbaev <ulan@chromium.org> Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Reviewed-by: Igor Sheludko <ishell@chromium.org> Cr-Commit-Position: refs/heads/master@{#64542}
targos
added a commit
to targos/node
that referenced
this pull request
Oct 24, 2019
Original commit message:
[api] Add possibility for BackingStore to keep Allocator alive
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.
For Background:
tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.
In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.
This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.
By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.
[1]: nodejs#30044
[2]: nodejs/node-v8#115
[3]: nodejs#29637
Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64542}
Refs: v8/v8@6b0a953
targos
added a commit
to targos/node
that referenced
this pull request
Oct 24, 2019
Original commit message:
[api] Add possibility for BackingStore to keep Allocator alive
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.
For Background:
tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.
In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.
This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.
By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.
[1]: nodejs#30044
[2]: nodejs/node-v8#115
[3]: nodejs#29637
Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64542}
Refs: v8/v8@6b0a953
targos
added a commit
to targos/node
that referenced
this pull request
Oct 28, 2019
Original commit message:
[api] Add possibility for BackingStore to keep Allocator alive
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.
For Background:
tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.
In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.
This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.
By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.
[1]: nodejs#30044
[2]: nodejs/node-v8#115
[3]: nodejs#29637
Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64542}
Refs: v8/v8@6b0a953
targos
added a commit
to targos/node
that referenced
this pull request
Nov 1, 2019
Original commit message:
[api] Add possibility for BackingStore to keep Allocator alive
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.
For Background:
tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.
In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.
This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.
By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.
[1]: nodejs#30044
[2]: nodejs/node-v8#115
[3]: nodejs#29637
Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64542}
Refs: v8/v8@6b0a953
targos
added a commit
to targos/node
that referenced
this pull request
Nov 8, 2019
Original commit message:
[api] Add possibility for BackingStore to keep Allocator alive
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.
For Background:
tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.
In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.
This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.
By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.
[1]: nodejs#30044
[2]: nodejs/node-v8#115
[3]: nodejs#29637
Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64542}
Refs: v8/v8@6b0a953
PR-URL: nodejs#30020
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos
added a commit
to targos/node
that referenced
this pull request
Nov 17, 2019
Original commit message:
[api] Add possibility for BackingStore to keep Allocator alive
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.
For Background:
tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.
In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.
This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.
By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.
[1]: nodejs#30044
[2]: nodejs/node-v8#115
[3]: nodejs#29637
Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64542}
Refs: v8/v8@6b0a953
PR-URL: nodejs#30020
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
MylesBorins
pushed a commit
that referenced
this pull request
Nov 21, 2019
Original commit message:
[api] Add possibility for BackingStore to keep Allocator alive
Add an `array_buffer_allocator_shared` field to the
`Isolate::CreateParams` struct that allows embedders to share
ownership of the ArrayBuffer::Allocator with V8, and which in
particular means that when this method is used that the
BackingStore deleter will not perform an use-after-free access to the
Allocator under certain circumstances.
For Background:
tl;dr: This is necessary for Node.js to perform the transition to
V8 7.9, because of the way that ArrayBuffer::Allocators and their
lifetimes currently work there.
In Node.js, each Worker thread has its own ArrayBuffer::Allocator.
Changing that would currently be impractical, as each allocator
depends on per-Isolate state. However, now that backing stores
are managed globally and keep a pointer to the original
ArrayBuffer::Allocator, this means that when transferring an
ArrayBuffer (e.g. from one Worker to another through postMessage()),
the original Allocator has to be kept alive until the ArrayBuffer
no longer exists in the receiving Isolate (or until that Isolate
is disposed). See [1] for an example Node.js test that fails with
V8 7.9.
This problem also existed for SharedArrayBuffers, where Node.js
was broken by V8 earlier for the same reasons (see [2] for the bug
report on that and [3] for the resolution in Node.js).
For SharedArrayBuffers, we already had extensive tracking logic,
so adding a shared_ptr to keep alive the ArrayBuffer::Allocator
was not a significant amount of work. However, the mechanism for
transferring non-shared ArrayBuffers is quite different, and
it seems both easier for us and better for V8 from an API standpoint
to keep the Allocator alive from where it is being referenced.
By sharing memory with the custom deleter function/data pair,
this comes at no memory overhead.
[1]: #30044
[2]: nodejs/node-v8#115
[3]: #29637
Bug: v8:9380
Change-Id: Ibc2c4fb6341b53653cbd637bd8cb3d4ac43809c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1874347
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64542}
Refs: v8/v8@6b0a953
Backport-PR-URL: #30513
PR-URL: #30020
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Keep the
ArrayBuffer::Allocatorbehind aSharedArrayBufferinstancealive for at least as long as the receiving Isolate lives, if the
SharedArrayBufferinstance isn’t already destroyed through GC.This is to work around the fact that V8 7.9 started refactoring
how backing stores for
SharedArrayBufferinstances work, changingthe timing of the call that releases the backing store to be
during Isolate disposal.
The flag added to the test is optional but helps verify that the
backing store is actually free’d at the end of the test and does not
leak memory.
Fixes: nodejs/node-v8#115
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passes