-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
loader: use default loader as cascaded loader in the in loader worker #47620
Conversation
Use the default loader as the cascaded loader in the loader worker. Otherwise we spawn loader workers in the loader workers indefinitely.
Review requested:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree it's a better fix than my PR, I would prefer if we used my test, but other than that LGTM. Thanks!
I’d like to add a few more tests, from the other PR and the simpler cases we discussed that could also trigger the issue, but then this LGTM. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems a nicely simple fix—thanks for putting it together!
Before landing I would like there to be a few more code comments and, if possible, leverage an existing, persistent fixture (I believe the repro of the issue this addresses does not require dynamically created files, so let's not over-complicate things).
@@ -165,4 +173,6 @@ module.exports = { | |||
initializeHooks, | |||
getDefaultConditions, | |||
getConditionsSet, | |||
loaderWorkerId: 'internal/modules/esm/worker', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: id
seems misleading to me, like it's a PID or similar, rather than a path.
loaderWorkerId: 'internal/modules/esm/worker', | |
loaderWorkerSpecifier: 'internal/modules/esm/worker', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's definitely not a path (because the internal module loader doesn't use paths), I think calling it an ID is more correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
O.o it is literally the path to the file (but with CJS's whacked exclusion of the file extension) and is even consumed as such:
node/lib/internal/modules/esm/hooks.js
Line 498 in 3707d00
this.#worker = new InternalWorker(loaderWorkerId, { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not the path to a file, because internal modules are not in the FS, it's more correct to think of them as an offset in the node
binary. InternalWorker
constructor doesn't take a path, it only accepts internal module IDs – you can try to replace loaderWorkerId
with a path at the line you linked to to convince yourself if you want.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it’s whatever we call the string that’s the input to internalBinding()
. The signature of that function is internalBinding(module)
, so I guess we could use loaderWorkerModule
. Not sure if that’s any better than loaderWorkerId
. It’s not really a specifier because we don’t use this with import
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
loaderWorkerModuleId
?
@@ -165,4 +173,6 @@ module.exports = { | |||
initializeHooks, | |||
getDefaultConditions, | |||
getConditionsSet, | |||
loaderWorkerId: 'internal/modules/esm/worker', | |||
isLoaderWorker, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: just make this a getter?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would assume a function call would be slower than just mapping directly to a primitive?
This isn’t author ready, there are many open questions and requests for more tests. If I have time I’ll add the tests, but this shouldn’t land as is. I think we agree with the general approach and have only nits for the changes themselves, but I at least have objections to the tests. |
I suggest we land ASAP (as is) and address the concerns regarding tests and such in a follow up PR, the current flakiness of the CI si very bad for everyone, and there are no blocking comments. |
I’ve added the tests and implemented the code suggestions that had clear resolutions. I left alone possibly renaming |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tests lgtm
Landed in f14d2e5 |
Use the default loader as the cascaded loader in the loader worker. Otherwise we spawn loader workers in the loader workers indefinitely. PR-URL: #47620 Fixes: #47566 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>
Use the default loader as the cascaded loader in the loader worker. Otherwise we spawn loader workers in the loader workers indefinitely. PR-URL: nodejs#47620 Fixes: nodejs#47566 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>
Use the default loader as the cascaded loader in the loader worker. Otherwise we spawn loader workers in the loader workers indefinitely. PR-URL: #47620 Backport-PR-URL: #50669 Fixes: #47566 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>
Use the default loader as the cascaded loader in the loader worker. Otherwise we spawn loader workers in the loader workers indefinitely. PR-URL: nodejs/node#47620 Backport-PR-URL: nodejs/node#50669 Fixes: nodejs/node#47566 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>
Use the default loader as the cascaded loader in the loader worker. Otherwise we spawn loader workers in the loader workers indefinitely. PR-URL: nodejs/node#47620 Backport-PR-URL: nodejs/node#50669 Fixes: nodejs/node#47566 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>
Use the default loader as the cascaded loader in the loader worker. Otherwise we spawn loader workers in the loader workers indefinitely.
Fixes: #47566