Module records should be be discarded on link failure #1545
Comments
|
Hmm. I vaguely remember this being intentional, but I am not sure. Can you help me work through the consequences of leaving it in the map? I thought the idea was that if a module fails to instantiate, then we should show the resulting error once, and further attempts to use it (e.g. by importing anything from it) will fail, since it has no exports. I think that is what is currently specified, and it seems to work OK. You seem to be advocating for a different semantic, where we remove it from the map, and thus allow it to be re-fetched? That seems kind of dubious, since we won't get a chance to re-fetch unless someone later does dynamic module loading, and in the meantime we're just going to cause HostResolveImportedModule to fail repeatedly. And re-fetching seems unlikely to fix the problem. |
The trouble is that's not what ES specifies for the behaviour of ModuleDeclarationInstantiation. This does not handle being called again after it fails the first time and as specced will complete successfully the second time without actually doing anything, leaving the module environment partially initialised. I asked about this on es-discuss and the answer was that module loaders are expected to discard module records if there is an error linking. If we remove module scripts from the map then as you say we will fail in HostResolveImportedModule or attempt to re-fetch. The latter does seem undesirable and perhaps we should record the fact that we failed and generate an appropriate error if the same module requested again at a later time. |
|
Hmm, I see. I'm re-reading the es-discuss thread now. I guess we can try to to deal with this ourselves. I still can't see what the problems are with letting ModuleDeclarationInstantiation complete successfully the second time, but I guess you found some in fuzz testing? |
|
If you have any ideas on how you are going to implement this, or how it could be specced, I'd appreciate them. Right now it looks quite hard to do with the current spec setup. In particular there's no real easy way to find out which entries in the module map need to get nulled out. I guess the closest I have is looping through the tree of [[RequestedModules]], using "resolve a module specifier" to get to the module map key, and then nulling it out from there after all the relevant keys are collected? |
|
Not sure if this is what you meant, but the problem with letting I agree that this looks hard to do with the current spec. I'll think some more about this. On 13 July 2016 at 17:51, Domenic Denicola notifications@github.com wrote:
|
Why would you call this an unrelated error? This seems like the correct response to me: you try to import from a module which has previously failed to load, and you get a ReferenceError using that binding. |
|
It's not necessarily that binding though, it's whatever the first uninitialised binding is that happens to be accessed when the previously-failed module is executed. |
|
Ah, I think I see. Meanwhile I guess the es-discuss thread has generated some interesting ideas... |
|
@jonco3 do you have a good understanding of the conclusion of the es-discuss thread, and how to possibly integrate it into the spec's model? |
|
@domenic Yes, I'm working on this at the moment. |
|
OK, I think this is what we need to do: Add the following state to a module script:
At the end of internal module script tree fetching procedure, when fetching the descendants of a module script asynchronously completes add an additional step 'instantiate fetched module script tree' This procedure would do something like this:
Update HostResolveImportedModule to check the whether the module script it finds has 'instantiation error' set and if so re-throw it. Remove the call to ModuleDeclarationInstantiation from 'run a module script' and instead check whether 'instantiation error' is set and if so re-throw it. Oh, I guess this also means we need to track the dependencies of a module script which are not recorded at the moment. I think implementations would need to do that anyway even though it's not explicit in the spec at the moment. How does this sound to you? It's more complicated than I'd like but I don't see a way around that. |
|
This sounds pretty doable. Thanks so much for going to the trouble of figuring it out. A few minor questions. First, I guess I just want to make sure you think it's correct to move ModuleDeclarationInstantiation earlier. It really feels like something that should be done at "run a module script" time, instead of "fetch a module script tree" time. But I think you make it work, by storing any errors and re-throwing then during run-time. So I think it works.
Which error object? The one from "call ModuleDeclarationInstantiation for result module script"? That will be the one from the first failed dependency, right? This step means that if Also, why set the module record to null? We'd need to start adding guards to prevent executing null module records, right? But aren't those guards already covered by the guard against "instantiation error"?
Since the error can be itself
I think this is reachable via the module script's module record's [[RequestedModules]], each being passed to "resolve a module specifier" and then the result looked up in the module map. A bit indirect, but it works :). |
I think it's fine. The ES spec says it's OK to instantiate modules ahead of time as long as error reporting is deferred until TopLevelModuleEvaluationJob runs (not that we're actually doing TopLevelModuleEvaluationJob exactly but something hopefully equivalent).
Yes that one.
No, that shouldn't happen. b and c will be instantiated separately at the end of the respective calls to internal module script tree fetching procedure for those modules. The situation in multiple modules will get marked as bad is if there is a cyclic dependency. Then all modules in the cycle will be marked as bad, but this is correct.
According to ES we must discard the module record if instantiation fails so it seems prudent to remove any reference to it.
Oh I guess it can, that's annoying. In that case we probably need a tri-state as you suggested.
Yes, that sounds like it would work. |
Ah, I see, after working out an example in detail this makes sense. Perfect. I'll start speccing this. Thanks so much for your help!! |
As discussed in #1545 and in https://esdiscuss.org/topic/moduledeclarationinstantiation-behaviour-after-failure, the spec would potentially call ModuleDeclarationInstantiation on a module that previously failed to instantiate, with cascading bad consequences for the rest of the spec's algorithms. The JavaScript spec expects us to track these failures and avoid calling ModuleDeclarationInstantiation a second time. This commit adds such state tracking, partially by moving the ModuleDeclarationInstantiation calls to fetch time (with any errors stored for later rethrowing during evaluation time). Fixes #1545.
As discussed in #1545 and in https://esdiscuss.org/topic/moduledeclarationinstantiation-behaviour-after-failure, the spec would potentially call ModuleDeclarationInstantiation on a module that previously failed to instantiate, with cascading bad consequences for the rest of the spec's algorithms. The JavaScript spec expects us to track these failures and avoid calling ModuleDeclarationInstantiation a second time. This commit adds such state tracking, partially by moving the ModuleDeclarationInstantiation calls to fetch time (with any errors stored for later rethrowing during evaluation time). Fixes #1545.
As discussed in #1545 and in https://esdiscuss.org/topic/moduledeclarationinstantiation-behaviour-after-failure, the spec would potentially call ModuleDeclarationInstantiation on a module that previously failed to instantiate, with cascading bad consequences for the rest of the spec's algorithms. The JavaScript spec expects us to track these failures and avoid calling ModuleDeclarationInstantiation a second time. This commit adds such state tracking, partially by moving the ModuleDeclarationInstantiation calls to fetch time (with any errors stored for later rethrowing during evaluation time). Fixes #1545.
This addresses tc39#862 by ensuring that repeated calls to ModuleDeclarationInstantiation() and ModuleEvaluation(), for Source Text Module Records, rethrow any exceptions they previously threw, instead of silently succeeding. This allows host environments to do top-down instantiation/evaluation of module graphs, instead of having to do bottom-up instantiation/evaluation in order to record individual failures and thus prevent future instantiation/evaluation. In the process, this helps formalize some of the invariants previously stated in a vague way, such as "ModuleDeclarationInstantiation must have completed successfully", replacing them instead with an explicit [[Status]] field whose contents can be asserted against. For background on the trouble caused by the previous approach of silent success, see: - whatwg/html#1545 - tc39#862 - whatwg/html#2629
This addresses tc39#862 by ensuring that repeated calls to ModuleDeclarationInstantiation() and ModuleEvaluation(), for Source Text Module Records, rethrow any exceptions they previously threw, instead of silently succeeding. This allows host environments to do top-down instantiation/evaluation of module graphs, instead of having to do bottom-up instantiation/evaluation in order to record individual failures and thus prevent future instantiation/evaluation. In the process, this helps formalize some of the invariants previously stated in a vague way, such as "ModuleDeclarationInstantiation must have completed successfully", replacing them instead with an explicit [[Status]] field whose contents can be asserted against. For background on the trouble caused by the previous approach of silent success, see: - whatwg/html#1545 - tc39#862 - whatwg/html#2629
This builds on tc39/ecma262#916, in which the JavaScript specification remembers errors during module instantiation and evaluation. That allows us to stop our error-prone approach of doing bottom-up instantiation (which currently fails for cyclic graphs; see #2629), and obviates the consequent need to propagate errors (which is also buggy; see #2630). Finally, it makes certain edge cases during evaluation nicer; see #2595 (comment). For background on why we originally went with a bottom-up approach, see #1545. At the time we didn't seem to believe changing the JavaScript spec was an option, but it's become increasingly clear it's the most reasonable one. Closes #2629. Closes #2630.
This builds on tc39/ecma262#916, in which the JavaScript specification remembers errors during module instantiation and evaluation. That allows us to stop our error-prone approach of doing bottom-up instantiation (which currently fails for cyclic graphs; see #2629), and obviates the consequent need to propagate errors (which is also buggy; see #2630). Finally, it makes certain edge cases during evaluation nicer; see #2595 (comment). For background on why we originally went with a bottom-up approach, see #1545. At the time we didn't seem to believe changing the JavaScript spec was an option, but it's become increasingly clear it's the most reasonable one. Closes #2629. Closes #2630.
This builds on tc39/ecma262#916, in which the JavaScript specification remembers errors during module instantiation and evaluation. That allows us to stop our error-prone approach of doing bottom-up instantiation (which currently fails for cyclic graphs; see #2629), and obviates the consequent need to propagate errors (which is also buggy; see #2630). Finally, it makes certain edge cases during evaluation nicer; see #2595 (comment). For background on why we originally went with a bottom-up approach, see #1545. At the time we didn't seem to believe changing the JavaScript spec was an option, but it's become increasingly clear it's the most reasonable one. Closes #2629. Closes #2630.
This closes #862 by ensuring that repeated calls to ModuleDeclarationInstantiation() and ModuleEvaluation(), for Source Text Module Records, rethrow any exceptions they previously threw, instead of silently succeeding. This clarifies how host environments can do top-down instantiation/evaluation of module graphs, instead of having to do bottom-up instantiation/evaluation in order to record individual failures and thus prevent future instantiation/evaluation. For background on the trouble caused by the previous approach of silent success, see: - whatwg/html#1545 - #862 - whatwg/html#2629 - whatwg/html#2595 (comment) In the process, this helps formalize some of the invariants previously stated in a vague way, such as "ModuleDeclarationInstantiation must have completed successfully", replacing them instead with an explicit [[Status]] field whose contents can be asserted against. It also clarifies the relationship of Module Records that are not Source Text Module Records with respect to these invariants; in brief, they are expected to be leaves in the graph, with no descendants. Finally, it also updates the machinery involved in module instantiation and evaluation, first by renaming the ModuleDeclarationInstantion() and ModuleEvaluation() abstract methods to Instantiate() and Evaluate(), and also by documenting all of the abstract operations and methods involved. This includes non-normative prose containing example Source Text Module Record graphs and how they are processed. Closes #916.
This builds on tc39/ecma262#916, in which the JavaScript specification remembers errors during module instantiation and evaluation. That allows us to stop our error-prone approach of doing bottom-up instantiation (which currently fails for cyclic graphs; see #2629), and obviates the consequent need to propagate errors (which is also buggy; see #2630). Finally, it makes certain edge cases during evaluation nicer; see #2595 (comment). For background on why we originally went with a bottom-up approach, see #1545. At the time we didn't seem to believe changing the JavaScript spec was an option, but it's become increasingly clear it's the most reasonable one. Closes #2629. Closes #2630.
This builds on tc39/ecma262#916, in which the JavaScript specification remembers errors during module instantiation and evaluation. That allows us to stop our error-prone approach of doing bottom-up instantiation (which currently fails for cyclic graphs; see #2629), and obviates the consequent need to propagate errors (which is also buggy; see #2630). Finally, it makes certain edge cases during evaluation nicer; see #2595 (comment). For background on why we originally went with a bottom-up approach, see #1545. At the time we didn't seem to believe changing the JavaScript spec was an option, but since then we've learned that doing so is the most reasonable way to go. There may be more work to do here in certain error-related edge cases, or to make potential optimizations more obvious. But now at least the setup is implementable and reasonable. Closes #2629. Closes #2630.
As discussed in whatwg#1545 and in https://esdiscuss.org/topic/moduledeclarationinstantiation-behaviour-after-failure, the spec would potentially call ModuleDeclarationInstantiation on a module that previously failed to instantiate, with cascading bad consequences for the rest of the spec's algorithms. The JavaScript spec expects us to track these failures and avoid calling ModuleDeclarationInstantiation a second time. This commit adds such state tracking, partially by moving the ModuleDeclarationInstantiation calls to fetch time (with any errors stored for later rethrowing during evaluation time). Fixes whatwg#1545.
This builds on tc39/ecma262#916, in which the JavaScript specification remembers errors during module instantiation and evaluation. That allows us to stop our error-prone approach of doing bottom-up instantiation (which currently fails for cyclic graphs; see whatwg#2629), and obviates the consequent need to propagate errors (which is also buggy; see whatwg#2630). Finally, it makes certain edge cases during evaluation nicer; see whatwg#2595 (comment). For background on why we originally went with a bottom-up approach, see whatwg#1545. At the time we didn't seem to believe changing the JavaScript spec was an option, but since then we've learned that doing so is the most reasonable way to go. There may be more work to do here in certain error-related edge cases, or to make potential optimizations more obvious. But now at least the setup is implementable and reasonable. Closes whatwg#2629. Closes whatwg#2630.
In https://html.spec.whatwg.org/#calling-scripts the spec currently says:
If the ModuleDeclarationInstantiation fails for a record (or for one of its descendants because this happens recursively) then that module script will be left in the module map and can be used to satisfy subsequent requests to fetch a module script. ModuleDeclarationInstantiation will not attempt to re-instantiate a module record if called again for the same record, and incorrect execution will follow.
It seems the expectation is that the entire module tree will be discarded if ModuleDeclarationInstantation fails.
The text was updated successfully, but these errors were encountered: