vm: add modules

Adds vm.Module, which wraps around ModuleWrap to provide an interface
for developers to work with modules in a more reflective manner.

Co-authored-by: Timothy Gu <timothygu99@gmail.com>
PR-URL: #17560
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>

doc/api/errors.md

@@ -1639,6 +1639,43 @@ entry types were found.
 
 Superseded by `ERR_OUT_OF_RANGE`
 
+<a id="ERR_VM_MODULE_ALREADY_LINKED"></a>
+### ERR_VM_MODULE_ALREADY_LINKED
+
+The module attempted to be linked is not eligible for linking, because of one of
+the following reasons:
+
+- It has already been linked (`linkingStatus` is `'linked'`)
+- It is being linked (`linkingStatus` is `'linking'`)
+- Linking has failed for this module (`linkingStatus` is `'errored'`)
+
+<a id="ERR_VM_MODULE_DIFFERENT_CONTEXT"></a>
+### ERR_VM_MODULE_DIFFERENT_CONTEXT
+
+The module being returned from the linker function is from a different context
+than the parent module. Linked modules must share the same context.
+
+<a id="ERR_VM_MODULE_LINKING_ERRORED"></a>
+### ERR_VM_MODULE_LINKING_ERRORED
+
+The linker function returned a module for which linking has failed.
+
+<a id="ERR_VM_MODULE_NOT_LINKED"></a>
+### ERR_VM_MODULE_NOT_LINKED
+
+The module must be successfully linked before instantiation.
+
+<a id="ERR_VM_MODULE_NOT_MODULE"></a>
+### ERR_VM_MODULE_NOT_MODULE
+
+The fulfilled value of a linking promise is not a `vm.Module` object.
+
+<a id="ERR_VM_MODULE_STATUS"></a>
+### ERR_VM_MODULE_STATUS
+
+The current module's status does not allow for this operation. The specific
+meaning of the error depends on the specific function.
+
 <a id="ERR_ZLIB_BINDING_CLOSED"></a>
 ### ERR_ZLIB_BINDING_CLOSED
 

doc/api/vm.md

@@ -43,6 +43,322 @@ console.log(x); // 1; y is not defined.
 *Note*: The vm module is not a security mechanism.
 **Do not use it to run untrusted code**.
 
+## Class: vm.Module
+<!-- YAML
+added: REPLACEME
+-->
+
+> Stability: 1 - Experimental
+
+*This feature is only available with the `--experimental-vm-modules` command
+flag enabled.*
+
+The `vm.Module` class provides a low-level interface for using ECMAScript
+modules in VM contexts. It is the counterpart of the `vm.Script` class that
+closely mirrors [Source Text Module Record][]s as defined in the ECMAScript
+specification.
+
+Unlike `vm.Script` however, every `vm.Module` object is bound to a context from
+its creation. Operations on `vm.Module` objects are intrinsically asynchronous,
+in contrast with the synchronous nature of `vm.Script` objects. With the help
+of async functions, however, manipulating `vm.Module` objects is fairly
+straightforward.
+
+Using a `vm.Module` object requires four distinct steps: creation/parsing,
+linking, instantiation, and evaluation. These four steps are illustrated in the
+following example.
+
+*Note*: This implementation lies at a lower level than the [ECMAScript Module
+loader][]. There is also currently no way to interact with the Loader, though
+support is planned.
+
+```js
+const vm = require('vm');
+
+const contextifiedSandbox = vm.createContext({ secret: 42 });
+
+(async () => {
+  // Step 1
+  //
+  // Create a Module by constructing a new `vm.Module` object. This parses the
+  // provided source text, throwing a `SyntaxError` if anything goes wrong. By
+  // default, a Module is created in the top context. But here, we specify
+  // `contextifiedSandbox` as the context this Module belongs to.
+  //
+  // Here, we attempt to obtain the default export from the module "foo", and
+  // put it into local binding "secret".
+
+  const bar = new vm.Module(`
+    import s from 'foo';
+    s;
+  `, { context: contextifiedSandbox });
+
+
+  // Step 2
+  //
+  // "Link" the imported dependencies of this Module to it.
+  //
+  // The provided linking callback (the "linker") accepts two arguments: the
+  // parent module (`bar` in this case) and the string that is the specifier of
+  // the imported module. The callback is expected to return a Module that
+  // corresponds to the provided specifier, with certain requirements documented
+  // in `module.link()`.
+  //
+  // If linking has not started for the returned Module, the same linker
+  // callback will be called on the returned Module.
+  //
+  // Even top-level Modules without dependencies must be explicitly linked. The
+  // callback provided would never be called, however.
+  //
+  // The link() method returns a Promise that will be resolved when all the
+  // Promises returned by the linker resolve.
+  //
+  // Note: This is a contrived example in that the linker function creates a new
+  // "foo" module every time it is called. In a full-fledged module system, a
+  // cache would probably be used to avoid duplicated modules.
+
+  async function linker(referencingModule, specifier) {
+    if (specifier === 'foo') {
+      return new vm.Module(`
+        // The "secret" variable refers to the global variable we added to
+        // "contextifiedSandbox" when creating the context.
+        export default secret;
+      `, { context: referencingModule.context });
+
+      // Using `contextifiedSandbox` instead of `referencingModule.context`
+      // here would work as well.
+    }
+    throw new Error(`Unable to resolve dependency: ${specifier}`);
+  }
+  await bar.link(linker);
+
+
+  // Step 3
+  //
+  // Instantiate the top-level Module.
+  //
+  // Only the top-level Module needs to be explicitly instantiated; its
+  // dependencies will be recursively instantiated by instantiate().
+
+  bar.instantiate();
+
+
+  // Step 4
+  //
+  // Evaluate the Module. The evaluate() method returns a Promise with a single
+  // property "result" that contains the result of the very last statement
+  // executed in the Module. In the case of `bar`, it is `s;`, which refers to
+  // the default export of the `foo` module, the `secret` we set in the
+  // beginning to 42.
+
+  const { result } = await bar.evaluate();
+
+  console.log(result);
+  // Prints 42.
+})();
+```
+
+### Constructor: new vm.Module(code[, options])
+
+* `code` {string} JavaScript Module code to parse
+* `options`
+  * `url` {string} URL used in module resolution and stack traces. **Default**:
+    `'vm:module(i)'` where `i` is a context-specific ascending index.
+  * `context` {Object} The [contextified][] object as returned by the
+    `vm.createContext()` method, to compile and evaluate this Module in.
+  * `lineOffset` {integer} Specifies the line number offset that is displayed
+    in stack traces produced by this Module.
+  * `columnOffset` {integer} Spcifies the column number offset that is displayed
+    in stack traces produced by this Module.
+
+Creates a new ES `Module` object.
+
+### module.dependencySpecifiers
+
+* {string[]}
+
+The specifiers of all dependencies of this module. The returned array is frozen
+to disallow any changes to it.
+
+Corresponds to the [[RequestedModules]] field of [Source Text Module Record][]s
+in the ECMAScript specification.
+
+### module.error
+
+* {any}
+
+If the `module.status` is `'errored'`, this property contains the exception thrown
+by the module during evaluation. If the status is anything else, accessing this
+property will result in a thrown exception.
+
+*Note*: `undefined` cannot be used for cases where there is not a thrown
+exception due to possible ambiguity with `throw undefined;`.
+
+Corresponds to the [[EvaluationError]] field of [Source Text Module Record][]s
+in the ECMAScript specification.
+
+### module.linkingStatus
+
+* {string}
+
+The current linking status of `module`. It will be one of the following values:
+
+- `'unlinked'`: `module.link()` has not yet been called.
+- `'linking'`: `module.link()` has been called, but not all Promises returned by
+  the linker function have been resolved yet.
+- `'linked'`: `module.link()` has been called, and all its dependencies have
+  been successfully linked.
+- `'errored'`: `module.link()` has been called, but at least one of its
+  dependencies failed to link, either because the callback returned a Promise
+  that is rejected, or because the Module the callback returned is invalid.
+
+### module.namespace
+
+* {Object}
+
+The namespace object of the module. This is only available after instantiation
+(`module.instantiate()`) has completed.
+
+Corresponds to the [GetModuleNamespace][] abstract operation in the ECMAScript
+specification.
+
+### module.status
+
+* {string}
+
+The current status of the module. Will be one of:
+
+- `'uninstantiated'`: The module is not instantiated. It may because of any of
+  the following reasons:
+
+  - The module was just created.
+  - `module.instantiate()` has been called on this module, but it failed for
+    some reason.
+
+  This status does not convey any information regarding if `module.link()` has
+  been called. See `module.linkingStatus` for that.
+
+- `'instantiating'`: The module is currently being instantiated through a
+  `module.instantiate()` call on itself or a parent module.
+
+- `'instantiated'`: The module has been instantiated successfully, but
+  `module.evaluate()` has not yet been called.
+
+- `'evaluating'`: The module is being evaluated through a `module.evaluate()` on
+  itself or a parent module.
+
+- `'evaluated'`: The module has been successfully evaluated.
+
+- `'errored'`: The module has been evaluated, but an exception was thrown.
+
+Other than `'errored'`, this status string corresponds to the specification's
+[Source Text Module Record][]'s [[Status]] field. `'errored'` corresponds to
+`'evaluated'` in the specification, but with [[EvaluationError]] set to a value
+that is not `undefined`.
+
+### module.url
+
+* {string}
+
+The URL of the current module, as set in the constructor.
+
+### module.evaluate([options])
+
+* `options` {Object}
+  * `timeout` {number} Specifies the number of milliseconds to evaluate
+    before terminating execution. If execution is interrupted, an [`Error`][]
+    will be thrown.
+  * `breakOnSigint` {boolean} If `true`, the execution will be terminated when
+    `SIGINT` (Ctrl+C) is received. Existing handlers for the event that have
+    been attached via `process.on("SIGINT")` will be disabled during script
+    execution, but will continue to work after that. If execution is
+    interrupted, an [`Error`][] will be thrown.
+* Returns: {Promise}
+
+Evaluate the module.
+
+This must be called after the module has been instantiated; otherwise it will
+throw an error. It could be called also when the module has already been
+evaluated, in which case it will do one of the following two things:
+
+- return `undefined` if the initial evaluation ended in success (`module.status`
+  is `'evaluated'`)
+- rethrow the same exception the initial evaluation threw if the initial
+  evaluation ended in an error (`module.status` is `'errored'`)
+
+This method cannot be called while the module is being evaluated
+(`module.status` is `'evaluating'`) to prevent infinite recursion.
+
+Corresponds to the [Evaluate() concrete method][] field of [Source Text Module
+Record][]s in the ECMAScript specification.
+
+### module.instantiate()
+
+Instantiate the module. This must be called after linking has completed
+(`linkingStatus` is `'linked'`); otherwise it will throw an error. It may also
+throw an exception if one of the dependencies does not provide an export the
+parent module requires.
+
+However, if this function succeeded, further calls to this function after the
+initial instantiation will be no-ops, to be consistent with the ECMAScript
+specification.
+
+Unlike other methods operating on `Module`, this function completes
+synchronously and returns nothing.
+
+Corresponds to the [Instantiate() concrete method][] field of [Source Text
+Module Record][]s in the ECMAScript specification.
+
+### module.link(linker)
+
+* `linker` {Function}
+* Returns: {Promise}
+
+Link module dependencies. This method must be called before instantiation, and
+can only be called once per module.
+
+Two parameters will be passed to the `linker` function:
+
+- `referencingModule` The `Module` object `link()` is called on.
+- `specifier` The specifier of the requested module:
+
+  <!-- eslint-skip -->
+  ```js
+  import foo from 'foo';
+  //              ^^^^^ the module specifier
+  ```
+
+The function is expected to return a `Module` object or a `Promise` that
+eventually resolves to a `Module` object. The returned `Module` must satisfy the
+following two invariants:
+
+- It must belong to the same context as the parent `Module`.
+- Its `linkingStatus` must not be `'errored'`.
+
+If the returned `Module`'s `linkingStatus` is `'unlinked'`, this method will be
+recursively called on the returned `Module` with the same provided `linker`
+function.
+
+`link()` returns a `Promise` that will either get resolved when all linking
+instances resolve to a valid `Module`, or rejected if the linker function either
+throws an exception or returns an invalid `Module`.
+
+The linker function roughly corresponds to the implementation-defined
+[HostResolveImportedModule][] abstract operation in the ECMAScript
+specification, with a few key differences:
+
+- The linker function is allowed to be asynchronous while
+  [HostResolveImportedModule][] is synchronous.
+- The linker function is executed during linking, a Node.js-specific stage
+  before instantiation, while [HostResolveImportedModule][] is called during
+  instantiation.
+
+The actual [HostResolveImportedModule][] implementation used during module
+instantiation is one that returns the modules linked during linking. Since at
+that point all modules would have been fully linked already, the
+[HostResolveImportedModule][] implementation is fully synchronous per
+specification.
+
 ## Class: vm.Script
 <!-- YAML
 added: v0.3.1
@@ -518,8 +834,14 @@ associating it with the `sandbox` object is what this document refers to as
 [`vm.createContext()`]: #vm_vm_createcontext_sandbox_options
 [`vm.runInContext()`]: #vm_vm_runincontext_code_contextifiedsandbox_options
 [`vm.runInThisContext()`]: #vm_vm_runinthiscontext_code_options
+[GetModuleNamespace]: https://tc39.github.io/ecma262/#sec-getmodulenamespace
+[ECMAScript Module Loader]: esm.html#esm_ecmascript_modules
+[Evaluate() concrete method]: https://tc39.github.io/ecma262/#sec-moduleevaluation
+[HostResolveImportedModule]: https://tc39.github.io/ecma262/#sec-hostresolveimportedmodule
+[Instantiate() concrete method]: https://tc39.github.io/ecma262/#sec-moduledeclarationinstantiation
 [V8 Embedder's Guide]: https://github.com/v8/v8/wiki/Embedder's%20Guide#contexts
 [contextified]: #vm_what_does_it_mean_to_contextify_an_object
 [global object]: https://es5.github.io/#x15.1
 [indirect `eval()` call]: https://es5.github.io/#x10.4.2
 [origin]: https://developer.mozilla.org/en-US/docs/Glossary/Origin
+[Source Text Module Record]: https://tc39.github.io/ecma262/#sec-source-text-module-records

lib/internal/errors.js

@@ -676,6 +676,15 @@ E('ERR_V8BREAKITERATOR', 'Full ICU data not installed. ' +
   'See https://github.com/nodejs/node/wiki/Intl');
 E('ERR_VALID_PERFORMANCE_ENTRY_TYPE',
   'At least one valid performance entry type is required');
+E('ERR_VM_MODULE_ALREADY_LINKED', 'Module has already been linked');
+E('ERR_VM_MODULE_DIFFERENT_CONTEXT',
+  'Linked modules must use the same context');
+E('ERR_VM_MODULE_LINKING_ERRORED',
+  'Linking has already failed for the provided module');
+E('ERR_VM_MODULE_NOT_LINKED',
+  'Module must be linked before it can be instantiated');
+E('ERR_VM_MODULE_NOT_MODULE', 'Provided module is not an instance of Module');
+E('ERR_VM_MODULE_STATUS', 'Module status %s');
 E('ERR_ZLIB_BINDING_CLOSED', 'zlib binding closed');
 E('ERR_ZLIB_INITIALIZATION_FAILED', 'Initialization failed');