Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner: support running tests in process #53927

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1091,6 +1091,20 @@ generated as part of the test runner output. If no tests are run, a coverage
report is not generated. See the documentation on
[collecting code coverage from tests][] for more details.

### `--experimental-test-isolation=mode`

<!-- YAML
added: REPLACEME
-->

> Stability: 1.0 - Early development

Configures the type of test isolation used in the test runner. When `mode` is
`'process'`, each test file is run in a separate child process. When `mode` is
`'none'`, all test files run in the same process as the test runner. The default
isolation mode is `'process'`. This flag is ignored if the `--test` flag is not
present. See the [test runner execution model][] section for more information.

### `--experimental-test-module-mocks`

<!-- YAML
Expand Down Expand Up @@ -2196,7 +2210,9 @@ added:
-->

The maximum number of test files that the test runner CLI will execute
concurrently. The default value is `os.availableParallelism() - 1`.
concurrently. If `--experimental-test-isolation` is set to `'none'`, this flag
is ignored and concurrency is one. Otherwise, concurrency defaults to
`os.availableParallelism() - 1`.

### `--test-coverage-exclude`

Expand Down Expand Up @@ -2361,7 +2377,7 @@ added: v22.3.0

> Stability: 1.0 - Early development

Regenerates the snapshot file used by the test runner for [snapshot testing][].
Regenerates the snapshot files used by the test runner for [snapshot testing][].
Node.js must be started with the `--experimental-test-snapshots` flag in order
to use this functionality.

Expand Down Expand Up @@ -3534,6 +3550,7 @@ node --stack-trace-limit=12 -p -e "Error.stackTraceLimit" # prints 12
[snapshot testing]: test.md#snapshot-testing
[syntax detection]: packages.md#syntax-detection
[test reporters]: test.md#test-reporters
[test runner execution model]: test.md#test-runner-execution-model
[timezone IDs]: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
[tracking issue for user-land snapshots]: https://github.com/nodejs/node/issues/44014
[ways that `TZ` is handled in other environments]: https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html
40 changes: 28 additions & 12 deletions doc/api/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -445,18 +445,26 @@ in the [test runner execution model][] section.

### Test runner execution model

Each matching test file is executed in a separate child process. The maximum
number of child processes running at any time is controlled by the
[`--test-concurrency`][] flag. If the child process finishes with an exit code
of 0, the test is considered passing. Otherwise, the test is considered to be a
failure. Test files must be executable by Node.js, but are not required to use
the `node:test` module internally.
When process-level test isolation is enabled, each matching test file is
executed in a separate child process. The maximum number of child processes
running at any time is controlled by the [`--test-concurrency`][] flag. If the
child process finishes with an exit code of 0, the test is considered passing.
Otherwise, the test is considered to be a failure. Test files must be executable
by Node.js, but are not required to use the `node:test` module internally.

Each test file is executed as if it was a regular script. That is, if the test
file itself uses `node:test` to define tests, all of those tests will be
executed within a single application thread, regardless of the value of the
`concurrency` option of [`test()`][].

When process-level test isolation is disabled, each matching test file is
imported into the test runner process. Once all test files have been loaded, the
top level tests are executed with a concurrency of one. Because the test files
are all run within the same context, it is possible for tests to interact with
each other in ways that are not possible when isolation is enabled. For example,
if a test relies on global state, it is possible for that state to be modified
by a test originating from another file.

## Collecting code coverage

> Stability: 1 - Experimental
Expand Down Expand Up @@ -933,7 +941,7 @@ the [`--experimental-test-snapshots`][] command-line flag.
Snapshot files are generated by starting Node.js with the
[`--test-update-snapshots`][] command-line flag. A separate snapshot file is
generated for each test file. By default, the snapshot file has the same name
as `process.argv[1]` with a `.snapshot` file extension. This behavior can be
as the test file with a `.snapshot` file extension. This behavior can be
configured using the `snapshot.setResolveSnapshotPath()` function. Each
snapshot assertion corresponds to an export in the snapshot file.

Expand Down Expand Up @@ -1239,6 +1247,9 @@ added:
- v18.9.0
- v16.19.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/53927
description: Added the `isolation` option.
- version: v22.6.0
pr-url: https://github.com/nodejs/node/pull/53866
description: Added the `globPatterns` option.
Expand Down Expand Up @@ -1274,8 +1285,13 @@ changes:
* `inspectPort` {number|Function} Sets inspector port of test child process.
This can be a number, or a function that takes no arguments and returns a
number. If a nullish value is provided, each process gets its own port,
incremented from the primary's `process.debugPort`.
**Default:** `undefined`.
incremented from the primary's `process.debugPort`. This option is ignored
if the `isolation` option is set to `'none'` as no child processes are
spawned. **Default:** `undefined`.
* `isolation` {string} Configures the type of test isolation. If set to
`'process'`, each test file is run in a separate child process. If set to
`'none'`, all test files run in the current process. **Default:**
`'process'`.
* `only`: {boolean} If truthy, the test context will only run tests that
have the `only` option set
* `setup` {Function} A function that accepts the `TestsStream` instance
Expand Down Expand Up @@ -1727,9 +1743,9 @@ added: v22.3.0

* `fn` {Function} A function used to compute the location of the snapshot file.
The function receives the path of the test file as its only argument. If the
`process.argv[1]` is not associated with a file (for example in the REPL),
the input is undefined. `fn()` must return a string specifying the location of
the snapshot file.
test is not associated with a file (for example in the REPL), the input is
undefined. `fn()` must return a string specifying the location of the snapshot
snapshot file.

This function is used to customize the location of the snapshot file used for
snapshot testing. By default, the snapshot filename is the same as the entry
Expand Down
3 changes: 3 additions & 0 deletions doc/node.1
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,9 @@ Enable the experimental node:sqlite module.
.It Fl -experimental-test-coverage
Enable code coverage in the test runner.
.
.It Fl -experimental-test-isolation Ns = Ns Ar mode
Configures the type of test isolation used in the test runner.
.
.It Fl -experimental-test-module-mocks
Enable module mocking in the test runner.
.
Expand Down
2 changes: 1 addition & 1 deletion lib/internal/main/test_runner.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ markBootstrapComplete();

const options = parseCommandLine();

if (isUsingInspector()) {
if (isUsingInspector() && options.isolation === 'process') {
process.emitWarning('Using the inspector with --test forces running at a concurrency of 1. ' +
'Use the inspectPort option to run with concurrency');
options.concurrency = 1;
Expand Down
1 change: 1 addition & 0 deletions lib/internal/test_runner/harness.js
Original file line number Diff line number Diff line change
Expand Up @@ -334,4 +334,5 @@ module.exports = {
after: hook('after'),
beforeEach: hook('beforeEach'),
afterEach: hook('afterEach'),
startSubtestAfterBootstrap,
};
Loading
Loading