Skip to content

Commit

Permalink
[test-visibility] Remove parallel mode restrictions (#23512)
Browse files Browse the repository at this point in the history 
* remove parallel restriction

* more simplification
  • Loading branch information
@juan-fernandez
juan-fernandez authored Jun 5, 2024
1 parent 7c3e2eb commit bc27cbd
Showing 1 changed file with 55 additions and 63 deletions.
118 changes: 55 additions & 63 deletions content/en/tests/setup/javascript.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@ Supported test frameworks:

| Test Framework | Version | Notes |
|---|---|---|
| Jest | >= 24.8.0 | Only `jsdom` (in the `jest-environment-jsdom` package) and `node` (in the `jest-environment-node` package) are supported as test environments. Custom environments like `@jest-runner/electron/environment` in `jest-electron-runner` are not supported.<br><br>Only [`jest-circus`][1] is supported as [`testRunner`][2].<br><br>Jest >= 28 is only supported from `dd-trace>=2.7.0`.<br><br>[`test.concurrent`](#jests-testconcurrent) is not supported. |
| Mocha | >= 5.2.0 | Mocha >= 9.0.0 has [partial support](#known-limitations).<br><br>Mocha [parallel mode](#mocha-parallel-tests) is not supported. |
| Cucumber | >= 7.0.0 | Cucumber-js [parallel mode](#cucumber-parallel-tests) is not supported. |
| Cypress | >= 6.7.0 | From `dd-trace>=1.4.0`. |
| Playwright | >= 1.18.0 | From `dd-trace>=3.13.0` and `dd-trace>=2.26.0` for 2.x release line. |
| Jest | >= 24.8.0 | Only `jsdom` (in the `jest-environment-jsdom` package) and `node` (in the `jest-environment-node` package) are supported as test environments. Custom environments like `@jest-runner/electron/environment` in `jest-electron-runner` are not supported.<br><br>Only [`jest-circus`][1] is supported as [`testRunner`][2].<br><br>[`test.concurrent`](#jests-testconcurrent) is not supported. |
| Mocha | >= 5.2.0 | Mocha >= 9.0.0 has [partial support](#known-limitations). |
| Cucumber | >= 7.0.0 |
| Cypress | >= 6.7.0 |
| Playwright | >= 1.18.0 |

The instrumentation works at runtime, so any transpilers such as TypeScript, Webpack, or Babel are supported out-of-the-box.

Expand Down Expand Up @@ -107,7 +107,7 @@ You can add custom tags to your tests by using the current active span:
})
```

To create filters or `group by` fields for these tags, you must first create facets. For more information about adding tags, see the [Adding Tags][101] section of the Node.js custom instrumentation documentation.
To create filters or `group by` fields for these tags, you must first create facets. For more information about adding tags, see the [Adding Tags][1] section of the Node.js custom instrumentation documentation.


### Adding custom measures to tests
Expand All @@ -123,10 +123,10 @@ Just like tags, you can add custom measures to your tests by using the current a
})
```

For more information about custom measures, see the [Add Custom Measures Guide][102].
For more information about custom measures, see the [Add Custom Measures Guide][2].

[101]: /tracing/trace_collection/custom_instrumentation/nodejs?tab=locally#adding-tags
[102]: /tests/guides/add_custom_measures/?tab=javascripttypescript
[1]: /tracing/trace_collection/custom_instrumentation/nodejs?tab=locally#adding-tags
[2]: /tests/guides/add_custom_measures/?tab=javascripttypescript
{{% /tab %}}

{{% tab "Playwright" %}}
Expand All @@ -148,7 +148,7 @@ NODE_OPTIONS="-r dd-trace/ci/init" DD_ENV=ci DD_SERVICE=my-javascript-app yarn t

### Adding custom tags to tests

You can add custom tags to your tests by using the [custom annotations API from Playwright][101]:
You can add custom tags to your tests by using the [custom annotations API from Playwright][1]:

```javascript
test('user profile', async ({ page }) => {
Expand Down Expand Up @@ -202,14 +202,14 @@ The format of the annotations is the following, where `$TAG_NAME` is a *string*
"description": $TAG_VALUE
}
```
**Note**: `description` values in annotations are [typed as strings][102]. Numbers also work, but you may need to disable the typing error with `// @ts-expect-error`.
**Note**: `description` values in annotations are [typed as strings][2]. Numbers also work, but you may need to disable the typing error with `// @ts-expect-error`.

<div class="alert alert-warning">
<strong>Important</strong>: The <code>DD_TAGS</code> prefix is mandatory and case sensitive.
</div>

[101]: https://playwright.dev/docs/test-annotations#custom-annotations
[102]: https://playwright.dev/docs/api/class-testinfo#test-info-annotations
[1]: https://playwright.dev/docs/test-annotations#custom-annotations
[2]: https://playwright.dev/docs/api/class-testinfo#test-info-annotations
{{% /tab %}}

{{% tab "Cucumber" %}}
Expand Down Expand Up @@ -242,7 +242,7 @@ You can add custom tags to your test by grabbing the current active span:
})
```

To create filters or `group by` fields for these tags, you must first create facets. For more information about adding tags, see the [Adding Tags][101] section of the Node.js custom instrumentation documentation.
To create filters or `group by` fields for these tags, you must first create facets. For more information about adding tags, see the [Adding Tags][1] section of the Node.js custom instrumentation documentation.


### Adding custom measures to tests
Expand All @@ -258,10 +258,10 @@ You may also add custom measures to your test by grabbing the current active spa
})
```

For more information about custom measures, see the [Add Custom Measures Guide][102].
For more information about custom measures, see the [Add Custom Measures Guide][2].

[101]: /tracing/trace_collection/custom_instrumentation/nodejs?tab=locally#adding-tags
[102]: /tests/guides/add_custom_measures/?tab=javascripttypescript
[1]: /tracing/trace_collection/custom_instrumentation/nodejs?tab=locally#adding-tags
[2]: /tests/guides/add_custom_measures/?tab=javascripttypescript
{{% /tab %}}

{{% tab "Cypress" %}}
Expand Down Expand Up @@ -311,7 +311,7 @@ module.exports = defineConfig({
{{< /code-block >}}

#### Cypress `after:run` event
Datadog requires the [`after:run`][10] Cypress event to work, and Cypress does not allow multiple handlers for that event. If you defined handlers for `after:run` already, add the Datadog handler manually by importing `'dd-trace/ci/cypress/after-run'`:
Datadog requires the [`after:run`][2] Cypress event to work, and Cypress does not allow multiple handlers for that event. If you defined handlers for `after:run` already, add the Datadog handler manually by importing `'dd-trace/ci/cypress/after-run'`:

{{< code-block lang="javascript" filename="cypress.config.js" >}}
const { defineConfig } = require('cypress')
Expand All @@ -332,7 +332,7 @@ module.exports = defineConfig({
{{< /code-block >}}

#### Cypress `after:spec` event
Datadog requires the [`after:spec`][11] Cypress event to work, and Cypress does not allow multiple handlers for that event. If you defined handlers for `after:spec` already, add the Datadog handler manually by importing `'dd-trace/ci/cypress/after-spec'`:
Datadog requires the [`after:spec`][3] Cypress event to work, and Cypress does not allow multiple handlers for that event. If you defined handlers for `after:spec` already, add the Datadog handler manually by importing `'dd-trace/ci/cypress/after-spec'`:

{{< code-block lang="javascript" filename="cypress.config.js" >}}
const { defineConfig } = require('cypress')
Expand All @@ -355,9 +355,9 @@ module.exports = defineConfig({

### Cypress before version 10

These are the instructions if you're using a version older than `cypress@10`. See the [Cypress documentation][9] for more information about migrating to a newer version.
These are the instructions if you're using a version older than `cypress@10`. See the [Cypress documentation][4] for more information about migrating to a newer version.

1. Set [`pluginsFile`][2] to `"dd-trace/ci/cypress/plugin"`, for example, through [`cypress.json`][3]:
1. Set [`pluginsFile`][5] to `"dd-trace/ci/cypress/plugin"`, for example, through [`cypress.json`][6]:
{{< code-block lang="json" filename="cypress.json" >}}
{
"pluginsFile": "dd-trace/ci/cypress/plugin"
Expand All @@ -372,7 +372,7 @@ module.exports = (on, config) => {
}
{{< /code-block >}}

2. Add the following line to the **top level** of your [`supportFile`][4]:
2. Add the following line to the **top level** of your [`supportFile`][7]:
{{< code-block lang="javascript" filename="cypress/support/index.js" >}}
// Your code can be before this line
// require('./commands')
Expand All @@ -382,7 +382,7 @@ require('dd-trace/ci/cypress/support')
{{< /code-block >}}

#### Cypress `after:run` event
Datadog requires the [`after:run`][10] Cypress event to work, and Cypress does not allow multiple handlers for that event. If you defined handlers for `after:run` already, add the Datadog handler manually by importing `'dd-trace/ci/cypress/after-run'`:
Datadog requires the [`after:run`][2] Cypress event to work, and Cypress does not allow multiple handlers for that event. If you defined handlers for `after:run` already, add the Datadog handler manually by importing `'dd-trace/ci/cypress/after-run'`:

{{< code-block lang="javascript" filename="cypress/plugins/index.js" >}}
module.exports = (on, config) => {
Expand All @@ -397,7 +397,7 @@ module.exports = (on, config) => {
{{< /code-block >}}

#### Cypress `after:spec` event
Datadog requires the [`after:spec`][11] Cypress event to work, and Cypress does not allow multiple handlers for that event. If you defined handlers for `after:spec` already, add the Datadog handler manually by importing `'dd-trace/ci/cypress/after-spec'`:
Datadog requires the [`after:spec`][3] Cypress event to work, and Cypress does not allow multiple handlers for that event. If you defined handlers for `after:spec` already, add the Datadog handler manually by importing `'dd-trace/ci/cypress/after-spec'`:

{{< code-block lang="javascript" filename="cypress/plugins/index.js" >}}
module.exports = (on, config) => {
Expand Down Expand Up @@ -441,7 +441,7 @@ it('renders a hello world', () => {
})
```

To create filters or `group by` fields for these tags, you must first create facets. For more information about adding tags, see the [Adding Tags][5] section of the Node.js custom instrumentation documentation.
To create filters or `group by` fields for these tags, you must first create facets. For more information about adding tags, see the [Adding Tags][8] section of the Node.js custom instrumentation documentation.

### Adding custom measures to tests

Expand All @@ -459,24 +459,24 @@ it('renders a hello world', () => {
})
```

For more information about custom measures, see the [Add Custom Measures Guide][6].
For more information about custom measures, see the [Add Custom Measures Guide][9].

### Cypress - RUM integration

If the browser application being tested is instrumented using [Browser Monitoring][7], the Cypress test results and their generated RUM browser sessions and session replays are automatically linked. For more information, see the [Instrumenting your browser tests with RUM guide][8].
If the browser application being tested is instrumented using [Browser Monitoring][10], the Cypress test results and their generated RUM browser sessions and session replays are automatically linked. For more information, see the [Instrumenting your browser tests with RUM guide][11].


[1]: https://docs.cypress.io/guides/tooling/plugins-guide#Using-a-plugin
[2]: https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests#Plugins-file
[3]: https://docs.cypress.io/guides/references/configuration#cypress-json
[4]: https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests#Support-file
[5]: /tracing/trace_collection/custom_instrumentation/nodejs?tab=locally#adding-tags
[6]: /tests/guides/add_custom_measures/?tab=javascripttypescript
[7]: /real_user_monitoring/browser/setup
[8]: /continuous_integration/guides/rum_integration/
[9]: https://docs.cypress.io/guides/references/migration-guide#Migrating-to-Cypress-100
[10]: https://docs.cypress.io/api/plugins/after-run-api
[11]: https://docs.cypress.io/api/plugins/after-spec-api
[2]: https://docs.cypress.io/api/plugins/after-run-api
[3]: https://docs.cypress.io/api/plugins/after-spec-api
[4]: https://docs.cypress.io/guides/references/migration-guide#Migrating-to-Cypress-100
[5]: https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests#Plugins-file
[6]: https://docs.cypress.io/guides/references/configuration#cypress-json
[7]: https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests#Support-file
[8]: /tracing/trace_collection/custom_instrumentation/nodejs?tab=locally#adding-tags
[9]: /tests/guides/add_custom_measures/?tab=javascripttypescript
[10]: /real_user_monitoring/browser/setup
[11]: /continuous_integration/guides/rum_integration/
{{% /tab %}}

{{< /tabs >}}
Expand Down Expand Up @@ -580,7 +580,7 @@ The following is a list of the most important configuration settings that can be
**Environment variable**: `DD_TRACE_AGENT_URL`<br/>
**Default**: `http://localhost:8126`

For more information about `service` and `env` reserved tags, see [Unified Service Tagging][20]. All other [Datadog Tracer configuration][7] options can also be used.
For more information about `service` and `env` reserved tags, see [Unified Service Tagging][7]. All other [Datadog Tracer configuration][8] options can also be used.

## Collecting Git metadata

Expand Down Expand Up @@ -706,31 +706,25 @@ NODE_OPTIONS="-r dd-trace/ci/init" DD_CIVISIBILITY_MANUAL_API_ENABLED=1 DD_ENV=c
## Known limitations

### ES modules
[Mocha >=9.0.0][8] uses an ESM-first approach to load test files. That means that if [ES modules][9] are used (for example, by defining test files with the `.mjs` extension), _the instrumentation is limited_. Tests are detected, but there isn't visibility into your test. For more information about ES modules, see the [Node.js documentation][9].
[Mocha >=9.0.0][9] uses an ESM-first approach to load test files. That means that if [ES modules][10] are used (for example, by defining test files with the `.mjs` extension), _the instrumentation is limited_. Tests are detected, but there isn't visibility into your test. For more information about ES modules, see the [Node.js documentation][10].

### Browser tests
Browser tests executed with `mocha`, `jest`, `cucumber`, `cypress`, and `playwright` are instrumented by `dd-trace-js`, but visibility into the browser session itself is not provided by default (for example, network calls, user actions, page loads, and more.).

If you want visibility into the browser process, consider using [RUM & Session Replay][10]. When using Cypress, test results and their generated RUM browser sessions and session replays are automatically linked. For more information, see the [Instrumenting your browser tests with RUM guide][11].
If you want visibility into the browser process, consider using [RUM & Session Replay][11]. When using Cypress, test results and their generated RUM browser sessions and session replays are automatically linked. For more information, see the [Instrumenting your browser tests with RUM guide][12].

### Cypress interactive mode

Cypress interactive mode (which you can enter by running `cypress open`) is not supported by CI Visibility because some cypress events, such as [`before:run`][12], are not fired. If you want to try it anyway, pass `experimentalInteractiveRunEvents: true` to the [cypress configuration file][13].

### Mocha parallel tests
Mocha's [parallel mode][15] is not supported. Tests run in parallel mode are not instrumented by CI Visibility.

### Cucumber parallel tests
Cucumber's [parallel mode][16] is not supported. Tests run in parallel mode are not instrumented by CI Visibility.
Cypress interactive mode (which you can enter by running `cypress open`) is not supported by CI Visibility because some cypress events, such as [`before:run`][13], are not fired. If you want to try it anyway, pass `experimentalInteractiveRunEvents: true` to the [cypress configuration file][14].

### Jest's `test.concurrent`
Jest's [test.concurrent][17] is not supported.
Jest's [test.concurrent][15] is not supported.

### Jest's `--forceExit`
Jest's [--forceExit][21] option may cause data loss. Datadog tries to send data immediately after your tests finish, but shutting down the process abruptly can cause some requests to fail. Use `--forceExit` with caution.
Jest's [--forceExit][16] option may cause data loss. Datadog tries to send data immediately after your tests finish, but shutting down the process abruptly can cause some requests to fail. Use `--forceExit` with caution.

### Mocha's `--exit`
Mocha's [--exit][22] option may cause data loss. Datadog tries to send data immediately after your tests finish, but shutting down the process abruptly can cause some requests to fail. Use `--exit` with caution.
Mocha's [--exit][17] option may cause data loss. Datadog tries to send data immediately after your tests finish, but shutting down the process abruptly can cause some requests to fail. Use `--exit` with caution.

## Best practices

Expand Down Expand Up @@ -782,18 +776,16 @@ When you use this approach, both the testing framework and CI Visibility can tel
[4]: https://github.com/DataDog/dd-trace-js#version-release-lines-and-maintenance
[5]: https://istanbul.js.org/
[6]: /tests/code_coverage/?tab=javascripttypescript
[7]: /tracing/trace_collection/library_config/nodejs/?tab=containers#configuration
[8]: https://github.com/mochajs/mocha/releases/tag/v9.0.0
[9]: https://nodejs.org/api/packages.html#packages_determining_module_system
[10]: /real_user_monitoring/browser/
[11]: /continuous_integration/guides/rum_integration/
[12]: https://docs.cypress.io/api/plugins/before-run-api
[13]: https://docs.cypress.io/guides/references/configuration#Configuration-File
[15]: https://mochajs.org/#parallel-tests
[16]: https://github.com/cucumber/cucumber-js/blob/63f30338e6b8dbe0b03ddd2776079a8ef44d47e2/docs/parallel.md
[17]: https://jestjs.io/docs/api#testconcurrentname-fn-timeout
[7]: /getting_started/tagging/unified_service_tagging
[8]: /tracing/trace_collection/library_config/nodejs/?tab=containers#configuration
[9]: https://github.com/mochajs/mocha/releases/tag/v9.0.0
[10]: https://nodejs.org/api/packages.html#packages_determining_module_system
[11]: /real_user_monitoring/browser/
[12]: /continuous_integration/guides/rum_integration/
[13]: https://docs.cypress.io/api/plugins/before-run-api
[14]: https://docs.cypress.io/guides/references/configuration#Configuration-File
[15]: https://jestjs.io/docs/api#testconcurrentname-fn-timeout
[16]: https://jestjs.io/docs/cli#--forceexit
[17]: https://mochajs.org/#-exit
[18]: https://jestjs.io/docs/api#testeachtablename-fn-timeout
[19]: https://www.npmjs.com/package/mocha-each
[20]: /getting_started/tagging/unified_service_tagging
[21]: https://jestjs.io/docs/cli#--forceexit
[22]: https://mochajs.org/#-exit

0 comments on commit bc27cbd

Please sign in to comment.