Support for Jest / use of ES modules dynamic import in SDK

[thibaudcolas]

Apologies if this isn’t the best place to report this! I couldn’t find information about Jest support anywhere in official resources so I’m not sure whether this would be considered a bug or not.

The problem

In #411, the import of SDK utils has been switched to use an ES modules dynamic import. This is problematic for users of Jest, because Jest uses Node’s VM API to execute code. The VM modules API for ES modules support is experimental (only available behind the --experimental-vm-modules feature flag).

For the Puppeteer SDK – this means v2.0.0 works fine in Jest, and v2.0.1 doesn’t work, with the following error when running percySnapshot:

TypeError: _vm(...).SyntheticModule is not a constructor

Environment

• Node version: v16.13.2
• @percy/cli version: v1.1.2
• Version of Percy SDK you’re using: @percy/puppeteer@2.0.1
• If needed, a build or snapshot ID:
• OS version: macOS Monterey 12.3
• Type of shell command-line [interface]: zsh?

Details

Jest has experimental support for ES modules, so this can be resolved by adding the correct feature flag for Node to expose its vm.Module API. I believe there are a few things worth addressing in the SDK nonetheless (assuming the SDK attempts to support Jest):

• There are many known issues with this experimental API in Node (hence why it’s experimental, and why Jest documents it as such), so at least as of now even with the latest Node 18 release, it won’t be as reliable as the equivalent CommonJS implementation.
• This change shouldn’t be in a patch release since it’s a breaking change for Jest users.
• If the SDK keeps using ES modules anyway, then this at least needs to be documented clearly.

Debug logs

The only relevant bit is the stack trace.

[…]
[percy:cli:exec] Running "jest" (1ms)
  ● Groups › loads

    TypeError: _vm(...).SyntheticModule is not a constructor

      39 |   beforeAll(async () => {
      40 |     await page.goto(`${TEST_ORIGIN}/admin/groups/2/`);
    > 41 |     await percySnapshot(page, 'test name')
         |           ^
      42 |   }, 10000);

      at Runtime.loadCjsAsEsm (../../../node_modules/jest-runtime/build/index.js:657:20)
      at percySnapshot (node_modules/@percy/puppeteer/index.js:9:15)
      at Object.<anonymous> (groups.test.js:41:11)

A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --detectOpenHandles to find leaks.

[…]
Ran all test suites.
[percy:core:browser] Closing browser (25951ms)
[percy:core:browser] Browser closed (29ms)
[percy:core] Build not created (0ms)

Code to reproduce issue

I can provide a small reproduction case once confirmed Jest is indeed supported.

---

Workarounds

For people running into this, there are three possible workarounds:

Revert to v2.0.0 of the Puppeteer SDK

It works.

Run the v2.0.1 SDK with Node flag

Following Jest’s documentation. Here is what it looks like:

NODE_OPTIONS=--experimental-vm-modules percy exec -- jest

This will output a lot of warning messages.

Use @percy/sdk-utils directly

The code of v2.0.0 of the SDK can be copy-pasted and tweaked as needed to improve Jest support.

[Robdel12]

Thanks for the issue! Hopefully one day Jest will catch up to ESM. Until then, we've removed the dynamic import (fixed in #430 & released in v2.0.2)