Cypress and Yarn 2 PNP support

[kirill-konshin]

Current behavior:

In Yarn 2 monorepo I have a package a and tests in package b.

When I do

// b/cypress/plugins/index
module.exports = require('a/src/cypress/plugins');

It works as expected.

But

// a/cypress/commands/index
require('a/src/cypress/commands');

Fails with error:

Oops...we found an error preparing this test file:

  /Users/xxx/b/cypress/support/index.js

The error was:

Error: Cannot find module 'a/src/cypress/support' from '/Users/xxx/b/cypress/support'

Looks like Yarn's Plug'n'play file .pnp.js is not present in second (commands) context.

Desired behavior:

Require of monorepo packages should work.

[kirill-konshin]

@bahmutov @jennifer-shehane please take a look at this issue, this is a huge blocker since there's no known workaround... Yarn 2.1.1 has been recently released, which means it's time to support it. Is there anything I can help?

@arcanis I was trying to find a way to import .pnp.js to enable proper resolution mechanics, but it did not help. Also I've tried https://yarnpkg.com/advanced/pnpify but seems that Cypress is launching some other context (for the browser?) which is not connected to main one, which is aware of PNP.

[bahmutov]

@kirill-konshin you can open a PR with at least some work showing what is going on in CLI when Yarn v2 is used to install Cypress

[kirill-konshin]

@bahmutov here you go https://github.com/kirill-konshin/cypress-test-tiny-yarn-pnp/pull/1/files (branch bug), monorepo is actually not needed, since none of PNP resolutions work.

Just run

$ yarn install
$ yarn test

And get an error:

Error: Cannot find module 'cypress-plugin-snapshots/commands' from '/Users/xxx/cypress-test-tiny-yarn-pnp/cypress/support'

[kirill-konshin]

@bahmutov Any updates? We're completely blocked for almost a month, are there any workarounds or hacks?

[bahmutov]

We did not work on this issue, since it has low priority overall. If you know how to make Cypress NPM module play nicely with Yarn 2, you can open a PR into Cypress with the fix.

PS: The error above seems to indicate a problem not with the Cypress itself, but with another plugin, no?

[kirill-konshin]

@bahmutov I wish I could make a PR, but I am not familiar with Cypress codebase at all.

I know that Yarn 2 is shipped with a set of tools https://next.yarnpkg.com/advanced/pnpapi and https://next.yarnpkg.com/advanced/pnpify that can be used to resolve packages. Superficially it looks like Cypress is creating a new Node (or Browser) context, which is not aware of PNP resolution (unlike the main process).

The error above is a Cypress problem because it would happen with ANY plugin, since it is related to resolution algorithm in general.

Given that this issue has ben linked from another issue, there is some traction, and Yarn 2 is already public, so please consider raising the priority.

[kirill-konshin]

@bahmutov any updates?

[bahmutov]

No updates, we always post PRs or anything done under issues

…

Sent from my iPhone

On Sep 1, 2020, at 15:33, Kirill Konshin ***@***.***> wrote:  @bahmutov any updates? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[kirill-konshin]

@bahmutov is there anything else that I can do to speed it up? Do you have any ETA?

[bahmutov]

I would do a pull request with the solution if yarn 2 is a high priority for your needs

…

Sent from my iPhone

On Sep 1, 2020, at 16:11, Kirill Konshin ***@***.***> wrote:  @bahmutov is there anything else that I can do to speed it up? Do you have any ETA? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[kirill-konshin]

Like I said before, I’m not familiar with Cypress architecture and this bug seems to be buried quite deep, in the code where cypress resolves dependencies. Pnp has to be injected there, or their resolution has to be used. If you can point out the direction or provide a fix - it would be much better.

[bahmutov]

But isn’t it just the part of Cypress CLI which is quite limited?

…

Sent from my iPhone

On Sep 1, 2020, at 16:24, Kirill Konshin ***@***.***> wrote:  Like I said before, I’m not familiar with Cypress architecture and this bug seems to be buried quite deep, in the code where cypress resolves dependencies. Pnp has to be injected there, or their resolution has to be used. If you can point out the direction or provide a fix - it would be much better. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[kirill-konshin]

It is definitely not part of CLI, it is the part that involves browser, because it all works in CLI. Please take a closer look at bug description: it works for plugins and does not work for commands.

[marcneander]

Adding the pnp-webpack-plugin is needed until cypress is using webpack 5.

const webpackPreprocessor = require('@cypress/webpack-preprocessor');
const PnpWebpackPlugin = require('pnp-webpack-plugin');

module.exports = (on) => {
    const options = {
        webpackOptions: {
            resolve: {
                plugins: [
                    PnpWebpackPlugin
                ]
            },
            resolveLoader: {
                plugins: [
                    PnpWebpackPlugin.moduleLoader(module)
                ]
            }
        },
        watchOptions: {}
    };

    on('file:preprocessor', webpackPreprocessor(options));
};

[kirill-konshin]

@marcneander it did work, thanks a lot! Totally makes sense why.

[masihjahangiri]

Cypress should support built-in yarn 2 (PNP algorithm) and it's really important.
We use yarn 2 for the zero-install feature to improve our CI build time, and If we use nodeLinker, we miss that feature.

[benwainwright]

+1 for this please. I'm currently completely unable to write my integration tests or any code related to it in TypeScript in my otherwise entirely TypeScript based yarn2/pnp project.

[kleingtm]

The pnp-webpack-plugin did not work for me, however...

I had luck in a TypeScript + Cypress project doing the following in the Cypress plugins/index.js file:

const deepmerge = require("deepmerge");
const path = require("path");

module.exports = (on, config) => {

        // deep-merge config with the actual reporter 
        config =  deepmerge(config, {
            reporter: require(
                require(path.join(config.projectRoot, ".pnp.js")).resolveToUnqualified("cypress-multi-reporters", __filename)
            ),
            reporterOptions: {
                ... // put reporter options here
            }
        })
        ... // other plugin config
      

The key being yarn pnp API's resolveToUnqualified function which will spit out the pnp-ed module location.
https://next.yarnpkg.com/advanced/pnpapi#resolvetounqualified

[johnwiseheart]

As per #15241, this is also broken for resolving typescript cypress config:

plugins/index.ts:

/// <reference types="cypress" />

const config: Cypress.PluginConfig = (on, config) => {
  // plugin config
};

export default config;

Fails with various errors due to it being run without transpilation.

This is not in a monorepo configuration.

[ysozer]

Has anyone succeeded with using PnpWebpackPlugin? I am also blocked with yarn 2 implementation

[missing1984]

Webpack 5 has built in pnp support. does Cypress have plan to switch default webpack preprocessor to webpack 5?

[AaronBuxbaum]

Is there any movement on this PR? It's pretty annoying that this is the only package that forces us to use nodeLinker: node-modules

[jennifer-shehane]

@AaronBuxbaum You can track the currently opened PR here: #15623 Although this PR will not address all of the issues preventing Cypress from fully working with Yarn 2.

[jpulec]

If anyone else comes across this, managed to get this to work by just making sure that the PnP plugin gets added to the webpack config that Cypress uses. This works with cypress@7.3.0 using PnP with yarn workspaces. The following just needs to get added to you cypress/plugins/index.js.

module.exports = (on) => {
    // You need to use your PnP file to resolve files. The following is just the path that traverses up to where my PnP file is in our repo root.
    const pnp = require('../../../../.pnp.js');
    // I couldn't find this documented, but this appears to monkey-patch require and do all the other necessary things to get PnP working
    pnp.setup();

    // This is where we'll be resolving packages from. In this case, it is a workspace package we have named, but using a relative path here should work as well to resolve from a specific folder.
    const targetModule = '@mypackages/package';

    // We need to patch the webpack config that cypress uses (which is v4 which needs a plugin for PnP support)
    const webpackPreprocessor = require(pnp.resolveToUnqualified('@cypress/webpack-preprocessor', targetModule));
    const PnpWebpackPlugin = require(pnp.resolveToUnqualified('pnp-webpack-plugin', targetModule));

    const options = {
        webpackOptions: {
            resolve: {
                plugins: [
                    PnpWebpackPlugin
                ]
            },
            resolveLoader: {
                plugins: [
                    PnpWebpackPlugin.moduleLoader(module)
                ]
            }
        },
        watchOptions: {}
    };

    on('file:preprocessor', webpackPreprocessor(options));
};

You also may need to add webpack@^4 as a dependency if you're using strict PnP mode, since it's peerDependency of @cypress/webpack-preprocessor.

[ottoo]

If anyone else comes across this, managed to get this to work by just making sure that the PnP plugin gets added to the webpack config that Cypress uses. This works with cypress@7.3.0 using PnP with yarn workspaces. The following just needs to get added to you cypress/plugins/index.js.

module.exports = (on) => {
    // You need to use your PnP file to resolve files. The following is just the path that traverses up to where my PnP file is in our repo root.
    const pnp = require('../../../../.pnp.js');
    // I couldn't find this documented, but this appears to monkey-patch require and do all the other necessary things to get PnP working
    pnp.setup();

    // This is where we'll be resolving packages from. In this case, it is a workspace package we have named, but using a relative path here should work as well to resolve from a specific folder.
    const targetModule = '@mypackages/package';

    // We need to patch the webpack config that cypress uses (which is v4 which needs a plugin for PnP support)
    const webpackPreprocessor = require(pnp.resolveToUnqualified('@cypress/webpack-preprocessor', targetModule));
    const PnpWebpackPlugin = require(pnp.resolveToUnqualified('pnp-webpack-plugin', targetModule));

    const options = {
        webpackOptions: {
            resolve: {
                plugins: [
                    PnpWebpackPlugin
                ]
            },
            resolveLoader: {
                plugins: [
                    PnpWebpackPlugin.moduleLoader(module)
                ]
            }
        },
        watchOptions: {}
    };

    on('file:preprocessor', webpackPreprocessor(options));
};

You also may need to add webpack@^4 as a dependency if you're using strict PnP mode, since it's peerDependency of @cypress/webpack-preprocessor.

Thanks! I've been looking for a solution for this for a while, I got it working with this example.

[kleingtm]

Hello! I'm not sure exactly what you mean by the "targetModule". With pnp, you'll notice that modules are stored in a different location than "node_modules/[thing]". So when using pnp, require("thing") gets translated to require("/some/long/cached/path/of/the/thing/module"). The function resolveUnqualified essentially does that transformation for you. It returns the long "/some/long/cached/path/of/the/thing/module". Webpack 5 handles this for you so you don't need to worry about it. However, something about the Cypress / Webpack 5 integration isn't complete / doesn't do this. So in my example, I manually go and get the location of where pnp stored cypress-multi-reporters so that it can be used. Hope that helps, Tom

…

On Tue, Jun 29, 2021 at 11:20 PM Chris Castillo ***@***.***> wrote: resolveToUnqualified I'm currently trying to implement your solution. Thank you for doing all this work. But I have a question if you have the time. Do you know what the targetModule resolves to? I have tried reading Yarn PNP docs, but it just does not make sense to me. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8008 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACXQICIILMEPQT4CVSAHZC3TVKEPRANCNFSM4O5EC3OA> .

[missing1984]

cc @merceyz this is the cypress issue i was talking about. Cypress is using a forked process (from its electron app) to do compilation through webpack.

I think in order to make cypress compatible with yarn pnp natively . We need to either:

1. Fix pnp-webpack-plugin cannot work outside of pnp runtime. arcanis/pnp-webpack-plugin#32 and make it part of the default webpack config.
2. Fix the webpack 5 enhanced-resolve so it can load the correct pnpapi bases on source file(Do we have an issue opened?). and wait for Cypress switch to use webpack 5.

[cypress-bot]

The code for this is done in cypress-io/cypress#17335, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[cypress-bot]

Released in 8.1.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v8.1.0, please open a new issue.