Yarn workspaces are not properly cached

[ehmicky]

Related: #399, #196.

When using Yarn workspaces, each new build is running yarn again. It seems that node_modules are not properly cached.

This behavior is unrelated the Netlify Build beta (it happens both with and without it).

Repository to reproduce:

package.json

{
  "private": true,
  "workspaces": ["site"]
}

netlify.toml

[build]
command = "node --version"

site/package.json

{
  "name": "site",
  "version": "1.0.0",
  "license": "MIT",
  "dependencies": {
    "gatsby": "^2.4.2"
  }
}

Run yarn to create a yarn.lock. Then each deploy keeps showing:

6:51:24 PM: Started restoring cached node modules
6:51:24 PM: Finished restoring cached node modules
6:51:24 PM: Started restoring cached yarn cache
6:51:24 PM: Finished restoring cached yarn cache
6:51:25 PM: Installing NPM modules using Yarn version 1.17.0
6:51:25 PM: yarn install v1.17.0
6:51:25 PM: [1/4] Resolving packages...
6:51:26 PM: [2/4] Fetching packages...
6:51:51 PM: info fsevents@2.1.3: The platform "linux" is incompatible with this module.
6:51:51 PM: info "fsevents@2.1.3" is an optional dependency and failed compatibility check. Excluding it from installation.
6:51:51 PM: info fsevents@1.2.13: The platform "linux" is incompatible with this module.
6:51:51 PM: info "fsevents@1.2.13" is an optional dependency and failed compatibility check. Excluding it from installation.
6:51:51 PM: [3/4] Linking dependencies...
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby@2.22.9" has unmet peer dependency "react@^16.4.2".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby@2.22.9" has unmet peer dependency "react-dom@^16.4.2".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > @pmmmwh/react-refresh-webpack-plugin@0.3.2" has incorrect peer dependency "react-refresh@^0.8.2".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > @reach/router@1.3.3" has unmet peer dependency "react@15.x || 16.x || 16.4.0-alpha.0911da3".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > @reach/router@1.3.3" has unmet peer dependency "react-dom@15.x || 16.x || 16.4.0-alpha.0911da3".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > gatsby-link@2.4.3" has unmet peer dependency "react@^16.4.2".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > gatsby-link@2.4.3" has unmet peer dependency "react-dom@^16.4.2".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > gatsby-react-router-scroll@3.0.1" has unmet peer dependency "react@^16.4.2".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > gatsby-react-router-scroll@3.0.1" has unmet peer dependency "react-dom@^16.4.2".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > react-hot-loader@4.12.21" has unmet peer dependency "react@^15.0.0 || ^16.0.0".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > react-hot-loader@4.12.21" has unmet peer dependency "react-dom@^15.0.0 || ^16.0.0".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > @reach/router > create-react-context@0.3.0" has unmet peer dependency "react@^0.14.0 || ^15.0.0 || ^16.0.0".
6:51:51 PM: warning "workspace-aggregator-bb3d6533-7bc7-4a3e-8af3-ed67450a57b9 > site > gatsby > @typescript-eslint/eslint-plugin > tsutils@3.17.1" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
6:52:02 PM: [4/4] Building fresh packages...
6:52:04 PM: success Saved lockfile.
6:52:04 PM: Done in 39.11s.

If the build was properly cached, the 4 steps of Yarn would stopped at step 1, and take less than 1 second (not 39 seconds).

Deploy URL: https://app.netlify.com/sites/compassionate-minsky-6a5f7e/deploys/5ec802e7a8a2ed00060a7e72

[bsgreenb]

Bumping this. Netlify should be fixing this if they're going to advertise their monorepo support https://www.netlify.com/blog/2019/10/09/launching-monorepo-support-for-netlify-sites/

[kaganjd]

Exciting update about Yarn workspaces: we’ve made improvements to how we cache dependencies for these projects. You can try out our new caching by creating an environment variable called NETLIFY_YARN_WORKSPACES and setting it to "true"

We want your feedback before we roll this out more widely- please let us know here or in the forum what your experience is like! You can also read more about this change here: https://answers.netlify.com/t/improved-caching-for-yarn-workspaces/36066

[JGAntunes]

#526 is now running for all of our builds, closing this issue 👍