Skip to content

Commit

Permalink
fix(cli): avoid calling rw-vite-build via yarn (#9624)
Browse files Browse the repository at this point in the history
@Josh-Walker-GM and I have been working on the memory issue in v6 (see
https://community.redwoodjs.com/t/memory-use-in-redwood-v6/5224/2). We
started by making a script that profiles the number of processes started
by `REDWOOD_DISABLE_TELEMETRY=1 yarn rw build --no-prerender` and the
amount of memory each one uses. While it's hard to get a consistent
number for memory (it differs between machines and OSes, among other
things), the number of processes spawned is consistent and sheds some
insight into what's going on.

In stable, `REDWOOD_DISABLE_TELEMETRY=1 yarn rw build --no-prerender`
spawns nine processes. While there's two esbuild ones that seem like
they could be deduplicated, there's three associated with
`rw-vite-build` that seem to be redundant that take up a sizable amount
of memory:

- `/bin/sh
/private/var/folders/2z/s01x66ln547dn8c9fqylc9f40000gn/T/xfs-c4a0728c/yarn
rw-vite-build
--webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web`
- `~/.nvm/versions/node/v18.18.2/bin/node
~/projects/redwood/rw-perf/test-project-6.4.1/.yarn/releases/yarn-3.7.0.cjs
rw-vite-build
--webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web`
- `~/.nvm/versions/node/v18.18.2/bin/node
~/projects/redwood/rw-perf/test-project-6.4.1/node_modules/@redwoodjs/vite/bins/rw-vite-build.mjs
--webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web`

There's three because internally in the build handler, we invoke
`rw-vite-build` via `yarn`:

```js
execa('yarn rw-vite-build ...')
```

These days on users' systems, yarn points to corepack, which looks up
which version of yarn to use based on `packageManager` in package.json.
There, it finds `3.7.0`, and also finds `.yarnrc.yml` that tell it to
use the bin in `.yarn/releases`. So it invokes that with the same args,
etc. Basically, it's a bunch of bin proxies one after another.
Ultimately yarn runs node on the file. It seems reasonable enough to
just do that ourselves via `require.resolve`. We understand that comes
with some tradeoffs when it comes to PnP, but we explicitly set the
`nodeLinker` to `node-modules` in `.yarnrc.yml` files.

When we do that, we have six processes instead of nine and a overall
reduction in memory. The three `rw-vite-build` processes are now just
one:

- `~/.nvm/versions/node/v18.18.2/bin/node
~/projects/redwood/rw-perf/test-project-6.4.1/node_modules/@redwoodjs/vite/bins/rw-vite-build.mjs
--webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web`

The plan is to get this into an RC and test with the community. It may
not be enough of a memory reduction for some projects but we'll continue
working on ways to reduce usage.
  • Loading branch information
jtoar authored Dec 5, 2023
1 parent db2f31f commit 8759c0c
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 2 deletions.
4 changes: 3 additions & 1 deletion packages/cli/src/commands/buildHandler.js
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,9 @@ export const handler = async ({
// We don't have any parallel tasks right now, but someone might add
// one in the future as a performance optimization.
await execa(
`yarn rw-vite-build --webDir="${rwjsPaths.web.base}" --verbose=${verbose}`,
`node ${require.resolve(
'@redwoodjs/vite/bins/rw-vite-build.mjs'
)} --webDir="${rwjsPaths.web.base}" --verbose=${verbose}`,
{
stdio: verbose ? 'inherit' : 'pipe',
shell: true,
Expand Down
3 changes: 2 additions & 1 deletion packages/vite/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@
"./react-node-loader": {
"types": "./dist/react-server-dom-webpack/node-loader.d.ts",
"default": "./dist/react-server-dom-webpack/node-loader.js"
}
},
"./bins/rw-vite-build.mjs": "./bins/rw-vite-build.mjs"
},
"bin": {
"rw-dev-fe": "./dist/devFeServer.js",
Expand Down

0 comments on commit 8759c0c

Please sign in to comment.