Skip to content

Commit

Permalink
fix(cli): avoid calling rw-vite-build via yarn (#9624)
Browse files Browse the repository at this point in the history
@Josh-Walker-GM and I have been working on the memory issue in v6 (see
https://community.redwoodjs.com/t/memory-use-in-redwood-v6/5224/2). We
started by making a script that profiles the number of processes started
by `REDWOOD_DISABLE_TELEMETRY=1 yarn rw build --no-prerender` and the
amount of memory each one uses. While it's hard to get a consistent
number for memory (it differs between machines and OSes, among other
things), the number of processes spawned is consistent and sheds some
insight into what's going on.

In stable, `REDWOOD_DISABLE_TELEMETRY=1 yarn rw build --no-prerender`
spawns nine processes. While there's two esbuild ones that seem like
they could be deduplicated, there's three associated with
`rw-vite-build` that seem to be redundant that take up a sizable amount
of memory:

- `/bin/sh
/private/var/folders/2z/s01x66ln547dn8c9fqylc9f40000gn/T/xfs-c4a0728c/yarn
rw-vite-build
--webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web`
- `~/.nvm/versions/node/v18.18.2/bin/node
~/projects/redwood/rw-perf/test-project-6.4.1/.yarn/releases/yarn-3.7.0.cjs
rw-vite-build
--webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web`
- `~/.nvm/versions/node/v18.18.2/bin/node
~/projects/redwood/rw-perf/test-project-6.4.1/node_modules/@redwoodjs/vite/bins/rw-vite-build.mjs
--webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web`

There's three because internally in the build handler, we invoke
`rw-vite-build` via `yarn`:

```js
execa('yarn rw-vite-build ...')
```

These days on users' systems, yarn points to corepack, which looks up
which version of yarn to use based on `packageManager` in package.json.
There, it finds `3.7.0`, and also finds `.yarnrc.yml` that tell it to
use the bin in `.yarn/releases`. So it invokes that with the same args,
etc. Basically, it's a bunch of bin proxies one after another.
Ultimately yarn runs node on the file. It seems reasonable enough to
just do that ourselves via `require.resolve`. We understand that comes
with some tradeoffs when it comes to PnP, but we explicitly set the
`nodeLinker` to `node-modules` in `.yarnrc.yml` files.

When we do that, we have six processes instead of nine and a overall
reduction in memory. The three `rw-vite-build` processes are now just
one:

- `~/.nvm/versions/node/v18.18.2/bin/node
~/projects/redwood/rw-perf/test-project-6.4.1/node_modules/@redwoodjs/vite/bins/rw-vite-build.mjs
--webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web`

The plan is to get this into an RC and test with the community. It may
not be enough of a memory reduction for some projects but we'll continue
working on ways to reduce usage.
  • Loading branch information
jtoar committed Dec 5, 2023
1 parent 276ae53 commit 0c494d2
Showing 1 changed file with 13 additions and 8 deletions.
21 changes: 13 additions & 8 deletions packages/cli/src/commands/buildHandler.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,14 +114,19 @@ export const handler = async ({
// it could affect other things that run in parallel while building.
// We don't have any parallel tasks right now, but someone might add
// one in the future as a performance optimization.
await execa(`yarn rw-vite-build --webDir="${rwjsPaths.web.base}"`, {
stdio: verbose ? 'inherit' : 'pipe',
shell: true,
// This is needed for yarn to find the rw-vite-build binary
// It won't change process.cwd for anything else here, in this
// process
cwd: rwjsPaths.web.base,
})
await execa(
`node ${require.resolve(
'@redwoodjs/vite/bins/rw-vite-build.mjs'
)} --webDir="${rwjsPaths.web.base}"`,
{
stdio: verbose ? 'inherit' : 'pipe',
shell: true,
// This is needed for yarn to find the rw-vite-build binary
// It won't change process.cwd for anything else here, in this
// process
cwd: rwjsPaths.web.base,
}
)
} else {
await execa(
`yarn cross-env NODE_ENV=production webpack --config ${require.resolve(
Expand Down

0 comments on commit 0c494d2

Please sign in to comment.