fix(cli): avoid calling rw-vite-build via yarn

[jtoar]

@Josh-Walker-GM and I have been working on the memory issue in v6 (see https://community.redwoodjs.com/t/memory-use-in-redwood-v6/5224/2). We started by making a script that profiles the number of processes started by REDWOOD_DISABLE_TELEMETRY=1 yarn rw build --no-prerender and the amount of memory each one uses. While it's hard to get a consistent number for memory (it differs between machines and OSes, among other things), the number of processes spawned is consistent and sheds some insight into what's going on.

In stable, REDWOOD_DISABLE_TELEMETRY=1 yarn rw build --no-prerender spawns nine processes. While there's two esbuild ones that seem like they could be deduplicated, there's three associated with rw-vite-build that seem to be redundant that take up a sizable amount of memory:

• /bin/sh /private/var/folders/2z/s01x66ln547dn8c9fqylc9f40000gn/T/xfs-c4a0728c/yarn rw-vite-build --webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web
• ~/.nvm/versions/node/v18.18.2/bin/node ~/projects/redwood/rw-perf/test-project-6.4.1/.yarn/releases/yarn-3.7.0.cjs rw-vite-build --webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web
• ~/.nvm/versions/node/v18.18.2/bin/node ~/projects/redwood/rw-perf/test-project-6.4.1/node_modules/@redwoodjs/vite/bins/rw-vite-build.mjs --webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web

There's three because internally in the build handler, we invoke rw-vite-build via yarn:

execa('yarn rw-vite-build ...')

These days on users' systems, yarn points to corepack, which looks up which version of yarn to use based on packageManager in package.json. There, it finds 3.7.0, and also finds .yarnrc.yml that tell it to use the bin in .yarn/releases. So it invokes that with the same args, etc. Basically, it's a bunch of bin proxies one after another. Ultimately yarn runs node on the file. It seems reasonable enough to just do that ourselves via require.resolve. We understand that comes with some tradeoffs when it comes to PnP, but we explicitly set the nodeLinker to node-modules in .yarnrc.yml files.

When we do that, we have six processes instead of nine and a overall reduction in memory. The three rw-vite-build processes are now just one:

• ~/.nvm/versions/node/v18.18.2/bin/node ~/projects/redwood/rw-perf/test-project-6.4.1/node_modules/@redwoodjs/vite/bins/rw-vite-build.mjs --webDir=~/projects/redwood/rw-perf/test-project-6.4.1/web

The plan is to get this into an RC and test with the community. It may not be enough of a memory reduction for some projects but we'll continue working on ways to reduce usage.

[Josh-Walker-GM]

The results of measuring via the little script we now have make it clear that this will improve our footprint by removing the overhead of additional processes.

[dac09]

Looks ok to me!

[dac09]

If this works, should we remove all cases of calling yarn internally to make it more consistent? We do call binaries via yarn as a standard pattern (which in itself is meant to be an optimisation) - but the important part is to call a script not the yarn part

[jtoar]

@dac09 i'd lean towards yes on that yeah. it would be quite a lift to move them all. i mean, ideally it's as easy as search and replace but i'm sure a few won't be happy for some random reason. josh and i will run the same analysis on other bins called via yarn and confirm it has this duplicative behavior