Yarn or npm for plugin development?

[HyperBrain]

This is a request for decision

Description

Currently the (development) infrastructure of the plugin is aimed to use npm as packaging tool
and contains a npm 5 compatible package-lock.json. To have reproducible PR and implementation verifications, only tools that cope with the existing lock files can and should be used.

We should decide if we stick with npm or move to yarn as development package manager. Currently yarn cannot use package-lock files, so the decision would be an "either or", as otherwise (in case we commit both lockfiles) care has to be taken to keep them synchronized - and that each time anyone submits a PR that changes dependencies, as well as when a new version is published (the version number changes).

Here are some interesting links that might be beneficial for the discussion:

Should I commit yarn.lock and package-lock.json files?
Yarn: Idea: support package-lock.json from npm 5
DOES NPM 5 DEPRECATE YARN?
Yarn determinism

Thanks to @franciscocpg for bringing this up 🥇

[franciscocpg]

Some scenarios

Without cache

yarn

First lets remove cache rm -rf node_modules && yarn cache clean
Then:
Command

time yarn

Result

yarn install v0.27.5
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 21.05s.
yarn  11,66s user 2,55s system 66% cpu 21,427 total

npm

First lets remove cache rm -rf node_modules && npm cache clean --force
Then:
Command

time npm i

Result

> spawn-sync@1.0.15 postinstall /home/francisco/programs/git/github.com/franciscocpg/serverless-webpack/node_modules/spawn-sync
> node postinstall


> serverless@1.18.0 postinstall /home/francisco/programs/git/github.com/franciscocpg/serverless-webpack/node_modules/serverless
> node ./scripts/postinstall.js

npm WARN serverless-webpack@2.0.0 requires a peer of webpack@* but none was installed.

added 371 packages in 46.809s
npm i  19,69s user 2,44s system 46% cpu 47,428 total

Conclusion

yarn - 21.427s
npm - 47.428s
In this scenario yarn runs about 2.21x faster.

With cache / without node_modules

yarn

First lets remove node_modules rm -rf node_modules
Then:
Command

time yarn

Result

yarn install v0.27.5
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 4.41s.
yarn  4,72s user 0,96s system 118% cpu 4,782 total

npm

First lets remove node_modules rm -rf node_modules
Then:
Command

time npm i

Result

> spawn-sync@1.0.15 postinstall /home/francisco/programs/git/github.com/franciscocpg/serverless-webpack/node_modules/spawn-sync
> node postinstall


> serverless@1.18.0 postinstall /home/francisco/programs/git/github.com/franciscocpg/serverless-webpack/node_modules/serverless
> node ./scripts/postinstall.js

npm WARN serverless-webpack@2.0.0 requires a peer of webpack@* but none was installed.

added 371 packages in 7.745s
npm i  14,21s user 1,58s system 189% cpu 8,330 total

Conclusion

yarn - 4.782s
npm - 8.330s
In this scenario yarn runs about 1.75x faster.

With cache and node_modules

yarn

Command

time yarn

Result

yarn install v0.27.5
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.37s.
yarn  0,82s user 0,05s system 115% cpu 0,751 total

npm

Command

time npm i

Result

npm WARN serverless-webpack@2.0.0 requires a peer of webpack@* but none was installed.

up to date in 1.642s
npm i  2,39s user 0,11s system 111% cpu 2,226 total

Conclusion

yarn - 0.751s
npm - 2.226s
In this scenario yarn runs about 2.96x faster.

Final Conclusion

So it looks like yarn is about 2x faster than npm in most cases.
Also node community are adopting yarn, eg docker-library/official-images#2705
IMO I see no reason not to adopt yarn.

[kikar]

npm is still too widely used to be abandoned. I would keep both of them for the moment being.

[HyperBrain]

Despite the speed of yarn, the linked articles show that npm 5 is currently more robust regarding dependency stability and integrity - yarn needs the package.json in combination to the yarn.lock to retrieve the dependencies, which might lead to different versions if the package.json changed independently, whereas package-lock.json contains the integral definition of all dependencies (including transient ones).

We could check in a package-lock.json as well as a yarn.lock in parallel (for master I can take care of that after merging PRs that change dependencies and for release preparations), so contributors can use their tool of choice to get the development environment up and running - as long as the package.json is not newer than the yarn.lock file.

As soon as yarn supports package.json files, we could even drop the yarn.lock (I do not really get why yarn does not support it already, because the format is the same as the shrinkwrap ;-) )

[franciscocpg]

@HyperBrain
So we are going to generate and commit them both (package-lock.json and yarn.lock) every time a dependencies change to keep it sync with both yarn and npm?

[HyperBrain]

Would imo be feasible (beginning with v3, as soon as master is prepared for that release). From the project history the only point where dependencies change (if at all) are releases or - very rare - if PRs introduce new dependencies. If it would happen frequently I'd go for npm for now to reduce efforts, but as I said I do not think that this is the case.

Are there any other opinions on that?

[franciscocpg]

It's fine for me

[HyperBrain]

Sorry for being silent for this issue for a long time now. Bringing the repo and project back on track left not really time to care about these more "uncritical" issues 😃 . I think we can bring that up again soon for further discussion (at least the examples now have current yarn.lock files thanks to @franciscocpg ).

[HyperBrain]

It turned out that having and maintaining both for the main repository adds a lot of overhead. So I'm in favor of keeping npm for the main repo, and yarn locks for the examples.
Any objections?

[franciscocpg]

Nope

[HyperBrain]

I'll close this now :)