pnpm-install-only

Install node_modules from package.json + package-lock.json

the pnpm install algorithm

1. build dependency tree from package.json and package-lock.json. this is handled by the lockTree function from npm/logical-tree. the original npm would deduplicate "transitive" dependencies and build a flat node_modules, but here we build a deep node_modules with symlinks to a local store in node_modues/.pnpm/.
2. unpack dependencies to the local store node_modues/.pnpm/. the *.tar.gz files are provided by npmlock2nix. to unpack, we call tar xf package.tar.gz
3. symlink first-level dependencies from node_modules/(name) to node_modues/.pnpm/(name)@(version)/node_modules/(name)
4. symlink second-level dependencies from node_modues/.pnpm/(name)@(version)/node_modules/(name) to node_modues/.pnpm/(parentName)@(parentVersion)/node_modules/(name)
5. for each dependency, run the lifecycle scripts preinstall install postinstall. process the dependencies in depth-first order (last-level dependencies first, first-level dependencies last), so that child-dependencies are available.
6. when the root package has a prepare or prepublish script, also install its devDependencies (TODO verify)
7. for the root package, run the lifecycle scripts preinstall install postinstall prepublish preprepare prepare postprepare. docs: lifecycle scripts (look for npm install)

details: lifecycle scripts

test file: package/package.json

{
  "name": "test-lifecycle-scripts",
  "version": "1.0.0",
  "scripts": {
    "preinstall": "node -p \"require.resolve('test')\" >preinstall.txt",
    "install": "node -p \"require.resolve('test')\" >install.txt",
    "postinstall": "node -p \"require.resolve('test')\" >postinstall.txt",
    "prepublish": "echo hello >prepublish.txt",
    "preprepare": "echo hello >preprepare.txt",
    "prepare": "echo hello >prepare.txt",
    "postprepare": "echo hello >postprepare.txt"
  },
  "dependencies": {
    "test": "*"
  }
}

test file: package.json

{
  "name": "test-project",
  "version": "1.0.0",
  "scripts": {
    "preinstall": "echo hello >preinstall.txt",
    "install": "echo hello >install.txt",
    "postinstall": "echo hello >postinstall.txt",
    "prepublish": "echo hello >prepublish.txt",
    "preprepare": "echo hello >preprepare.txt",
    "prepare": "echo hello >prepare.txt",
    "postprepare": "echo hello >postprepare.txt"
  },
  "dependencies": {
    "test-lifecycle-scripts": "file:package.tar.gz"
  }
}

# save package.json
mkdir package
# save package/package.json
rm package.tar.gz
tar czf package.tar.gz package
rm -rf node_modules
rm package-lock.json
npm init -y
npm i package.tar.gz
cat node_modules/*/*.txt

result

ls node_modules/*/*.txt -t -r | cat
node_modules/test-lifecycle-scripts/preinstall.txt
node_modules/test-lifecycle-scripts/install.txt
node_modules/test-lifecycle-scripts/postinstall.txt

cat node_modules/test-lifecycle-scripts/*.txt 
/tmp/test-project/node_modules/test/test.js
/tmp/test-project/node_modules/test/test.js
/tmp/test-project/node_modules/test/test.js

ls *.txt -t -r | cat
preinstall.txt
install.txt
postinstall.txt
prepublish.txt
preprepare.txt
prepare.txt
postprepare.txt

npm ci

why use npm ci

npm ci or "npm clean install" will

• delete any old node_modules
• only use locked dependencies from package-lock.json
• not modify package.json

tests

todo

this program should produce the same result as pnpm install, so testing can be as simple as

1. prepare a set of package.json and package-lock.json files. these files must be valid, since this program will do no validation
2. run this script, move node_modules to node_modules-actual
3. run pnpm import (to produce a pnpm-lock.yaml file) and run pnpm install, move node_modules to node_modules-expected
4. compare the two folders with diff -r node_modules-actual node_modules-expected

the only difference should be pnpm-internal files, like node_modules/.pnpm/lock.yaml (the current lockfile of pnpm)

non-standard behavior

this program should produce the same result as pnpm install

except for obvious bugs in pnpm, like pnpm does not install peerDependencies like npm v7. in this case, npm (the original nodejs package manager) defines the expected behavior

update: pnpm has now implemented the option auto-install-peers

pnpm config set auto-install-peers true

but this is NOT satisfying, because it has NO effect on pnpm install. it only has an effect on pnpm add some-package

so currently, pnpm can NOT be used as a drop-in replacement for npmv7

(and thanks to the insane complexity of pnpm, its hard to add this feature)

workaround: add a pnpm hook

pnpm/pnpm#3995 (comment)

// .pnpmfile.cjs

function readPackage(pkg) {
  pkg.dependencies = {
    ...pkg.peerDependencies,
    ...pkg.dependencies,
  }
  pkg.peerDependencies = {};

  return pkg;
}

module.exports = {
  hooks: {
    readPackage,
  },
};

similar projects

js2nix

https://github.com/canva-public/js2nix

The nodeModules is a Nix derivation that contains a compatible with Node.js module resolution algorithm layout. Note that the layout of the resulting node_modules is similar to what PNPM package manager is providing, that is not a flat layout but rather the canonical layout with symlinked (from the Nix store) npm packages into it.

To find out more about the project, its background, implementation details, how to use it please go to the documentation space.