[BUG] npx node-gyp@version runs the copy of node-gyp bundled with npm instead

[DeeDeeG]

Current Behavior:

• npx node-gyp@8 runs the older node-gyp 7.1.2 that is bundled with npm.
  • /home/[me]/n-prefix/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js

Same result if doing plain npx node-gyp. The latest version is v8.0.0, but v7.1.2 is used instead. Happens even if node-gyp v8 is installed globally, and no node-gyp is installed in the location npx is run from.

Expected Behavior:

A separate npx-specific copy of node-gyp v8 should run. Or when node-gyp v8 is installed globally, that could be used.

Steps To Reproduce:

npx node-gyp or npx node-gyp@8 or npx node-gyp@8.0.0, etc.

Environment:

• OS: Ubuntu 20.04, macOS 10.15.7 Catalina, Windows 10
• Node: v14.16.1, v16.0.0-nightly2021041040ace47396
• npm: 7.9.0

[nlf]

i can confirm this, and i see the reason it's happening, but i need to do a bit of research to see why the code is the way it is before we can consider changing it. i'll do some digging and follow up here when i have an answer

[DeeDeeG]

I filed another issue that seems related or at least tangentially related: #2839. I suppose they might be solved at the same time, depending on what the exact details turn out to be.

[DeeDeeG]

I think this started with npm/run-script@e4c7cb3.

As of that commit, @npmcli/run-script adds its bundled node-gyp executable/shim to the PATH. That shim is added relatively early in the PATH (earlier than the dir with npx's downloaded modules). And then @npmcli/run-script runs sh -c node-gyp [args].

So I think the wrong node-gyp is run via what's first in the PATH. (It is always @npmcli/run-script's shim.)

When debugging the execution with node --inspect-brk /home/[me]/npm/cli/bin/npx-cli.js I end up with a PATH var like this:

/home/[me]/[project I'm working in]/node_modules/.bin:/home/[me]/node_modules/.bin:/home/node_modules/.bin:/node_modules/.bin:/home/[me]/npm/cli/node_modules/@npmcli/run-script/lib/node-gyp-bin:/home/[me]/.npm/_npx/[lots o' digits]/node_modules/.bin:[ . . . rest of PATH is my usual PATH content and is not npm-related . . . ]

Note these two:

• /home/[me]/npm/cli/node_modules/@npmcli/run-script/lib/node-gyp-bin
• home/[me]/.npm/_npx/[lots o' digits]/node_modules/.bin

First we have @npmcli/run-script's dir with a node-gyp shim in it, just before home/[me]/.npm/_npx/[lots o' digits]/node_modules/.bin where the npx modules are stored, including the npx copy of node-gyp.

So I do think this has a similar cause to #2839.

[tavrez]

I think this is fixed in v8 of npm
Edit: nope, it's not

[DeeDeeG]

Yeah, it's probably not fixed, just that node-gyp 8 is now the bundled version of node-gyp as of npm 8.

Edit: Yep, the latest npm, npm 8.1.3, is still affected.

[wraithgar]

This should be fixed as of npm@8.16.0

~/D/n/s/npx $ npx node-gyp@9 --version
v9.0.0
~/D/n/s/npx $ npx node-gyp@8 --version
Need to install the following packages:
  node-gyp@8.4.1
Ok to proceed? (y) 
v8.4.1