[BUG] The npm_config_node_gyp environment variable and npm config set node-gyp config don't work with npm 7

[DeeDeeG]

What / Why

I have been trying to test npm 7 with the npm_config_node_gyp env var set, or doing npm config set node-gyp=some_path. It isn't working. I believe I've traced back the problem to a basically hard-coded value here:

run-script/lib/make-spawn-args.js

Line 5 in f976fd3

const npm_config_node_gyp = require.resolve('node-gyp/bin/node-gyp.js')

By the way: Editing that (probably too simplistically) to the following makes it configurable again with the env var:

const npm_config_node_gyp = process.env.npm_config_node_gyp || require.resolve('node-gyp/bin/node-gyp.js')

When

Any time I try to use a custom location of node-gyp with the npm_config_node_gyp env var or after doing npm config set node-gyp=path/to/node-gyp/bin/node-gyp.js with npm 7.

Where

Any npm command I run that should run node-gyp to rebuild native code is affected.

How

Current Behavior

npm 7 always always runs its built-in copy of node-gyp. (Or, if there is a copy of node-gyp as a top-level dependency in the current project's node_modules folder, that copy from the local node_modules/node-gyp folder is used.)

Steps to Reproduce

Set up a project with native C++ code somewhere in the dependencies and try to get npm to run node-gyp on it. Example steps below, and example output below that:

• Clone or download node-gyp to your computer somewhere, install its dependencies
• Set the environment variable npm_config_node_gyp to /path/to/node-gyp/bin/node-gyp.js.
  • OR Set npm config set npm-config=/path/to/node-gyp/bin/node-gyp.js
• Set up a new npm project with mkdir x && cd x && npm init -y && npm install [some package that needs to build native C++ code]
• Make sure you're running npm 7 for the next step...
• Run npm ci --verbose --foreground-scripts to see verbose output, including the path node-gyp is at.

Relevant bit of the verbose output (click to expand):

The built-in node-gyp path:

[...]
> node-gyp rebuild

gyp info it worked if it ends with oko run [email protected] install node_modules/tree-sitter-carp node-gyp rebuild
gyp verb cli [
gyp verb cli   '/Users/[user]/n-prefix/bin/node',
gyp verb cli   '/Users/[user]/n-prefix/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js',
gyp verb cli   'rebuild'
gyp verb cli ]
gyp info using [email protected]
[...]

If you manage to customize the path to node-gyp:

> node-gyp rebuild

gyp info it worked if it ends with oko run [email protected] install node_modules/tree-sitter-carp node-gyp rebuild
gyp verb cli [
gyp verb cli   '/Users/[user]/n-prefix/bin/node',
gyp verb cli   '/Users/[user]/node-gyp/bin/node-gyp.js',
gyp verb cli   'rebuild'
gyp verb cli ]

Expected Behavior

• The path/copy of node-gyp that runs should be configurable, as it has been in npm 6, and as all the code/documentation seems to imply will work.

Who

• n/a

References

• n/a

[DeeDeeG]

I think this commit e4c7cb3 is the cause.

I think it is also the cause of the bugs I posted at npm/cli:

• [BUG] config var "node-gyp" not respected in npm 7 cli#2839
• [BUG] npx node-gyp@version runs the copy of node-gyp bundled with npm instead cli#3058

[Hazmi35]

Any updates on this issue? I just run into this issue.

This should be fixed, I need to use Visual Studio 2022, but node-gyp only supports it on v8.4.0, meanwhile, the bundled version from npm is only v8.2.0.
This is an undocumented change, no mention of this even in https://github.com/nodejs/node-gyp/blob/master/docs/Updating-npm-bundled-node-gyp.md

[legobeat]

@DeeDeeG @Hazmi35 @achou11 In case this is still relevant for either of you: Is #221 sufficient to fix the issue for you? If not, does #222 do the trick?

[Hazmi35]

@DeeDeeG @Hazmi35 @achou11 In case this is still relevant for either of you: Is #221 sufficient to fix the issue for you? If not, does #222 do the trick?

My only use case is to use the latest version of my own installed node-gyp instead of the bundled node-gyp from npm, either way is fine.