error with node related tools when using DEVON_SOFTWARE_PATH

[hohwille]

This issue might be related to PR #934 that I tested this on:

*** Setting up npm ***
doInstall nodejs  silent  /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node
Cloning or updating download mirrors
Already up to date.
Resolved latest version of nodejs to v19.0.0 from /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/mirrors/nodejs/available-versions
doInstall npm    /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node/node_modules/npm
Cloning or updating download mirrors
Already up to date.
Resolved latest version of npm to 9.0.0 from /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/mirrors/npm/available-versions
Starting installation of npm in version 9.0.0 to /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node/node_modules/npm
Software npm seems to be already available in /Users/hohwille/projects/software/default/npm/9.0.0
Linking /Users/hohwille/projects/software/default/npm/9.0.0 to /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node/node_modules/npm
ln: /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node/node_modules/npm: No such file or directory
Trying to verify installation of npm (npm -v)
8.19.2
Success: verify installation of npm (npm)
cp: /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node/node_modules/npm/bin/npm: No such file or directory
cp: /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node/node_modules/npm/bin/npm.cmd: No such file or directory
cp: /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node/node_modules/npm/bin/npx: No such file or directory
cp: /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node/node_modules/npm/bin/npx.cmd: No such file or directory
Command 'npm setup' failed with exit code 1

******** ATTENTION ********
Failed to run devon npm setup
We are sorry for the inconvenience. Please check the above errors, resolve them and try again.
Exit code was 1

[hohwille]

Quite related error:

*** Setting up npm ***
doInstall nodejs  silent  /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node
Cloning or updating download mirrors
Already up to date.
Resolved latest version of nodejs to v19.0.0 from /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/mirrors/nodejs/available-versions
Starting installation of nodejs in version v19.0.0 to /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node
doDownload  /Users/hohwille/Downloads/devonfw-ide nodejs v19.0.0
Cloning or updating download mirrors
Already up to date.
Remapped architecture from x86_64 to x64 for software nodejs in version v19.0.0
Mapped extension to tar.gz for OS mac and software nodejs in version v19.0.0
Remapped OS from mac to darwin for software nodejs in version v19.0.0
Reading mirror URLs from /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/mirrors/nodejs/urls
Trying to download nodejs-v19.0.0-mac.tar.gz from https://nodejs.org/dist/v19.0.0/node-v19.0.0-darwin-x64.tar.gz
Artifact already exists at /Users/hohwille/Downloads/devonfw-ide/nodejs-v19.0.0-mac.tar.gz
To force update please delete the file and run again.
Received download file nodejs-v19.0.0-mac.tar.gz
Trying to run command tar xf '/Users/hohwille/Downloads/devonfw-ide/nodejs-v19.0.0-mac.tar.gz' -C '/Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/updates/extracted/node'
Success: run command tar
Successfully extracted archive nodejs-v19.0.0-mac.tar.gz to /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/updates/extracted/node
Changing source directory from /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/updates/extracted/node to /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/updates/extracted/node/node-v19.0.0-darwin-x64 due to single folder.
Trying to run command mv '/Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/updates/extracted/node/node-v19.0.0-darwin-x64' '/Users/hohwille/projects/software/default/nodejs/v19.0.0'
Success: run command mv
Successfully installed nodejs
The software node has been added. You need to rerun 'devon' command without arguments or restart your terminal to update your PATH so the newly installed software will be found.
Linking /Users/hohwille/projects/software/default/nodejs/v19.0.0 to /Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/software/node
Trying to verify installation of nodejs (node -v)
/Users/hohwille/projects/devon/workspaces/main/ide/scripts/target/integration-test/test-setup/scripts/command/../functions: line 85: node: command not found

******** ATTENTION ********
Failed to verify installation of nodejs (node)
We are sorry for the inconvenience. Please check the above errors, resolve them and try again.
Exit code was 127
Command 'node setup silent' failed with exit code 127

[hohwille]

Error seemss to be presennt also without PR #934
The problem starts here:

ide/scripts/src/main/resources/scripts/command/npm

Line 37 in d9ebcad

local software_dir="${NODE_HOME}/node_modules/npm"

and next line is:

ide/scripts/src/main/resources/scripts/command/npm

Line 38 in d9ebcad

doInstall "-" "${software_dir}" "npm" "${software_version}"

So if DEVON_SOFTWARE_PATH is configured, doInstall will install npm into DEVON_SOFTWARE_PATH instead and then tries to create a symbolic link to make that central installation visible in the current devonfw-ide installation.
As it seems that is usually some folder inside the software folder that is assumed to exist.
Here we have some problems:

1. IMHO we get an error because the symlink cannot be created as software/node/node_modules/npm since software/node/node_modules does not (yet) exist. So as a simple fix, we could first do mkdir -p on basedir before creating the symlink.
2. A general problem is that some tools do not follow the philosophy we expect in devonfw-ide that each tool itself should be pristine and have no side effects on other tools. Fode node based stuff this is not the case: When I install node version XYZ and then I install npm version ABC, then this npm version ABC is installed into node XYZ. By default this is fine as you only mess your own local devonfw-ide installation. However, when using DEVON_SOFTWARE_PATH the problem is that one devonfw-ide installation has side effects on the other if they both want different versions of npm for the same node version it will cause problems. I was already aware of this problem due to plugin installations so I documented a warning for this feature (see caution): https://github.com/devonfw/ide/blob/master/documentation/software.asciidoc#shared

[hohwille]

We should clarify why npm gets installed into node.
Maybe it can also "live" beside in its own folder what would solve the problem 1. easily.
However, I assume that something in node tooling might prevent this...

[hohwille]

My gut feeling was correct:

npm -v
node:internal/modules/cjs/loader:998
  throw err;
  ^

Error: Cannot find module '/Users/hohwille/projects/software/default/nodejs/v19.0.0/bin/node_modules/npm/bin/npm-cli.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:995:15)
    at Module._load (node:internal/modules/cjs/loader:841:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:82:12)
    at node:internal/main/run_main_module:23:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Node.js v19.0.0
Could not determine Node.js install directory

A trick might be a special handling here and if DEVON_SOFTWARE_PATH is defined, we would create relative symlinks to ../../npm inside node/node_modules/ folder.
However, this does not work (at least on MacOS) as nowadays symlinks are proceessed much more accurate so that even if you are logically in a path containing symlinks a relative path is interpolated relative to is physical location.

[hohwille]

So the only chance I see left here to fix this is to detect if DEVON_SOFTWARE_PATH is defined but our intallation path is already pointing inside a symlinked installation then we should behave as if DEVON_SOFTWARE_PATH was not defined for the current installation.