git dependencies with #semver: don't seem to resolve and dedup correctly

[reshadi]

I'm opening this issue because:

• npm is crashing.
• npm is producing an incorrect install.
• [?] npm is doing something I don't understand.
• Other (see below for feature requests):

What's going wrong?

If you use git #semver: in two different projects, they won't dedup unless the versions specified in the #semver: is exactly the same.

How can the CLI team reproduce the problem?

Take a look a this example I setup:
There are three small projects (p1, p2, p3).
p1 is on version 1.0.2 and has a git tag "v1.0.2" point to that version.

p2 depends on p1 using the following:
"p1": "git+https://github.com/reshadi/p1.git#semver:^1.0.0"

p3 depends on p1 and p2 using the following:
"p1": "git+https://github.com/reshadi/p1.git#semver:^1.0.1",
"p2": "git+https://github.com/reshadi/p2.git#semver:^1.0.0"

if you clone p3 (https://github.com/reshadi/p3) and call "npm install", you will see that p1@1.0.2 is repeated twice. That means both "p1.git#semver:^1.0.1" and "p1.git#semver:^1.0.0" correctly resolve to the "v1.0.2" tag of p1, but p1 is duplicated in the tree. Even calling "npm dedup" does not remove it.

However, if p3's package.json references p1 exactly same way as p2 using the following, p1 is dedupped properly:
"p1": "git+https://github.com/reshadi/p1.git#semver:^1.0.0",

seems like semantic version is not properly applied to these dependencies.

supporting information:

• npm -v prints: 5.7.1
• node -v prints: v9.8.0
• npm config get registry prints: https://registry.npmjs.org/
• Windows, OS X/macOS, or Linux?: both Windows and Linux
• Network issues:
  • Geographic location where npm was run:
  • I use a proxy to connect to the npm registry.
  • I use a proxy to connect to the web.
  • I use a proxy when downloading Git repos.
  • I access the npm registry via a VPN
  • I don't use a proxy, but have limited or unreliable internet access.
• Container:
  • I develop using Vagrant on Windows.
  • I develop using Vagrant on OS X or Linux.
  • I develop / deploy using Docker.
  • I deploy to a PaaS (Triton, Heroku).

[mattgaspar]

I have been encountering this problem as well. But in my case it happens randomly even when semver is the same. I wasn't able to reproduce consistently but it has happened multiple times and caused major headaches. Has happened both after doing npm update and new installs.
In my case, aurelia needed the dependency to be a singleton and when 2 separate instances were imported it broke parts of the application (aurelia-router).

webpack also has an open issue that sounds like it's being caused by this issue in npm
webpack/webpack#5593