-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
package-lock.json (missing resolved/integrity) not noticed and not repaired automatically by npm #4460
Comments
Warning due to missing integrity might be a good idea, but requires some discussion. This would be a great thing to bring up at an RFC meeting since this would be new behavior. Additionally, We don't currently consider a lack of integrity to be a "broken" package-lock.json, as there are several reasons that a package might not have an integrity field. If you want to update the package-lock.json, @DavHau Thanks for filing this! Since this is working as intended, can you please open an RFC issue with some details on how you could/would like to see this changed? |
I brought this up in today's RFC meeting, and the consensus was that we do want to change this behavior. Actions:
This means that we'll get warnings from local file, tarball, and git references, as well as packages from registries that do not include an integrity. It also means that the package-lock.json will mutate upon npm-install even if a spec is not included. |
It's good to see that the RFC team wants to move forward on improving this.
If the reason integrity hashes are stored within the lock file, is to ensure reproducibility and security, then any mismatch must be treated as a critical error. If it's not, then what is the point of storing integrity hashes in the first place? EDIT: OK, I just noticed that NPM actually does trigger an error on a integrity mismatch, if the package isn't cached. That's good. But as said, I think a missing integrity should be treated exactly the same (except if there is a good reason why integrity isn't needed, like with local/symlinked modules etc.) |
Why? If there’s a missing integrity value, you wouldn’t want every install to suddenly start failing. It’s about maximal reproducibility, not guaranteed reproducibility. |
If you really care about security and reproducibility, you'd want exactly that. Because it protects you from threat/breakages and it gives you the chance to fix it. AFAIU, the lock file can reference arbitrary tarball URLs and git repos. Integrity checking not being enforced results in a lot of potential attack surface, and things that can go wrong accidentally. In case you want to ignore the error and continue anyways, you could have a flag like
Your idea of |
It’s as much as is possible, without breaking the common case. If you care that much about security, how did a non-registry dep or a missing integrity value get committed in the first place? |
Note that there's another layer here, where:
This whole area is a real surprise. What is the point of these half-guarantees and false sense of [literal] security? I've been operating as if NPM had my back on this stuff, and it's really jarring to realize that not only does it not have my back, but maintainers seem to be insisting that I shouldn't have my back, and I should... have my own? (I presume the disjunct is for some reason I don't quite grok, and so I'm trying to have good-faith that there's some meta-consideration that makes this make sense for the ecosystem as a whole) Anyhow, I appreciate you all, but just a little baffled (but also trying to be humble about my bafflement) ❤️ |
@patcon to be clear, if you do intentionally break the hash such that it no longer matches the package being installed, i would definitely expect |
Ah ok, so maybe this is another issue. Sorry to conflate. |
In case helpful, sharing our encounter here: GCTC-NTGC/gc-digital-talent#2414 (comment) Can you confirm that the Alt Reproduction described there is unexpected for NPM v8.3.0? If so, should this be a new issue, or can we continue discussing here? Thanks! |
@patcon can you reproduce it with npm v8.6? testing in "not the latest" npm doesn't necessarily help :-) |
EDIT2: Bah. What am I saying. This isn't hard. Sorry, my brain was stubbornly staying on my task, but I'll try 8.6.0 🙃 |
ok, finito. (Ha!) node 14.18.1 (stuck on this) both scenarios still hold -- both malformed integrity hashes and well-formed but unmatching hashes run without failure in |
Honestly, I am not surprised about this. My default behaviour is to install yarn and use it instead, and set as organisation policy to not to use npm, since it is guaranteed instanity. just stop using it. |
My 2 cents for me too ;)
Why this point of view? |
I appear to have hit this issue running the latest NPM version (8.13.1) - caused by a developer manually updating the versions in both package.json and package-lock.json without updating the integrity value.
-> at this point npm ci runs without complaint.
Failed:
We found this because only completely new starters were hitting the EINTEGRITY error because they did not have any files cached but as soon as you have the cache, NPM no longer respects the integrity check. I would suggest that this is potentially exploitable if someone can intercept the registry and place a malicious package in its place as the package can now have any signature and be of any size since it is not validated. |
On 8.12.1, I deleted package-lock and ran npm install in order to remove a stubborn peer dependency, and it generated a new lockfile without any The only way to get back a proper lockfile was to delete both node_modules and package-lock before doing npm install. The default should be to add the hashes, not to remove them. I imagine there are now a lot of projects with lockfiles that have no integrity hashes, without people being aware of it. Surely this must be avoided? |
npm cache clean --force doesn't cut it for me I need all the integrity and resolved filelds, becuase of cache-only policy, so what I need to do is to npm install package-missing-integrity-fields --save |
I'm also seeing this and I can't figure out why npm silently deletes If anyone has tips on how to restore |
@DavHau a missing integrity field MUST NOT be treated the same as a integrity mismatch. reproducible build means: This use case is similar to the one from @ath0mas but not the same! but none the less should NPM recreate missing resolved/integrity fields it finds and issue a warning (no error) about it every time it find such a dependency. |
Is there a published version of npm with this behavior? If not are there plans of implementing this? |
Originally these fields were missing due to outstanding issue with npm: - npm/cli#4460 - npm/cli#6301
Originally these fields were missing due to outstanding issue with npm: - npm/cli#4460 - npm/cli#6301
Originally these fields were missing due to outstanding issue with npm: - npm/cli#4460 - npm/cli#6301
Due to npm bug: npm/cli#4460 Updated using npm-lockfile-fix: https://github.com/jeslie0/npm-lockfile-fix
Due to npm bug: npm/cli#4460 Updated using npm-lockfile-fix: https://github.com/jeslie0/npm-lockfile-fix
Due to npm bug: npm/cli#4460 Updated using npm-lockfile-fix: https://github.com/jeslie0/npm-lockfile-fix
For anyone like me who is looking for a workaround to back-fill these missing fields, I published a small CLI to the npm registry you can use:
This will fill in any missing |
Is there an existing issue for this?
This issue exists in the latest npm version
Current Behavior
In order to assure reproducible installations, every package listed in a
package-lock.json
which is fetched from a registry should contain aresolved
andintegrity
field.This doesn't seem to be the case looking at some existing
lockfileVersion = 2
based file.See for example: https://raw.githubusercontent.com/directus/directus/2938821be05eaf195872c34eed709ac9b4a430b4/package-lock.json
Inspecting the entries for
camelcase@6.2.0
(and many others), neitherresolved
norintegrity
exist.Checking out the repository and executing
npm install
happily installscamelcase@6.2.0
while:To fix the lock file, one currently has to:
node_modules
directoriespackage-lock.json
filenpm install
Expected Behavior
(the problem should be of equivalent importance than a mismatching integrity)
(not necessarily automatically, but the user should be informed about the problem and instructed on how to fix it)
Steps To Reproduce
lock file is still broken (check entry
camelcase@6.2.0
)Environment
The text was updated successfully, but these errors were encountered: