npm install issues on latest master (Seaport support)

[mohoff]

Hi, my package.json includes:

"opensea-js-seaport": "git+https://github.com/ProjectOpenSea/opensea-js.git#2e2d779cde7fcdda6298846de6b5f8dd32df8294",
"opensea-js": "3.0.2",

The 1st line has the latest master commit with Seaport support. The latter is the soon-legacy SDK which installs fine.

On npm install I'm getting an integrity check error:

14271 verbose node v16.11.0
14272 verbose npm  v6.14.15
14273 error code EINTEGRITY
14274 error Verification failed while extracting wyvern-js@git+ssh://git@github.com/ProjectOpenSea/wyvern-js.git#f7704ad2571f05136f9a42735966dcd3f1485474:
14274 error Verification failed while extracting wyvern-js@git+ssh://git@github.com/ProjectOpenSea/wyvern-js.git#f7704ad2571f05136f9a42735966dcd3f1485474:
14274 error sha512-KmmtiSg01F5PVtgl/nOhU7Lz2oSPGhv7lKSlqlbtUZCqa2JhvTnl0UVlzKAmTPDv/sYe6r8ol2e6pMg0paHObw== integrity checksum failed when using sha512: wanted sha512-KmmtiSg01F5PVtgl/nOhU7Lz2oSPGhv7lKSlqlbtUZCqa2JhvTnl0UVlzKAmTPDv/sYe6r8ol2e6pMg0paHObw== but got sha512-Wo9M/6+z/NyyR9jXVmwtSBDGPzJ/AeWEUrQuh1uBAc/YrFUOrm1dHL3MrY57yidzCnSaPoezW+p6SL6FRtMLrA==. (849531 bytes)
14275 verbose exit [ 1, true ]

I know the node+npm version combo is weird but after trying different setups between npm@6,7,8 and node@12,16 this got me the furthest. The integrity sha512 of the log is not part of the lock file so can't delete that follwing the advice in https://stackoverflow.com/a/65983202. Using yarn is not an option in my case.

Thanks in advance! Excited for Seaport!

[marcelltoth]

These git dependencies did bite me, too, even using Yarn (3). Any chance to replace them with normal regular deps that "just work"? 🙏 Or even bundle them if you need to.

[esteban-OpenSea]

Thanks for bringing this up, my suspicion is that the package-lock.json file might need to be updated. But just want to confirm, yarn didn't work for you either?

[marcelltoth]

@esteban-OpenSea I keep reading this argument when someone brings up the installation difficulties, but I don't see why.

If you release a well-conforming package, any package manager should be able to install it. You can't make someone swap out a major piece of their stack, just because your package doesn't work well in some cases. It's like saying "then use Firefox" when someone reports your site is broken in Chrome.

If you ever worked on a large scale JS project (let's say a monorepo), you know that it may take a lot of effort to make the switch, even for testing purposes.

What do you mean by: "my suspicion is that the package-lock.json file might need to be updated." ? NPM keeps package-lock up to date by default, there's normally nothing manual needing to be done there. If you are going to suggest deleting it and installing from scratch, please don't do that: it's a nuke, it should be never needed, and on larger projects no-one will ever accept a PR that does so (and packs a 20k-line change in the lockfile)

I tell you what's wrong:

When you are producing a library, you want to make sure it's compatible with as much of the ecosystem as possible. This involves some things:

• In general it's a bad idea to rely on non-standard or non-battle tested features, like:
  • git references
    
  • even worse: mutable git referrences
    
  • HTTP tgz references:
    
  • Reliance on lifecycle scripts
    
  • Those lifecycle scripts then make a lot of assumptions of the surrounding environment (package manager, OS, shell)
  • Submodules (again, combined with git references, so behavior depends on the consumer's system)
    
• In general you want your dependency constraints as wide as you can, so you don't cause duplicate dependencies (which lead to bundle size bloat and conflicts) for your consumers. Currently this is a nightmare, for example:
  • opensea-js@latest -> wyvern-js@7823dfdf5a272ebbc6a46e66d23563a9d6cc1be2 -> ethers@^4.0.49
  • opensea-js@latest -> ethers@^5.6.6
  • opensea-js@latest -> seaport-js@1.0.1 -> ethers@5.6.7 (note the fixed version here)
• Event wyvern-js is duplicated...
  • opensea-js@latest -> wyvern-js@7823dfdf5a272ebbc6a46e66d23563a9d6cc1be2
  • opensea-js@latest -> wyvern-schemas@0a8d569931ddb6faa6e96f5a60fa2f83f0a8750e -> wyvern-js@f7704ad2571f05136f9a42735966dcd3f1485474
• etc.

If you are interested I can perhaps give more guidance in the form of a consultancy session (or similar), but either way, these packages need a lot of cleanup before they are reliable for production usage. I'm currently considering cutting it from my project (and reimplementing required functionality by hand) as it's making it totally unusable with the size bloat and installation flakiness: Since I installed my yarn install runs in CI fail about 40% of the time, with unchanging code. And I see multiple other folks reporting similar problems, too. Of course it may very well be all of us screwing up at the same time, but maybe the system is not as stable as we'd like it to be either ;)

[bmayen]

@marcelltoth, you are 100% correct. Unfortunately, all of these issues have been repeatedly brought up over many issues in this repo and have largely all been closed without being addressed. The project is poorly maintained, which isn't a necessarily a problem in and of itself, but consistent feedback and offers to help go ignored, and that certainly is. We've come to the similar conclusion, at great personal financial expense, that this project's definition of "production ready" is not remotely aligned with ours and it is all but unusable.

[jtaaa]

@mohoff if possible can you try upgrading to npm greater than version 8.5.2. As of that version npm will no longer check the integrity hash. Here's a related issue in npm repo for reference
npm/cli#2846 (comment)

[volkyeth]

For anyone using yarn in need of a workaround:
Add this to your package.json (if you're using workspaces, add to the root package.json):

 "resolutions": {
    "wyvern-schemas": "https://github.com/ProjectOpenSea/wyvern-schemas.git#commit=8ef52c7a4c0b8647bc8ae2ceee3f05ae7faa83cf"
  },

This will force the resolution of a different version of wyvern-schemas than the one required by opensea-js (which is broken because of a checksum mismatch).
The commit I picked is for the latest available tag on https://github.com/ProjectOpenSea/wyvern-schemas/tags as of now (v0.15.1)

[mohoff]

Ended up upgrading to latest npm@8.13.1 (thanks @jtaaa) and installed yarn just to make some prepublish step work. This made the installation work for me.

Since this thread mentions solutions for both npm and yarn, I'm closing. Thanks

[jayaddison]

Although this was resolved, for the record: as I understand it, npm's package integrity values are calculated by collecting the source content, compressing it using zlib, and then calculating a crytographic hash of the results.

The problem is the compression step -- because zlib may produce different results on different architectures and platforms.

So, despite the fact that a git checkout from a fixed commit should contain the same contents when retrieved on two systems, in the context of NPM the expected integrity value for that content may be different based on the machine where npm install / npm clean-install has run.

This isn't a problem for installations of packages from the central NPM package repository, because the file contents and integrity hash were calculated on a fixed machine once, and the same file content should be delivered to whichever machine installs it -- that machine does not have to re-compress any content.

As of NPM v8.5.2, installation of dependencies from git no longer verifies the integrity value.

And an aside:

Being able to install and verify packages reliably from git commit references would be convenient; in some ways it could be considered stronger in terms of integrity guarantees than installing from a package version number (where the originating commit ref isn't always clear -- or indeed could even be modified over time, as can occur for unsigned git tags).

Version numbers are certainly more convenient for human-readability (including the value provided by various versioning schemes like SemVer, CalVer, etc), but ultimately if+when anyone wants to inspect the precise changes between two package versions, the common denominator is a pair of commits.