ipfs-npm = registry-mirror.next()

[daviddias]

WIP

• new UX design
• get ipfs-blob-store with all the latest IPFS update dependencies successfuly ipfs-blob-store#17
• features
  • ipnpm registry clone
    • core functionality
    • spawn a daemon if none is available
    • option: seq-number
    • option: flush
    • option: ipfs
    • option: log
    • bug Getting a lot of ECONNRESET when calling files mkdir exhaustively ipfs/kubo#2918
  • ipnpm registry index publish
    • core functionality
    • spawn a daemon if none is available
    • option: ipfs
  • ipnpm registry index fetch
    • core functionality
    • spawn a daemon if none is available
    • option: ipfs
    • option: name
  • ipnpm daemon
    • core functionality
    • spawn a daemon if none is available
    • option: ipfs
  • ipnpm install
    • core functionality
    • spawn a daemon if none is available
    • option: ipfs
• other dependencies
  • new version of ipfsd-ctl using bin-wrapper Upgrade js-ipfsd-ctl to use bin-wrapper ipfs/js-ipfsd-ctl#78

[daviddias]

@dignifiedquire could you write a status update on what you found so far?

[dignifiedquire]

Current findings: I came close twice to finish a full clone of the registry and ipfs blew up underneath me. The first time I hadn't flushed at all, the second time I did flush but still after the crash and restart files ls / was returning nothingness. I am currently waiting for @whyrusleeping to investigate what the failure was.

[dignifiedquire]

So with the latest multicore updates the cloning process was speed up nicely such that we have now a full clone inside IPFS.

I just published it here:

> ipfs name publish /ipfs/QmWtyxBrThZzZb5m9bdM9NN3WF6m4Vk85AukjzeQmhgnu6
Published to QmcmzV8FXsEfF5bQFf8vyJxhBZcV2SxTBApRcQbP5BQYGz: /ipfs/QmWtyxBrThZzZb5m9bdM9NN3WF6m4Vk85AukjzeQmhgnu6

The current repo size is about 1.7T but the actual size is probably lower, given that there are things from previous attempts in there.

Pinning this root hash does destroy the daemon currently.

[dignifiedquire]

> ipfs files ls /npm-registry |wc -l
208677

[daviddias]

@dignifiedquire what is the status on this? Last time I heard was that modules were not being properly stored anymore. I know you changed the cloner to a multicore one, might be because of that?

[dignifiedquire]

The status is, I started to write a fully tested cloner here: https://github.com/dignifiedquire/registry-cloner but I will need some time to finish it.