Hashing refactor - meta + timeline

[zelig]

Storage Refactor Part II

Part II: Hashing refactor

Part I : #1027

Phase 0 - prerequisites

• swarm hash
  • BMT API change
  • Filehasher - swarm/storage: Filehasher prototypes #969
  • remove

Phase 1 - functionality

• hasher extensions
  • storage/db
  • encryption
• remove

Phase 2 - itegration

• filehasher integration
  • tags
  • BMT API change
  • Filehasher
  • remove

Phase 3 - optimisation

• benchmarks
  • chunk size optimisations
• refactor
  • hash package
  • (multihash - Enable multihash support for swarm root hashes & ENS #186)
  • bmt, filehasher
  • hashing service using pool - swarm/hash: Centralize hasher functionality #634
  • remove chunker, pyramid, hasherstore
• erasure coding

Phase 4 - optimisation - extensions

likely emerge as new projects:

• CRS per level erasure coding
  • backend - wrapper around CRS lib
  • split - CRS encode
  • retrieve - CRS decode
  • fallback mode on lazy chunk reader
  • race mode using
  • benchmark - simulations with network throughput and dropout models
• crash POC
  • crash calculation as hasher extension