spec: Repo

[daviddias]

As notes by:

• IPFS Repo Spec update #43

We need to update the Repo Spec, this includes:

• Discrepancies to address:
  • fs-repo: version format
  • fs-repo: pid in repo.lock
  • fs-repo: ipfs repo fsck to fix lock issues.
  • fs-repo: blocks dir layout (fanout/depth)
  • fs-repo: come up with a name for api-file-only repo

[daviddias]

Notes from the discussion with @dignifiedquire on March 15, 2017

Today, go-ipfs uses CID as keys for blockstore (the key that is base32 encoded and used as a path to store on disk).

In August 2016, during the sign of the "Lisbon treaty" (aka the birth of CID), we questioned ourselves if we should use CIDs as keys, where the tradeoff was:

• With CID, we would know the type of the objects stored, if they use CIDv1
• With CID, if two applications use the same block, but one treats the block as raw and the other as a format, then the dedup factor would be 0.

At the time, the feeling was torwards not using CID, but that changed during the development and it is what lives on go-ipfs master. The js-ipfs crowd will follow.

To mitigate the issue of dedupping, we might consider having a tool that tried to dedup by checking every block, trying different hash functions (same issue) and comparing things without the codec. This is an idea, not a wip.