From bf23e48e19121c29bc48313d574fd0809acb88e0 Mon Sep 17 00:00:00 2001 From: Alex Potsides Date: Wed, 28 Jul 2021 17:18:13 +0100 Subject: [PATCH] docs: update ipld/multiformats docs (#3771) Improve docs around how to configure custom block codecs/multibase codecs, etc. --- docs/IPLD.md | 111 +++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 86 insertions(+), 25 deletions(-) diff --git a/docs/IPLD.md b/docs/IPLD.md index 28bb5f6e6e..7b262567ef 100644 --- a/docs/IPLD.md +++ b/docs/IPLD.md @@ -3,29 +3,46 @@ ## Table of Contents - [Overview](#overview) -- [Bundled Codecs](#bundled-codecs) -- [Adding additional codecs](#adding-additional-codecs) +- [Bundled BlockCodecs](#bundled-blockcodecs) +- [Bundled Multihashes](#bundled-multihashes) +- [Bundled Multibases](#bundled-multibases) +- [Adding additional BlockCodecs, Multihashes and Multibases](#adding-additional-blockcodecs-multihashes-and-multibases) - [Next steps](#next-steps) ## Overview -The IPFS repo contains a blockstore that holds [Blocks](https://github.com/ipld/js-ipld-block). These blocks can be thought of as a [CID][] and associated byte array. +The IPFS repo contains a blockstore that holds the data that makes up the files on the IPFS network. These blocks can be thought of as a [CID][] and associated byte array. -The [CID][] contains a `codec` property that lets us know how to interpret the byte array associated with it. +The [CID][] contains a `code` property that lets us know how to interpret the byte array associated with it. -In order to perform that interpretation, an [IPLD Format][] must be loaded that corresponds to the `codec` property of the [CID][]. +In order to perform that interpretation, a [BlockCodec][] must be loaded that corresponds to the `code` property of the [CID][]. -## Bundled Codecs +Similarly implementations of [Multihash][]es or [Multibase][]s must be available to be used. -js-IPFS ships with three bundled codecs, the ones that are required to create and interpret [UnixFS][] structures. +## Bundled BlockCodecs + +js-IPFS ships with four bundled codecs, the ones that are required to create and interpret [UnixFS][] structures. These are: -1. [ipld-dag-pb](https://github.com/ipld/js-ipld-dag-pb) - used for file and directory structures -2. [ipld-raw](https://github.com/ipld/js-ipld-raw) - used for file data where imported with `raw-leaves=true` -3. [ipld-dag-cbor](https://github.com/ipld/js-ipld-dag-cbor) - used for general storage of JavaScript Objects +1. [@ipld/dag-pb](https://github.com/ipld/js-dag-pb) - used for file and directory structures +2. [raw](https://github.com/multiformats/js-multiformats/blob/master/src/codecs/raw.js) - used for file data where imported with `--raw-leaves=true` +3. [@ipld/dag-cbor](https://github.com/ipld/js-dag-cbor) - used for storage of JavaScript Objects with [CID] links to other blocks +4. [json](https://github.com/multiformats/js-multiformats/blob/master/src/codecs/json.js) - used for storage of plain JavaScript Objects + +## Bundled Multihashes + +js-IPFS ships with all multihashes [exported by js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/src/hashes), including `sha2-256` and others. + +Additional hashers can be configured using the `hashers` config property. + +## Bundled Multibases + +js-IPFS ships with all multibases [exported by js-multiformats](https://github.com/multiformats/js-multiformats/tree/master/src/bases), including `base58btc`, `base32` and others. -## Adding additional codecs +Additional bases can be configured using the `bases` config property. + +## Adding additional BlockCodecs, Multihashes and Multibases If your application requires support for extra codecs, you can configure them as follows: @@ -35,14 +52,34 @@ If your application requires support for extra codecs, you can configure them as const node = await ipfs({ ipld: { - // either specify them as part of the `formats` list - formats: [ - require('my-format') + // either specify BlockCodecs as part of the `codecs` list + codecs: [ + require('custom-blockcodec') ], - // or supply a function to load them dynamically - loadFormat: async (format) => { - return require(format) + // and/or supply a function to load them dynamically + loadCodec: async (codecNameOrCode) => { + return require(codecNameOrCode) + }, + + // either specify Multibase codecs as part of the `bases` list + bases: [ + require('custom-multibase') + ], + + // and/or supply a function to load them dynamically + loadBase: async (baseNameOrCode) => { + return require(baseNameOrCode) + }, + + // either specify Multihash hashers as part of the `hashers` list + hashers: [ + require('custom-multibase') + ], + + // and/or supply a function to load them dynamically + loadHasher: async (hashNameOrCode) => { + return require(hashNameOrCode) } } }) @@ -53,14 +90,34 @@ If your application requires support for extra codecs, you can configure them as const client = ipfsHttpClient({ url: 'http://127.0.0.1:5002', ipld: { - // either specify them as part of the `formats` list - formats: [ - require('my-format') + // either specify BlockCodecs as part of the `codecs` list + codecs: [ + require('custom-blockcodec') + ], + + // and/or supply a function to load them dynamically + loadCodec: async (codecNameOrCode) => { + return require(codecNameOrCode) + }, + + // either specify Multibase codecs as part of the `bases` list + bases: [ + require('custom-multibase') ], - // or supply a function to load them dynamically - loadFormat: async (format) => { - return require(format) + // and/or supply a function to load them dynamically + loadBase: async (baseNameOrCode) => { + return require(baseNameOrCode) + }, + + // either specify Multihash hashers as part of the `hashers` list + hashers: [ + require('custom-multibase') + ], + + // and/or supply a function to load them dynamically + loadHasher: async (hashNameOrCode) => { + return require(hashNameOrCode) } } }) @@ -69,7 +126,11 @@ If your application requires support for extra codecs, you can configure them as ## Next steps * See [examples/custom-ipld-formats](https://github.com/ipfs/js-ipfs/tree/master/examples/custom-ipld-formats) for runnable code that demonstrates the above with in-process IPFS nodes, IPFS run as a daemon and also the http client +* Also [examples/traverse-ipld-graphs](https://github.com/ipfs/js-ipfs/tree/master/examples/traverse-ipld-graphs) which uses the [ipld-format-to-blockcodec](https://www.npmjs.com/package/ipld-format-to-blockcodec) module to use older [IPLD format][]s that have not been ported over to the new [BlockCodec][] interface, as well as additional [Multihash Hashers](https://www.npmjs.com/package/multiformats#multihash-hashers). -[cid]: https://www.npmjs.com/package/cids -[ipld format]: https://github.com/ipld/interface-ipld-format +[cid]: https://docs.ipfs.io/concepts/content-addressing/ +[blockcodec]: https://www.npmjs.com/package/multiformats#multicodec-encoders--decoders--codecs [unixfs]: https://github.com/ipfs/specs/blob/master/UNIXFS.md +[ipld format]: https://github.com/ipld/interface-ipld-format +[multihash]: https://github.com/multiformats/multihash +[multibase]: https://github.com/multiformats/multibase \ No newline at end of file