refactor: remove invalid string2code mappings

[lidel]

Problem: string2codec maps are a mess

The current state is not sustainable, string mappings are invalid or missing, and we don't have correct ones for core IPLD formats like dag-pb, dag-json and dag-cbor 🙈

Solution: cleanup

Problem 2: cleanup may introduce silent bugs into people's codebases

da55247 demonstrates the map changes we would have to do to restore sanity: fixes mapping, and adds IPLD formats along with non-dag generic variants for Protobuf, JSON and CBOR, so go-cid finally match the source of truth.

This approach would change the meaning of code 0x70 and 0x71 and strings cbor and protobuf:

• protobuf string now correctly points at code 0x50 (was 0x70, which is dag-pb)
  • 0x70 code is mapped to dag-pb (was protobuf before)
• cbor string now correctly points at code 0x51 (was 0x71, which is dag-cbor)
  • 0x71 code is now mapped to dag-cbor (was cbor before)

Pretty risky – people dont read release notes, this could cause silent errors in people's software.
Also, we have proper code2string mappings in go-multicodec (generated, and not written by hand)

Solution

Remove invalid/redundant string2code mappings from go-ipfs, point ecosystem at using go-multicodec directly.

BREAKING CHANGE

Implemented Approach B: Remove invalid mappings entirely 💚

I applied this in 6ca2617

We already have go-multicodec which is generated from the source of truth.

We remove hardcoded maps, then people will be forced to refactor their code to use go-multicodec (multiformats/go-multicodec#65) – forcing people to refactor makes it less likely they will just blindly bump the dependency without reading release notes.

---

Refactor guide

Converting code to string and vice versa will no longer be done with go-cid, go-multicodec should be used instead

Old way (go-cid)

cid "github.com/ipfs/go-cid"

// string → code
code := cid.Codecs["libp2p-key"]

// code → string
string := cid.CodecToStr[code]

New way (go-multicodec)

mc "github.com/multiformats/go-multicodec"

// string → code
var mc.Code code 
code.Set("libp2p-key")

// code → string
code.String()

[BigLep]

2022-03-24 conversation:

1. We want to delete the table. We generate this programmatically already in https://github.com/multiformats/go-multicodec

[lidel]

I went with Approach B (Remove mappings).
Filled multiformats/go-multicodec#65 to figure out the best way existing users could migrate, will ship this as soon we figure that out.

[lidel]

Based on multiformats/go-multicodec#65 (comment) I wrote migration guide (will include it in release notes).

This is now ready to merge as soon I get some reviews.

[mvdan]

SGTM with the understanding that we're okay with the breaking change. An alternative would be to have go-cid import the new go-multicodec, filling the global map at init time - it would avoid breaking users, but longer term it's a worse approach.

If you're breaking users, have you thought about removing the constants and pointing users towards go-multicodec directly? It might be a bit too much in terms of breakage, but if we're already replacing the from/to string APIs, replacing the constants as well would seem consistent.

[aschmahmann]

@mvdan the reason to break users with the maps but not the constants is because the maps are wrong. The most egregious issue is that "cbor" is mapped to dag-cbor (0x71) instead of to cbor (0x51). There's also the protobuf vs dag-pb issue, but that's less terrible. If we just fixed the maps then people would silently break which is IMO the worst of the options.

If you're breaking users, have you thought about removing the constants and pointing users towards go-multicodec directly?

Why would you choose to break users when you could just not at trivial cost? We could add Deprecated markers pointing people at go-multicodec though.

It might be a bit too much in terms of breakage, but if we're already replacing the from/to string APIs, replacing the constants as well would seem consistent.

With some basic searching:

• https://grep.app/search?current=11&q=cid.Raw&words=true has 110 results
• https://grep.app/search?q=cid.Codecs&words=true has 6
• https://grep.app/search?q=cid.CodecToStr&words=true has 11

So breaking the maps seems to cause much less trouble for people than also breaking the constants.

[mvdan]

ACK, I hadn't caught the detail about the bad map entries.

[aschmahmann]

LGTM. While generally, it'd be nice to see some PRs in the downstream breakages (e.g. go-ipfs) to see how this plays out I suspect it's fine here to merge. This repo doesn't get too many changes so it's fair to assume we won't make more before we deal with the breakages in go-ipfs and related repos and we can figure this out later.

Whether you want to tag now, before we have PRs resolving the breakages in the few effected projects is your call.

[lidel]

note to self: before I merge this, I'll test this in go-ipfs, see how painful refactor is etc
(I'll do that as part of review of ipfs/kubo#8568)

[lidel]

• Added deprecation notice in comment and updated README in 85198b9 (non functional changes, just docs)
• Identified impact (wip, this list may grow):
  • go-libp2p-core@v0.14.0/peer/peer.go:167:12: undefined: cid.CodecToStr
    • fix in fix: switch to go-multicodec mappings libp2p/go-libp2p-core#240

[BigLep]

2022-03-31 conversation: the work isn't blocked, but it won't get merged and released until the libp2p resource manager work for ipfs.

[lidel]

This is unblocked and ok to merge:

• Both resource manager (feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) kubo#8680) and multicodec cleanup in block put and cid codecs commands (ipfs get -o file $address fails (minor argument parse bug) kubo#856 landed in go-ipfs master.
• Improved README landed in docs(readme): improved usage examples multiformats/go-multicodec#66

TLDR: if you've found this PR because you need to map CODE to STRING or STRING TO CODE, see usage https://github.com/multiformats/go-multicodec#usage

[github-actions]

Suggested version: v0.2.0
Comparing to: v0.1.0 (diff)

Changes in go.mod file(s):

diff --git a/go.mod b/go.mod
index e307ff9..6e3b580 100644
--- a/go.mod
+++ b/go.mod
@@ -4,7 +4,17 @@ require (
 	github.com/multiformats/go-multibase v0.0.3
 	github.com/multiformats/go-multihash v0.0.15
 	github.com/multiformats/go-varint v0.0.6
+)
+
+require (
+	github.com/klauspost/cpuid/v2 v2.0.4 // indirect
+	github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1 // indirect
+	github.com/minio/sha256-simd v1.0.0 // indirect
+	github.com/mr-tron/base58 v1.2.0 // indirect
+	github.com/multiformats/go-base32 v0.0.3 // indirect
+	github.com/multiformats/go-base36 v0.1.0 // indirect
 	golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf // indirect
+	golang.org/x/sys v0.0.0-20210309074719-68d13333faf2 // indirect
 )
 
-go 1.15
+go 1.17

gorelease says:

# github.com/ipfs/go-cid
## incompatible changes
CodecToStr: removed
Codecs: removed
## compatible changes
DagJSON: added

# summary
Suggested version: v0.2.0

gocompat says:

(empty)