IPIP-337: Delegated Content Routing HTTP API

[guseggert]

No description provided.

[ajnavarro]

👏 👏 🎉

The main thing I'm missing is type specification apart from the JSON example:

• How CIDs are encoded
• How PeerIDs are encoded
• How IPNS IDs are encoded
• And so on.

Also, I'm missing some limitations on results and payloads.

[lidel]

We should land GET spec when Kubo 0.18 ships with the initial implementation.

@guseggert will you have time to move PUTs to a separate IPIP created on top of this one?
It will enable us to land GETs and continue work on PUTs in less noisy PR.

[lidel]

This IPIP mostly LGTM, reflects GET-only API used in Kubo 0.18 and cid.contact.
Some things are still not implemented by anybody, asked for clarifications below.

Let's wait with merge until IPFS Implementers Sync this Thursday.
If there are no concerns raised during the sync, and we decide what to do with things that are not implemented yet, we will merge after the Thursday call.

[lidel]

Just want to flag that transfer parameter is a hardcoded list of pre-vetted strings, which is making future changes extremely difficult:

• it will be impossible to query for providers that expose novel third-party protocols
• every new protocol needs to be manually added to the spec here, then to libraries, and then rolled out to indexers and clients

Feels like gatekeeping system which makes it impossible to experiment with novel transfer protocols.

Last year I suggested to not invent own strings, avoid transport protocol codes from table.csv.

Since transfer is not implemented yet, I suggest removing it, or adding escape hatch for third-party protocols not specified in this document.

If we want to keep it, need to ensure we are not gatekeeping:

Option A

Ensure clients can talk the same version of bitswap by reusing strings from libp2p identify's protocols list:

$ ipfs id | jq .Protocols
[
  "/ipfs/bitswap",
  "/ipfs/bitswap/1.0.0",
  "/ipfs/bitswap/1.1.0",
  "/ipfs/bitswap/1.2.0",
  "/libp2p/fetch/0.0.1",
  ...
]

We could update spec and say that every value that starts with / should be interpreted as libp2p protocol prefix:

transfer=/ipfs/bitswap would return all peers that speak any version of bitswap.

Option B

Alternative, is to just accept number, and allow people to use codes from reserved private range

[aschmahmann]

IMO using the numbers from the global table feels pretty bad for experimentation as well. The existing IPNI implementation doesn't support them and they require some gatekeeping anyway. For the time being I've put up ipni/specs#6.

It seems like we could reasonably integrate this with something like the / proposal.

transfer=/ipfs/bitswap would return all peers that speak any version of bitswap.

Unfortunately, this type of query isn't supported in IPNI today and if it did it'd probably be with some specific semantics (e.g. adding a field to the "BitswapMetadata" section that's a list of protocolIDs) or a similar thing for GraphSync, unless they're going to reserve a new number every time they modify the protocol).

If we started leveraging named-record in addition to (or instead of) numbers in the global table we could just query against those. Basically, this would mean that the query could be fulfilled using custom logic per-number, or just using the named-record wrapper.

Since transfer is not implemented yet, I suggest removing it

Since the general policy is not to merge specs without implementations we should remove and then re-add later with the implementation, right?

[masih]

experimentation
require some gatekeeping

IPNI imposes no restriction on the protocol ID in metadata; this can be any number and treated as arbitrary bytes by the indexers.

Unfortunately, this type of query isn't supported in IPNI today

Happy to capture an issue on this if this is needed?

if it did it'd probably be with some specific semantics

It'd probably be varint prefix matching.

If we started leveraging named-record in addition to (or instead of) numbers in the global table we could just query against those

i'm sorry i am struggling to see how using names instead of numbers would make a difference in this case. Surely we can do the same with numbers?

[guseggert]

I think the consensus is to remove this, but the idea was that these are treated by this spec as opaque strings used for filtering, and the servers have no requirement to enforce any particular values (unless they want/need to). But happy to have this conversation another day, since we never implemented this anyway.

[lidel]

As agreed a while ago, this is GETs-only that shipped in Kubo 0.18.
We've ratified it a while ago, but stewards had no bandwidth to clean up and merge until now.

I made small editorial changes, filled some gaps, and removed left-overs from removed things (pagination).
Unanswered questions are moved to #377

Merging to unblock follow-up IPIPs @masih and @guseggert want to submit (streaming, PUTs) and future work (peer records, IPNS records, privacy-preserving double hashing etc).