Libp2p endpoint that lists supported transport protocols

[dirkmc]

Goals

Storage Providers now support protocols beyond libp2p for retrieval. It should be possible for a client to query a SP to ask which protocols it supports, and what is the endpoint for the protocol.

Description

Expose a new libp2p endpoint that returns supported protocols and the endpoint for the protocol, eg

Libp2p: /ip4/1.2.3.4/tcp/1234/12D3KooWE8yt84RVwW3sFcd6WMjbUdWrZer2YtT4dmtj3dHdahSZ
Http: /dns4/foo.com/tcp/80/http

Endpoint

The endpoint is at /fil/retrieval/transports

Note that an SP typically listens on these libp2p endpoints:

  /fil/datatransfer/1.2.0
  /fil/kad/testnetnet/kad/1.0.0
  /fil/retrieval/qry/0.0.1
  /fil/retrieval/qry/1.0.0
  /fil/storage/ask/1.0.1
  /fil/storage/ask/1.1.0
  /fil/storage/mk/1.0.1
  /fil/storage/mk/1.1.0
  /fil/storage/mk/1.1.1
  /fil/storage/mk/1.2.0
  /fil/storage/status/1.0.1
  /fil/storage/status/1.1.0
  /fil/storage/status/1.2.0
  /floodsub/1.0.0
  ...

Schema

type Protocol struct {
  # The name of the protocol eg "libp2p" or "http"
  Name string
  # The address of the endpoint in multiaddr format
  Endpoint Multiaddr
}

type QueryResponse struct {
  Protocols [Protocol]
}

[willscott]

can endpoint be a multiaddr? we have ways to encode http endpoints as multiaddrs e.g. https://github.com/filecoin-project/go-legs/blob/main/httpsync/multiaddr/convert.go

[dirkmc]

I'm not sure there's an advantage to forcing multiaddr encoding - if the endpoint for HTTP is a multiaddr you will just need to convert it to an HTTP URL to use it (and in future there may be transports that aren't encodable as multiaddr)

[willscott]

Philosophically it's about using our stack where it doesn't hurt us to do so. this seems like a place where that's true, and it means we could add additional metadata into multiaddr clients without worrying about breaking parsers.

[dirkmc]

we could add additional metadata into multiaddr clients

Could you expand on that, what kind of metadata are you thinking about?

The counter-case is that if in future we add support for retrieval protocols that are not supported by multiaddr then we will have clients in the wild who will error out when reading the response from this endpoint.

[willscott]

I get that everything serializes to strings, but multiaddrs are supposed to be the extensible type for this.
I'm having trouble imagining a retrieval protocol we would support that we don't make a multiaddr encoding for - we'd need to if we were indicating it through the indexer system, which is using multiaddrs as the endpoint type.

[willscott]

things i could imagine wanting to tag, which could be done in a backwards-compatible way by adding them as extra segments in the multiaddrs include:

• relative preferences of different endpoints
• geographic location tagging of where replicas are
• support for e.g. http resumption

[dirkmc]

Ok seems reasonable 👍
I changed the type for Endpoint to Multiaddr