feat: add support for arbitrary service modules

[saul-jb]

It would be nice if libp2p support arbitrary modules so that they can automatically be initialized with the libp2p instance so this PR adds this functionality. This is useful because there are many modules one might want to use that do not fit in the predefined categories and it allows them to add those modules alongside the standard ones.

This PR adds a configuration option to the createLibp2p method allowing you to add custom modules:

const node = await createLibp2p({
  // Normal libp2p modules
  // ...

  // Custom libp2p modules
  modules: {
    myModule: createMyModule(options)
  }
})

// Use 'myModule' later on.
node.modules.mymodule.doSomething();

[mpetrunic]

This makes sense to me, but docs should be updated^^
I'm also curious about what @achingbrain thinks about this^^

[achingbrain]

I like this idea and would ideally extend it to other components that are or should be optional.

For example in helia I'm creating disposable dial-only libp2p nodes for use as an RPC client, so they don't need the protocols that are enabled by default - identify, ping etc. Identify in particular causes loads of noise in the logs as the connection tends to get torn down before identify has completed so it'd be great to not have that enabled on the dial-only node at all.

Also I think there was interest from lodestar and others to generally have a more minimal libp2p with a smaller surface area.

Given that we have things like FetchService and PingService already and not module, I'm open to ideas but I think the API I'd like to see is something like:

const libp2p = await createLibp2p({
  transports: [
    tcp()
  ],
  connectionEncryption: [
    noise()
  ],
  streamMuxers: [
    yamux()
  ],
  services: [
    ping(),
    identify(),
    fetch(),
    nat(),
    myCustomService()
  ]
})

// then later
const value = await libp2p.services.fetch(someKey)
const foo = await libp2p.services.myCustomService.someMethod(bar)

We'd then remove .fetch, .ping etc from the libp2p interface as you'd access them via the .services key.

I'm not sure how this would work with the types though? As in, how to derive the type of the .services key from the configuration passed to createLibp2p?

[mpetrunic]

I'm not sure how this would work with the types though? As in, how to derive the type of the .services key from the configuration passed to createLibp2p?

With your example, it's probably impossible, but with original proposal where you pass service map it could be done

[saul-jb]

I have updated the API to use a services configuration field instead of modules and I have gotten types to work for these services.

const libp2p = await createLibp2p({
  transports: [
    tcp()
  ],
  connectionEncryption: [
    noise()
  ],
  streamMuxers: [
    yamux()
  ],
  services: {
    ping: ping(),
    identify: identify(),
    fetch: fetch(),
    nat: nat(),
    myCustomService: myCustomService()
  }
})

const value = await libp2p.services.fetch(someKey)
const foo = await libp2p.services.myCustomService.someMethod(bar)

Note that I have expanded the return type of createLibp2p from Libp2p to include the services property but this should probably be added to the @libp2p/interface-libp2p interface.

@achingbrain, if it were possible to have services as an array instead of a map and still have types work, how would we reference a service from the node by name? E.g. how would the following array map to the following object:

[ ping(), fetch() ] => { ping: (service), fetch: (service) }

Would services be required to return an object with a property that identities the name of the service? If this were the case you could also require they have a type property the identifies the type of service (or perhaps just infer it) and it might be possible to move transports, muxers, etc. to the services list.

[achingbrain]

This is a good idea and it's something we're going to do, I just shy away from it a little because it's going to be quite disruptive. Maybe after the current round of breaking changes.

[p-shahi]

@saul-jb This PR was discussed in today's js-libp2p Triage meeting. In terms of timeline, we are thinking of including this in a js-libp2p release after IPFS Thing (April 15-19). Until then, we want to limit disruptive changes to js-libp2p and keep focus on bugfixes/hardening and general WebRTC related upkeep (new features landing soon in js-libp2p-webrtc).
Thanks for your patience!

[saul-jb]

@achingbrain, @p-shahi, What is the state of this, is there anything more I can do to help?

[p-shahi]

@saul-jb Per this comment #1707 (comment) @achingbrain intends to include this in the upcoming release (0.45.0)

[maschad]

This is going to really improve the flexibility, security and size of js-libp2p so thanks a lot for this @saul-jb and @achingbrain . It looks good to me, I only have two comments. I also wonder if we should include #1684 in this PR but it's not a requirement to land this.

[wemeetagain]

LGTM
I didn't check it out and run it, but just looking at the structure it looks good.

Would still love IdentifyService#identify to return the result but I guess thats a separate discussion.