From e8123d3f7994d3a2f8d1ed75f72e08a834979636 Mon Sep 17 00:00:00 2001 From: Chad Nehemiah Date: Wed, 4 Oct 2023 11:57:09 -0500 Subject: [PATCH] docs: add initial architecture diagrams for js-libp2p (#2121) --- doc/ARCHITECTURE.md | 217 +++++++++++++++++++++++++++++++ doc/CONFIGURATION.md | 2 + doc/CONNECTION_MANAGER.md | 3 - doc/DIALER.md | 32 ----- examples/nat-traversal/README.md | 2 - 5 files changed, 219 insertions(+), 37 deletions(-) create mode 100644 doc/ARCHITECTURE.md delete mode 100644 doc/CONNECTION_MANAGER.md delete mode 100644 doc/DIALER.md delete mode 100644 examples/nat-traversal/README.md diff --git a/doc/ARCHITECTURE.md b/doc/ARCHITECTURE.md new file mode 100644 index 0000000000..59489f3f42 --- /dev/null +++ b/doc/ARCHITECTURE.md @@ -0,0 +1,217 @@ +# Libp2p Architecture + +js-libp2p is comprised of a number of components that work together to provide functionality such as dailling peers, managing connections, registering protocols, storing information about peers and much more. This document aims to provide a high level overview of the components and how they interact with each other. + +- [Libp2p Architecture](#libp2p-architecture) + - [Component Diagram](#component-diagram) + - [Sequence Diagrams](#sequence-diagrams) + - [Dialing a Peer](#dialing-a-peer) + - [Opening a stream on a connection](#opening-a-stream-on-a-connection) + +## Component Diagram + +```mermaid +flowchart TB + direction TB + subgraph Components + direction TB + PeerId ~~~ + Events ~~~ + ConnectionGater ~~~ + Upgrader + AddressManager ~~~ + ConnectionManager ~~~ + TransportManager ~~~ + Registrar + PeerStore ~~~ + Datastore ~~~ + PeerRouting ~~~ + _xx[ ] + ContentRouting ~~~ + Metrics ~~~ + ConnectionProtector ~~~ + _x[ ] + + style _x opacity:0; + style _xx opacity:0; + + end + + subgraph Connections[Connection Configuration] + direction TB + + subgraph Transports + direction TB + TCP + WebRTC + Websocket + Webtransport + end + + subgraph Encryption[Connection Encryptions] + direction TB + Noise + Plaintext + end + + subgraph Multiplexer[Stream Multiplexers] + direction TB + Yamux + Mplex + end + + Multiplexer ~~~ Encryption ~~~ Transports + + end +``` + +```mermaid +--- +title: Components Dependency Graph +--- +flowchart TD + PeerId + Events + ConnectionGater + Upgrader + AddressManager + ConnectionManager + TransportManager + Registrar + PeerStore + Datastore + PeerRouting + ContentRouting + Metrics + ConnectionProtector + + %% AddressManager + PeerId --> AddressManager + TransportManager --> AddressManager + PeerStore --> AddressManager + Events --> AddressManager + + %% ConnectionManager + PeerId --> ConnectionManager + Metrics --> ConnectionManager + PeerStore --> ConnectionManager + TransportManager --> ConnectionManager + ConnectionGater --> ConnectionManager + Events --> ConnectionManager + + %% TransportManager + Metrics --> TransportManager + AddressManager --> TransportManager + Upgrader --> TransportManager + Events --> TransportManager + + %% Upgrader + PeerId --> Upgrader + Metrics --> Upgrader + ConnectionManager --> Upgrader + ConnectionGater --> Upgrader + ConnectionProtector --> Upgrader + Registrar --> Upgrader + PeerStore --> Upgrader + Events --> Upgrader + + %% Registrar + PeerId --> Registrar + ConnectionManager --> Registrar + PeerStore --> Registrar + Events --> Registrar + + %% PeerStore + PeerId --> PeerStore + Datastore --> PeerStore + Events --> PeerStore + + %% PeerRouting + PeerId --> PeerRouting + PeerStore --> PeerRouting + + %% ContentRouting + PeerStore --> ContentRouting +``` + +## Sequence Diagrams + +These diagrams show the interactions between the components in common scenarios. They are not exhaustive and are intended to provide a high level overview of the interactions between the components. + +### Dialing a Peer + +This illustrates an outbound connection being established to a peer. + +```mermaid +%% how an outbound connection is opened when a user calls .dial(), +%% assuming user is not connected to the PeerId for the +%% Multiaddr that was dialed. +%% +%% This is +%% +sequenceDiagram + User->>+libp2p: dial a multiaddr `.dial()` + libp2p->>+Connection Manager: open a connection for me to MA `.openConnection()` + %% obfuscating the dial queue. + %% Connection Manager->>+Transport Manager: Choose transport to use for Multiaddr + Connection Manager->>+Transport Manager: Network level reach out `.dial()` + Transport Manager->>+Transport: Get MultiaddrConn `socket + multiaddr` + %% Transport->>+Transport Manager: Return MultiaddrConn `socket + multiaddr` + %% how the upgrade happens is transport specific, so transports directly call upgrader + Transport-->>+Upgrader: upgrade my connection?? + Upgrader-->>+Upgrader: Perform upgrade (see other diagram) + Upgrader->>+Connection Manager: Connection (link to interface) + %% Connection Manager->>+Connection Manager: Connection (link to interface) + Connection Manager->>+User: Connection (link to interface) +``` + + +### Opening a stream on a connection + +This illustrates a stream being opened on an existing connection that will echo a message back to the sender. This assumes that a stable connection has been established between the two peers. + +```mermaid +%% pushing data over stream +%% register stream handler, local opens a stream for proto, send data, +%% remote receives data and sends data back +%% local receives data +%% stream may or may not then be closed. +%% Local is the node sending data, Remote is other peer the conn is with +%% Echo protocol +sequenceDiagram + box Local side + participant Local + participant Connection + participant LocalMuxer + end + participant Stream + box pink Connection + end + box Remote side + participant Remote + participant RemoteMuxer + participant RemoteUpgrader + participant RemoteRegistrar + participant RemoteStreamHandler + end + + Remote->>RemoteRegistrar: Register Stream Handler `libp2p.handle` + %% only register stream handlers when you want to listen for protocols. SENDERs do not need to listen + Local->>Connection: Open outbound stream + Connection->>LocalMuxer: Open stream + LocalMuxer->>RemoteMuxer: Open stream + RemoteMuxer->>RemoteUpgrader: notify Stream created + Note over Connection,RemoteUpgrader: multi stream select handles protocol negotiation + Connection->>Local: return Stream + RemoteUpgrader->>RemoteRegistrar: select stream handler + RemoteRegistrar->>RemoteStreamHandler: handle stream + Note over RemoteStreamHandler,Local: Stream data flow & control is dictated by protocol, below is example of "echo" + activate Stream + Local->>Stream: send bytes "hello world" + Stream->>RemoteStreamHandler: receive bytes "hello world" + %% RemoteStreamHandler->>+RemoteStreamHandler: [echo] pipe back received bytes + RemoteStreamHandler->>Stream: echo bytes back to sender + Stream->>Local: receive echoed bytes + deactivate Stream + +``` \ No newline at end of file diff --git a/doc/CONFIGURATION.md b/doc/CONFIGURATION.md index 4aea12b2f9..dcba663f72 100644 --- a/doc/CONFIGURATION.md +++ b/doc/CONFIGURATION.md @@ -44,6 +44,8 @@ libp2p is a modular networking stack. It's designed to be able to suit a variety Regardless of how you configure libp2p, the top level [API](./API.md) will always remain the same. **Note**: if some modules are not configured, like Content Routing, using those methods will throw errors. +To get a high-level overview of the js-libp2p architecture, please read the [Architecture](./ARCHITECTURE.md) document. + ## Modules `js-libp2p` acts as the composer for this modular p2p networking stack using libp2p compatible modules as its subsystems. For getting an instance of `js-libp2p` compliant with all types of networking requirements, it is possible to specify the following subsystems: diff --git a/doc/CONNECTION_MANAGER.md b/doc/CONNECTION_MANAGER.md deleted file mode 100644 index a22a14fcb1..0000000000 --- a/doc/CONNECTION_MANAGER.md +++ /dev/null @@ -1,3 +0,0 @@ -# Connection Manager - -The documentation here has moved to https://libp2p.github.io/js-libp2p-interfaces/modules/_libp2p_interface_connection_manager.html - please update your bookmarks! diff --git a/doc/DIALER.md b/doc/DIALER.md deleted file mode 100644 index bba39d5b70..0000000000 --- a/doc/DIALER.md +++ /dev/null @@ -1,32 +0,0 @@ -# js-libp2p Dialer - -**Synopsis** -* Parallel dials to the same peer will yield the same connection/error when the first dial settles. -* All Dial Requests in js-libp2p must request a token(s) from the Dialer. - * The number of tokens requested should be between 1 and the MAX_PER_PEER_DIALS max set in the Dialer. - * If the number of available tokens is less than requested, the Dialer may return less than requested. -* The number of tokens a DialRequest obtains reflects the maximum number of parallel Multiaddr Dials it can make. -* If no tokens are available a DialRequest should immediately end and throw. -* As tokens are limited, DialRequests should be given a prioritized list of Multiaddrs to minimize the potential request time. -* Once a Multiaddr Dial has succeeded, all pending dials in that Dial Request should be aborted. -* If DIAL_TIMEOUT time has elapsed before any one Multiaddr Dial succeeds, all remaining dials in the DialRequest should be aborted. -* When a Multiaddr Dial is settled, if there are no more addresses to dial, its token should be released back to the dialer. -* Once the DialRequest is settled, any remaining tokens should be released to the dialer. - -## Multiaddr Confidence - -An effective dialing system should involve the inclusion of a Confidence system for Multiaddrs. This enables ranking of Multiaddrs so that a prioritized list can be passed to DialRequests to maximize usage of Dialer Tokens, and minimize connection times. - -**Not Yet Implemented**: This system will be designed and implemented in a future update. - -## Notes - -* A DialRequest gets a set of tokens from the Dialer, up to the MAX_PER_PEER_DIALS max. -* A DialRequest SHOULD fail if no dial tokens are available. The DialRequest MAY proceed without tokens, but this should be reserved for High Priority actions and should be rare. -* A DialRequest MUST NOT request more tokens than it has addresses to dial. Example: If the MAX_PER_PEER_DIALS is 4 and a DialRequest has 1 address, it should only request 1 token. -* A DialRequest SHOULD execute parallel dials for each of its addresses up the total number of tokens it has. -* On a successful dial, the DialRequest MUST abort any other in progress dials, return the successful connection and release all tokens. -* A new DialRequest SHOULD be given a descending list of prioritized Multiaddrs, based on their confidence. Having higher confidence Multiaddrs first can help minimize the time a DialRequest is active. -* A failed dial to a Multiaddr SHOULD add that Multiaddr to a temporary denyList. -* A failed dial to a Multiaddr SHOULD lower the confidence of that Multiaddr. -* A successful dial to a Multiaddr SHOULD increase the confidence of that Multiaddr. diff --git a/examples/nat-traversal/README.md b/examples/nat-traversal/README.md deleted file mode 100644 index 38f33ceff9..0000000000 --- a/examples/nat-traversal/README.md +++ /dev/null @@ -1,2 +0,0 @@ -# WIP - This example is still in the works -![](http://1.bp.blogspot.com/-tNvSnCW0KlQ/U-KOKGVoJkI/AAAAAAAAA3Q/aiSLMeSJFtw/s1600/WIP-sign.jpg)