This repository has been archived by the owner on Jul 21, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 60
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
1. Queries are plain functions now 1. Queries are run by the QueryManager 1. Query functions yield events - dial, send request, receive response, value, error, etc 1. Timeouts have been replaced with AbortSignals 1. Halting in-progress queries is done by aborting an AbortSignal from the top level or breaking out of the `for await..of` loop 1. Internal components do not depend on the top-level DHT component any more 1. Some methods have been moved to other components to break circular dependencies 1. The top level KadDHT class now has a ts interface and a factory function 1. There are two DHTs, lan (for private addresses) and wan (for public addresses) 1. Client mode is now on by default for the wan DHT, off for the lan DHT 1. DHT queries for CIDs search for the multihash rather than the full CID
- Loading branch information
1 parent
eff54bf
commit 1f8bc6a
Showing
72 changed files
with
4,436 additions
and
4,623 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| name: ci | ||
| on: | ||
| push: | ||
| branches: | ||
| - master | ||
| pull_request: | ||
| branches: | ||
| - master | ||
|
|
||
| jobs: | ||
| check: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - run: npm install | ||
| - run: npx aegir lint | ||
| - uses: gozala/typescript-error-reporter-action@v1.0.8 | ||
| - run: npx aegir build | ||
| - run: npx aegir dep-check | ||
| - uses: ipfs/aegir/actions/bundle-size@master | ||
| name: size | ||
| with: | ||
| github_token: ${{ secrets.GITHUB_TOKEN }} | ||
| test-node: | ||
| needs: check | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| matrix: | ||
| os: [windows-latest, ubuntu-latest, macos-latest] | ||
| node: [14, 16] | ||
| fail-fast: true | ||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - uses: actions/setup-node@v1 | ||
| with: | ||
| node-version: ${{ matrix.node }} | ||
| - run: npm install | ||
| - run: npx nyc --reporter=lcov aegir test -t node -- --bail | ||
| - uses: codecov/codecov-action@v1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -41,3 +41,4 @@ test/test-data/go-ipfs-repo/LOG.old | |
| # while testing npm5 | ||
| package-lock.json | ||
| yarn.lock | ||
| .nyc_output | ||
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,67 +1,26 @@ | ||
| # js-libp2p-kad-dht | ||
| # js-libp2p-kad-dht <!-- omit in toc --> | ||
|
|
||
| js-libp2p-kad-dht is a JavaScript implementation of the [Kademlia DHT](http://www.scs.stanford.edu/%7Edm/home/papers/kpos.pdf) with some features of S/Kademlia. A "provider" node uses the DHT to advertise that it has a particular piece of content, and "querying" nodes will search the DHT for peers that have a particular piece of content. Content is modeled as a value that is identified by a key, where the key and value are Buffers. | ||
| js-libp2p-kad-dht is a JavaScript implementation of the [Kademlia DHT][] with some features of [S/Kademlia][]. A "provider" node uses the DHT to advertise that it has a particular piece of content, and "querying" nodes will search the DHT for peers that have a particular piece of content. Content is modelled as a value that is identified by a key, where the key and value are Uint8Arrays. | ||
|
|
||
| #### DHT Identifiers | ||
| ## Table of contents <!-- omit in toc --> | ||
|
|
||
| The DHT uses a sha2-256 hash for identifiers: | ||
| - For peers the DHT identifier is the hash of the [PeerId][PeerId] | ||
| - For content the DHT identifier is the hash of the key (eg a Block CID) | ||
| - [Spec](#spec) | ||
| - [Extensions](#extensions) | ||
| - [Disjoint paths](#disjoint-paths) | ||
|
|
||
| #### FIND_NODE | ||
| ## Spec | ||
|
|
||
| `findPeer (PeerId):` [PeerInfo][PeerInfo] | ||
| js-libp2p-kad-dht follows the [libp2p/kad-dht spec](https://github.com/libp2p/specs/tree/master/kad-dht) and implements the algorithms described in the [IPFS DHT documentation](https://docs.ipfs.io/concepts/dht/). | ||
|
|
||
| The address space is so large (256 bits) that there are big gaps between DHT ids, and nodes frequently join and leave the DHT. | ||
| ## Extensions | ||
|
|
||
| To find a particular node | ||
| - the `querying node` converts the [PeerId][PeerId] to a DHT id | ||
| - the `querying node` sends a request to the nearest peers to that DHT id that it knows about | ||
| - those peers respond with the nearest peers to the DHT id that they know about | ||
| - the `querying node` sorts the responses and recursively queries the closest peers to the DHT id, continuing until it finds the node or it has queried all the closest peers. | ||
| js-libp2p-kad-dht implements some additional functionality not described in the libp2p KAD-DHT spec. | ||
|
|
||
| #### PUT | ||
| ### Disjoint paths | ||
|
|
||
| `put (Key, Value)` | ||
|
|
||
| To store a value in the DHT, the `provider node` | ||
| - converts the key to a DHT id | ||
| - follows the "closest peers" algorithm as above to find the nearest peers to the DHT id | ||
| - sends the value to those nearest peers | ||
|
|
||
| Note that DHT nodes will only store values that are accepted by its "validators", configurable functions that validate the key/value to ensure the node can control what kind of content it stores (eg IPNS records). | ||
|
|
||
| #### GET | ||
|
|
||
| `get (Key): [Value]` | ||
|
|
||
| To retrieve a value from the DHT | ||
| - the `querying node` converts the key to a DHT id | ||
| - the `querying node` follows the "closest peers" algorithm to find the nearest peers to the DHT id | ||
| - at each iteration of the algorithm, if the peer has the value it responds with the value itself in addition to closer peers. | ||
|
|
||
| Note that the value for a particular key is stored by many nodes, and these nodes receive `PUT` requests asynchronously, so it's possible that nodes may have distinct values for the same key. For example if node A `PUT`s the value `hello` to key `greeting` and node B concurrently `PUT`s the value `bonjour` to key `greeting`, some nodes close to the key `greeting` may receive `hello` first and others may receive `bonjour` first. | ||
|
|
||
| Therefore a `GET` request to the DHT may collect distinct values (eg `hello` and `bonjour`) for a particular key from the nodes close to the key. The DHT has "selectors", configurable functions that choose the "best" value (for example IPNS records include a sequence number, and the "best" value is the record with the highest sequence number). | ||
|
|
||
| #### PROVIDE | ||
|
|
||
| `provide (Key)` | ||
|
|
||
| To advertise that it has the content for a particular key | ||
| - the `provider node` converts the key to a DHT id | ||
| - the `provider node` follows the "closest peers" algorithm to find the nearest peers to the DHT id | ||
| - the `provider node` sends a "provide" message to each of the nearest peers | ||
| - each of the nearest peers saves the association between the "provider" peer and the key | ||
|
|
||
| #### FIND_PROVIDERS | ||
|
|
||
| `findProviders (Key):` [[PeerInfo][PeerInfo]] | ||
|
|
||
| To find providers for a particular key | ||
| - the `querying node` converts the key to a DHT id | ||
| - the `querying node` follows the "closest peers" algorithm to find the nearest peers to the DHT id | ||
| - at each iteration of the algorithm, if the peer knows which nodes are providing the value it responds with the provider nodes in addition to closer peers. | ||
| js-libp2p-kad-dht uses disjoint paths when performing DHT queries. These are described in the [S/Kademlia][] paper. | ||
|
|
||
| [Kademlia DHT]: (http://www.scs.stanford.edu/%7Edm/home/papers/kpos.pdf) | ||
| [S/Kademlia]: (https://git.gnunet.org/bibliography.git/plain/docs/SKademlia2007.pdf) | ||
| [PeerId]: https://github.com/libp2p/js-peer-id | ||
| [PeerInfo]: https://github.com/libp2p/js-peer-info |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.