From 2455d11c94a0d0c20f1d965e3192083e42f58111 Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Mon, 30 Nov 2020 11:55:33 +0100 Subject: [PATCH 01/10] feat: add implementation --- README.md | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++- package.json | 45 ++++++++++++++++++-- src/bin.js | 105 +++++++++++++++++++++++++++++++++++++++++++++ src/index.js | 79 ++++++++++++++++++++++++++++++++++ src/utils.js | 41 ++++++++++++++++++ 5 files changed, 381 insertions(+), 6 deletions(-) create mode 100644 src/bin.js create mode 100644 src/index.js create mode 100644 src/utils.js diff --git a/README.md b/README.md index 18e0d38..8b60948 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,115 @@ -# js-libp2p-hop-relay-server -A out of the box libp2p relay server with HOP +# js-libp2p-hop-relay-server + +[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://protocol.ai) +[![](https://img.shields.io/badge/project-libp2p-yellow.svg?style=flat-square)](http://libp2p.io/) +[![](https://img.shields.io/badge/freenode-%23libp2p-yellow.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23libp2p) +[![](https://img.shields.io/discourse/https/discuss.libp2p.io/posts.svg)](https://discuss.libp2p.io) +[![GitHub Workflow Status](https://img.shields.io/github/workflow/status/libp2p/js-libp2p-hop-relay-server/ci?label=ci&style=flat-square)](https://github.com/libp2p/js-libp2p-hop-relay-server/actions?query=branch%3Amaster+workflow%3Aci+) + +> An out of the box libp2p relay server with HOP + +## Lead Maintainer + +[Vasco Santos](https://github.com/vasco-santos) + +## Table of Contents + +- [Background](#background) +- [Usage](#usage) + - [Install](#install) + - [CLI](#cli) + - [Docker](#docker) +- [Contribute](#contribute) +- [License](#license) + +## Background + +Libp2p nodes acting as circuit relay aim to establish connectivity between libp2p nodes (e.g. IPFS nodes) that wouldn't otherwise be able to establish a direct connection to each other. + +A relay is needed in situations where nodes are behind NAT, reverse proxies, firewalls and/or simply don't support the same transports (e.g. go-libp2p vs. browser-libp2p). The circuit relay protocol exists to overcome those scenarios. Nodes with the `auto-relay` feature enabled can automatically bind themselves on a relay to listen for connections on their behalf. + +You can read more in its [SPEC](https://github.com/libp2p/specs/tree/master/relay). + +## Usage + +### Install + +```bash +> npm install --global libp2p-hop-relay-server +``` + +Now you can use the cli command `libp2p-hop-relay-server` to spawn an out of the box libp2p hop relay server. + +### CLI + +After installing the relay server, you can use its binary. It accepts several arguments: `--peerId`, `--listenMultiaddrs`, `--announceMultiaddrs`, `--metricsMultiaddr`, `--disableMetrics`, `--delegateMultiaddr` and `--disableAdvertise`. + +```sh +libp2p-hop-relay-server [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] [--metricsMultiaddr ] [--disableMetrics] [--delegateMultiaddr ] [--disableAdvertise] +``` + +#### PeerId + +You can create a [PeerId](https://github.com/libp2p/js-peer-id) via its [CLI](https://github.com/libp2p/js-peer-id#cli). + +```sh +libp2p-hop-relay-server --peerId id.json +``` + +#### Multiaddrs + +You can specify the libp2p rendezvous server listen and announce multiaddrs. This server is configured with [libp2p-tcp](https://github.com/libp2p/js-libp2p-tcp) and [libp2p-websockets](https://github.com/libp2p/js-libp2p-websockets) and addresses with this transports should be used. It can always be modified via the API. + +```sh +libp2p-hop-relay-server --peerId id.json --listenMultiaddrs '/ip4/127.0.0.1/tcp/15002/ws' '/ip4/127.0.0.1/tcp/8000' --announceMultiaddrs '/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' '/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' +``` + +By default it listens on `/ip4/127.0.0.1/tcp/15002/ws` and has no announce multiaddrs specified. + +#### Metrics + +Metrics are enabled by default on `/ip4/127.0.0.1/tcp/8003` via Prometheus. This address can also be modified with: + +```sh +libp2p-hop-relay-server --metricsMultiaddr '/ip4/127.0.0.1/tcp/8000' +``` + +Moreover, metrics can also be disabled with: + +```sh +libp2p-hop-relay-server --disableMetrics +``` + +#### Advertise + +The relay server will advertise its HOP capability by default, using a delegate node on `/dns4/node0.delegate.ipfs.io/tcp/443/https`. This is important for peers that will try to find HOP relays on the network to bind themselves. + +This advertise can be disabled with: + +```sh +libp2p-hop-relay-server --disableAdvertise +``` + +You can also customize the delegate node to use with: + +```sh +libp2p-hop-relay-server --delegateMultiaddr '/dns4/node1.delegate.ipfs.io/tcp/443/https' +``` + +Note: In the future this will leverage libp2p's [DHT](https://github.com/libp2p/js-libp2p-kad-dht). + +### Docker + +TODO + +## Contribute + +Feel free to join in. All welcome. Open an [issue](https://github.com/libp2p/js-libp2p-hop-relay-server/issues)! + +This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). + +[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/contributing.md) + +## License + +MIT - Protocol Labs 2020 \ No newline at end of file diff --git a/package.json b/package.json index 6c01cee..92e9e4f 100644 --- a/package.json +++ b/package.json @@ -2,21 +2,58 @@ "name": "libp2p-hop-relay-server", "version": "0.0.0", "description": "A out of the box libp2p relay server with HOP", + "leadMaintainer": "Vasco Santos ", "main": "src/index.js", + "bin": { + "libp2p-hop-relay-server": "src/bin.js" + }, "scripts": { - "test": "echo \"Error: no test specified\" && exit 1" + "lint": "aegir lint", + "test": "aegir test", + "test:node": "aegir test -t node", + "test:browser": "aegir test -t browser", + "test:types": "aegir ts -p check", + "build": "aegir build", + "release": "aegir release", + "release-minor": "aegir release --type minor", + "release-major": "aegir release --type major", + "docs": "aegir docs", + "size": "aegir build -b" }, + "files": [ + "src", + "dist" + ], "repository": { "type": "git", "url": "git+https://github.com/vasco-santos/js-libp2p-hop-relay-server.git" }, "keywords": [ - "libp2p" + "libp2p", + "relay", + "auto relay", + "hop" ], - "author": "", + "author": "Vasco Santos ", "license": "MIT", "bugs": { "url": "https://github.com/vasco-santos/js-libp2p-hop-relay-server/issues" }, - "homepage": "https://github.com/vasco-santos/js-libp2p-hop-relay-server#readme" + "homepage": "https://github.com/vasco-santos/js-libp2p-hop-relay-server#readme", + "dependencies": { + "ipfs-http-client": "^48.1.2", + "libp2p": "libp2p/js-libp2p#0.30.x", + "libp2p-delegated-content-routing": "^0.8.0", + "libp2p-mplex": "^0.10.1", + "libp2p-noise": "^2.0.1", + "libp2p-tcp": "^0.15.1", + "libp2p-websockets": "^0.14.0", + "menoetius": "0.0.2", + "minimist": "^1.2.5", + "multiaddr": "^8.1.1", + "peer-id": "^0.14.2" + }, + "devDependencies": { + "aegir": "^29.1.0" + } } diff --git a/src/bin.js b/src/bin.js new file mode 100644 index 0000000..ed82289 --- /dev/null +++ b/src/bin.js @@ -0,0 +1,105 @@ +#!/usr/bin/env node + +'use strict' + +// Usage: $0 [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] +// [--metricsMultiaddr ] [--disableMetrics] [--delegateMultiaddr ] [--disableAdvertise] + +/* eslint-disable no-console */ + +const debug = require('debug') +const log = debug('libp2p:hop-relay:bin') + +const fs = require('fs') +const http = require('http') +const menoetius = require('menoetius') +const argv = require('minimist')(process.argv.slice(2)) + +const multiaddr = require('multiaddr') +const PeerId = require('peer-id') + +const { getAnnounceAddresses, getListenAddresses } = require('./utils') +const createRelay = require('./index') + +async function main () { + // Metrics + let metricsServer + const metrics = !(argv.disableMetrics) + const metricsMa = multiaddr(argv.metricsMultiaddr || argv.ma || '/ip4/127.0.0.1/tcp/8003') + const metricsAddr = metricsMa.nodeAddress() + + // multiaddrs + const listenAddresses = getListenAddresses(argv) + const announceAddresses = getAnnounceAddresses(argv) + + // Should advertise + const shouldAdvertise = !(argv.disableAdvertise) + + // Delegate + let delegateOptions + if (argv.delegateMultiaddr || argv.dm) { + const delegateAddr = multiaddr(argv.delegateMultiaddr || argv.dm).toOptions() + delegateOptions = { + host: delegateAddr.host, + protocol: delegateAddr.port === '443' ? 'https' : 'http', + port: delegateAddr.port + } + } + + // PeerId + let peerId + if (argv.peerId) { + const peerData = fs.readFileSync(argv.peerId) + peerId = await PeerId.createFromJSON(JSON.parse(peerData)) + } else { + peerId = await PeerId.create() + log('You are using an automatically generated peer.') + log('If you want to keep the same address for the server you should provide a peerId with --peerId ') + } + + // Create Relay + const relay = await createRelay({ + peerId, + listenAddresses, + announceAddresses, + shouldAdvertise, + delegateOptions + }) + + relay.peerStore.on('change:multiaddrs', ({ peerId: changedPeerId, multiaddrs }) => { + if (peerId.equals(changedPeerId)) { + console.log('Relay server listening on:') + multiaddrs.forEach((m) => console.log(m)) + } + }) + + await relay.start() + + if (metrics) { + log('enabling metrics') + metricsServer = http.createServer((req, res) => { + if (req.url !== '/metrics') { + res.statusCode = 200 + res.end() + } + }) + + menoetius.instrument(metricsServer) + + metricsServer.listen(metricsAddr.port, metricsAddr.address, () => { + console.log(`metrics server listening on ${metricsAddr.port}`) + }) + } + + const stop = async () => { + console.log('Stopping...') + await relay.stop() + metricsServer && await metricsServer.close() + process.exit(0) + } + + process.on('SIGTERM', stop) + process.on('SIGINT', stop) +} + +main() diff --git a/src/index.js b/src/index.js new file mode 100644 index 0000000..5e4cfec --- /dev/null +++ b/src/index.js @@ -0,0 +1,79 @@ +'use strict' + +const Libp2p = require('libp2p') +const TCP = require('libp2p-tcp') +const Websockets = require('libp2p-websockets') +const Muxer = require('libp2p-mplex') +const { NOISE: Crypto } = require('libp2p-noise') +const DelegatedContentRouting = require('libp2p-delegated-content-routing') +const ipfsHttpClient = require('ipfs-http-client') + +/** + * @typedef {import('peer-id')} PeerId + */ + +/** + * @typedef {Object} DelegateOptions + * @property {string} host + * @property {string} protocol + * @property {number} port + */ + + const defaulDelegateOptions = { + host: 'node0.delegate.ipfs.io', + protocol: 'https', + port: 443 + } + +/** + * @typedef {Object} HopRelayOptions + * @property {PeerId} peerId + * @property {DelegateOptions} [delegateOptions = defaulDelegateOptions] + * @property {string[]} [listenAddresses = []] + * @property {string[]} [announceAddresses = []] + * @property {boolean} [shouldAdvertise = true] + */ + +/** + * Create a Libp2p Relay with HOP service + * + * @param {HopRelayOptions} options + * @returns {Promise} + */ +function create ({ peerId, delegateOptions = defaulDelegateOptions, listenAddresses = [], announceAddresses = [], shouldAdvertise = true }) { + let contentRouting = [] + + if (shouldAdvertise) { + const httpClient = ipfsHttpClient(delegateOptions) + contentRouting.push(new DelegatedContentRouting(peerId, httpClient)) + } + + return Libp2p.create({ + peerId, + modules: { + transport: [Websockets, TCP], + streamMuxer: [Muxer], + connEncryption: [Crypto], + contentRouting + }, + peerId, + addresses: { + listen: listenAddresses, + announce: announceAddresses + }, + config: { + relay: { + enabled: true, // Allows you to dial and accept relayed connections. Does not make you a relay. + hop: { + enabled: true, // Allows you to be a relay for other peers + active: true // You will attempt to dial destination peers if you are not connected to them + }, + advertise: { + enabled: shouldAdvertise // Allows you to advertise the Hop service + } + } + } + }) +} + +module.exports = create diff --git a/src/utils.js b/src/utils.js new file mode 100644 index 0000000..d7f8d40 --- /dev/null +++ b/src/utils.js @@ -0,0 +1,41 @@ +'use strict' + +const multiaddr = require('multiaddr') + +function getAnnounceAddresses(argv) { + const announceAddr = argv.announceMultiaddrs || argv.am + const announceAddresses = announceAddr ? [multiaddr(announceAddr)] : [] + + if (argv.announceMultiaddrs || argv.am) { + const flagIndex = process.argv.findIndex((e) => e === '--announceMultiaddrs' || e === '--am') + const tmpEndIndex = process.argv.slice(flagIndex + 1).findIndex((e) => e.startsWith('--')) + const endIndex = tmpEndIndex !== -1 ? tmpEndIndex : process.argv.length - flagIndex - 1 + + for (let i = flagIndex + 1; i < flagIndex + endIndex; i++) { + announceAddresses.push(multiaddr(process.argv[i + 1])) + } + } + + return announceAddresses +} + +module.exports.getAnnounceAddresses = getAnnounceAddresses + +function getListenAddresses(argv) { + const listenAddr = argv.listenMultiaddrs || argv.lm || '/ip4/127.0.0.1/tcp/0/ws' + const listenAddresses = [multiaddr(listenAddr)] + + if (argv.listenMultiaddrs || argv.lm) { + const flagIndex = process.argv.findIndex((e) => e === '--listenMultiaddrs' || e === '--lm') + const tmpEndIndex = process.argv.slice(flagIndex + 1).findIndex((e) => e.startsWith('--')) + const endIndex = tmpEndIndex !== -1 ? tmpEndIndex : process.argv.length - flagIndex - 1 + + for (let i = flagIndex + 1; i < flagIndex + endIndex; i++) { + listenAddresses.push(multiaddr(process.argv[i + 1])) + } + } + + return listenAddresses +} + +module.exports.getListenAddresses = getListenAddresses From 8229eda90b770734050df5047e6b57c407f2748e Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Mon, 30 Nov 2020 12:10:07 +0100 Subject: [PATCH 02/10] chore: use main brain in gh action --- .github/workflows/main.yml | 10 ++-------- Dockerfile | 35 +++++++++++++++++++++++++++++++++++ src/utils.js | 2 +- 3 files changed, 38 insertions(+), 9 deletions(-) create mode 100644 Dockerfile diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index f5a14b1..1f61772 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -2,10 +2,10 @@ name: ci on: push: branches: - - master + - main pull_request: branches: - - master + - main jobs: check: @@ -15,12 +15,6 @@ jobs: - run: yarn - run: yarn lint - run: yarn build - - uses: gozala/typescript-error-reporter-action@v1.0.4 - - run: yarn aegir dep-check -- -i aegir - - uses: ipfs/aegir/actions/bundle-size@master - name: size - with: - github_token: ${{ secrets.GITHUB_TOKEN }} test-node: needs: check runs-on: ${{ matrix.os }} diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..469dd87 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,35 @@ +FROM node:lts-buster + +# Install deps +RUN apt-get update && apt-get install -y + +# Get dumb-init to allow quit running interactively +RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 && chmod +x /usr/local/bin/dumb-init + +# Setup directories for the `node` user +RUN mkdir -p /home/node/app/hop-relay-server/node_modules && chown -R node:node /home/node/app/hop-relay-server + +WORKDIR /home/node/app/hop-relay-server + +# Install node modules +COPY package.json ./ +# Switch to the node user for installation +USER node +RUN npm install --production + +# Copy over source files under the node user +COPY --chown=node:node ./src ./src +COPY --chown=node:node ./README.md ./ + +# Environment variables + + +# hop-relay-server defaults to 15003 +EXPOSE 15003 + +# metrics defaults to 8003 +EXPOSE 8003 + +# Available overrides (defaults shown): +# Server logging can be enabled via the DEBUG environment variable +CMD [ "/usr/local/bin/dumb-init", "node", "src/server/bin.js"] \ No newline at end of file diff --git a/src/utils.js b/src/utils.js index d7f8d40..0e83a75 100644 --- a/src/utils.js +++ b/src/utils.js @@ -22,7 +22,7 @@ function getAnnounceAddresses(argv) { module.exports.getAnnounceAddresses = getAnnounceAddresses function getListenAddresses(argv) { - const listenAddr = argv.listenMultiaddrs || argv.lm || '/ip4/127.0.0.1/tcp/0/ws' + const listenAddr = argv.listenMultiaddrs || argv.lm || '/ip4/127.0.0.1/tcp/15003/ws' const listenAddresses = [multiaddr(listenAddr)] if (argv.listenMultiaddrs || argv.lm) { From a1595643794878bce814d8281b548b96af247eb0 Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Mon, 30 Nov 2020 15:22:11 +0100 Subject: [PATCH 03/10] chore: add docker setup --- .env | 1 + .github/workflows/main.yml | 1 - Dockerfile | 11 ++----- README.md | 31 +++++++++++++++++- package.json | 4 +-- src/bin.js | 18 +++++++---- src/index.js | 13 ++++---- src/utils.js | 28 +++++++++------- test/fixtures/peers.js | 27 ++++++++++++++++ test/index.spec.js | 66 ++++++++++++++++++++++++++++++++++++++ 10 files changed, 161 insertions(+), 39 deletions(-) create mode 100644 .env create mode 100644 test/fixtures/peers.js create mode 100644 test/index.spec.js diff --git a/.env b/.env new file mode 100644 index 0000000..a6980aa --- /dev/null +++ b/.env @@ -0,0 +1 @@ +LISTEN_MULTIADDRS='' \ No newline at end of file diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 1f61772..11ac525 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -14,7 +14,6 @@ jobs: - uses: actions/checkout@v2 - run: yarn - run: yarn lint - - run: yarn build test-node: needs: check runs-on: ${{ matrix.os }} diff --git a/Dockerfile b/Dockerfile index 469dd87..fac0710 100644 --- a/Dockerfile +++ b/Dockerfile @@ -21,15 +21,8 @@ RUN npm install --production COPY --chown=node:node ./src ./src COPY --chown=node:node ./README.md ./ -# Environment variables - - -# hop-relay-server defaults to 15003 -EXPOSE 15003 - -# metrics defaults to 8003 -EXPOSE 8003 +ENV DEBUG libp2p* # Available overrides (defaults shown): # Server logging can be enabled via the DEBUG environment variable -CMD [ "/usr/local/bin/dumb-init", "node", "src/server/bin.js"] \ No newline at end of file +CMD [ "/usr/local/bin/dumb-init", "node", "src/bin.js"] \ No newline at end of file diff --git a/README.md b/README.md index 8b60948..7cfd40e 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,8 @@ - [Install](#install) - [CLI](#cli) - [Docker](#docker) + - [SSL](#ssl) + - [Debug](#debug) - [Contribute](#contribute) - [License](#license) @@ -100,7 +102,34 @@ Note: In the future this will leverage libp2p's [DHT](https://github.com/libp2p/ ### Docker -TODO +When running the hop relay server in Docker, you can configure the same parameters via environment variables, as follows: + +```sh +PEER_ID='./id.json' +LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' +ANNOUNCE_MULTIADDRS='/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA,/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' +METRICS_MULTIADDR='/ip4/127.0.0.1/tcp/8000' +DISABLE_METRICS='true' +DELEGATE_MULTIADDR='/dns4/node1.delegate.ipfs.io/tcp/443/https' +DISABLE_ADVERTISE='true' +``` + +Please note that you should expose expose the used ports with the docker run command. The default ports used are `8003` for the metrics and `150003` for the websockets listener + +```sh +docker build NAME -t libp2p-hop-relay +docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -d libp2p-hop-relay +``` + +### SSL + +You should setup an SSL certificate with nginx and proxy to the API. You can use a service that already offers an SSL certificate with the server and configure nginx, or you can create valid certificates with for example [Letsencrypt](https://certbot.eff.org/lets-encrypt/osx-nginx). Letsencrypt won’t give you a cert for an IP address (yet) so you need to connect via SSL to a domain name. + +With this, you should specify in your relay the announce multiaddrs for your listening transports. This is specially important for browser peers that will leverage this relay, as browser nodes can only dial peers behind a `DNS+WSS` multiaddr. + +### Debug + +You can debug the relay by setting the `DEBUG` environment variable. For instance, you can set it to `libp2p*`. ## Contribute diff --git a/package.json b/package.json index 92e9e4f..5c7ff1a 100644 --- a/package.json +++ b/package.json @@ -9,10 +9,8 @@ }, "scripts": { "lint": "aegir lint", - "test": "aegir test", + "test": "aegir test -t node", "test:node": "aegir test -t node", - "test:browser": "aegir test -t browser", - "test:types": "aegir ts -p check", "build": "aegir build", "release": "aegir release", "release-minor": "aegir release --type minor", diff --git a/src/bin.js b/src/bin.js index ed82289..d2d041c 100644 --- a/src/bin.js +++ b/src/bin.js @@ -24,21 +24,24 @@ const createRelay = require('./index') async function main () { // Metrics let metricsServer - const metrics = !(argv.disableMetrics) - const metricsMa = multiaddr(argv.metricsMultiaddr || argv.ma || '/ip4/127.0.0.1/tcp/8003') + const metrics = !(argv.disableMetrics || process.env.DISABLE_METRICS) + const metricsMa = multiaddr(argv.metricsMultiaddr || argv.ma || process.env.METRICS_MULTIADDR || '/ip4/0.0.0.0/tcp/8003') const metricsAddr = metricsMa.nodeAddress() // multiaddrs const listenAddresses = getListenAddresses(argv) const announceAddresses = getAnnounceAddresses(argv) + log(`listenAddresses: ${listenAddresses.map((a) => a)}`) + announceAddresses.length && log(`announceAddresses: ${announceAddresses.map((a) => a)}`) + // Should advertise - const shouldAdvertise = !(argv.disableAdvertise) + const shouldAdvertise = !(argv.disableAdvertise || process.env.DISABLE_ADVERTISE) // Delegate let delegateOptions - if (argv.delegateMultiaddr || argv.dm) { - const delegateAddr = multiaddr(argv.delegateMultiaddr || argv.dm).toOptions() + if (argv.delegateMultiaddr || argv.dm || process.env.DELEGATE_MULTIADDR) { + const delegateAddr = multiaddr(argv.delegateMultiaddr || argv.dm || process.env.DELEGATE_MULTIADDR).toOptions() delegateOptions = { host: delegateAddr.host, protocol: delegateAddr.port === '443' ? 'https' : 'http', @@ -48,9 +51,10 @@ async function main () { // PeerId let peerId - if (argv.peerId) { - const peerData = fs.readFileSync(argv.peerId) + if (argv.peerId || process.env.PEER_ID) { + const peerData = fs.readFileSync(argv.peerId || process.env.PEER_ID) peerId = await PeerId.createFromJSON(JSON.parse(peerData)) + log('PeerId provided was loaded.') } else { peerId = await PeerId.create() log('You are using an automatically generated peer.') diff --git a/src/index.js b/src/index.js index 5e4cfec..68328fa 100644 --- a/src/index.js +++ b/src/index.js @@ -19,11 +19,11 @@ const ipfsHttpClient = require('ipfs-http-client') * @property {number} port */ - const defaulDelegateOptions = { - host: 'node0.delegate.ipfs.io', - protocol: 'https', - port: 443 - } +const defaulDelegateOptions = { + host: 'node0.delegate.ipfs.io', + protocol: 'https', + port: 443 +} /** * @typedef {Object} HopRelayOptions @@ -41,7 +41,7 @@ const ipfsHttpClient = require('ipfs-http-client') * @returns {Promise} */ function create ({ peerId, delegateOptions = defaulDelegateOptions, listenAddresses = [], announceAddresses = [], shouldAdvertise = true }) { - let contentRouting = [] + const contentRouting = [] if (shouldAdvertise) { const httpClient = ipfsHttpClient(delegateOptions) @@ -56,7 +56,6 @@ function create ({ peerId, delegateOptions = defaulDelegateOptions, listenAddres connEncryption: [Crypto], contentRouting }, - peerId, addresses: { listen: listenAddresses, announce: announceAddresses diff --git a/src/utils.js b/src/utils.js index 0e83a75..a380b59 100644 --- a/src/utils.js +++ b/src/utils.js @@ -1,19 +1,21 @@ 'use strict' -const multiaddr = require('multiaddr') +function getAnnounceAddresses (argv) { + let announceAddresses = [] + const argvAddr = argv.announceMultiaddrs || argv.am -function getAnnounceAddresses(argv) { - const announceAddr = argv.announceMultiaddrs || argv.am - const announceAddresses = announceAddr ? [multiaddr(announceAddr)] : [] + if (argvAddr) { + announceAddresses = [argvAddr] - if (argv.announceMultiaddrs || argv.am) { const flagIndex = process.argv.findIndex((e) => e === '--announceMultiaddrs' || e === '--am') const tmpEndIndex = process.argv.slice(flagIndex + 1).findIndex((e) => e.startsWith('--')) const endIndex = tmpEndIndex !== -1 ? tmpEndIndex : process.argv.length - flagIndex - 1 for (let i = flagIndex + 1; i < flagIndex + endIndex; i++) { - announceAddresses.push(multiaddr(process.argv[i + 1])) + announceAddresses.push(process.argv[i + 1]) } + } else if (process.env.ANNOUNCE_MULTIADDRS) { + announceAddresses = process.env.ANNOUNCE_MULTIADDRS.split(',') } return announceAddresses @@ -21,18 +23,22 @@ function getAnnounceAddresses(argv) { module.exports.getAnnounceAddresses = getAnnounceAddresses -function getListenAddresses(argv) { - const listenAddr = argv.listenMultiaddrs || argv.lm || '/ip4/127.0.0.1/tcp/15003/ws' - const listenAddresses = [multiaddr(listenAddr)] +function getListenAddresses (argv) { + let listenAddresses = ['/ip4/127.0.0.1/tcp/15003/ws'] + const argvAddr = argv.listenMultiaddrs || argv.lm + + if (argvAddr) { + listenAddresses = [argvAddr] - if (argv.listenMultiaddrs || argv.lm) { const flagIndex = process.argv.findIndex((e) => e === '--listenMultiaddrs' || e === '--lm') const tmpEndIndex = process.argv.slice(flagIndex + 1).findIndex((e) => e.startsWith('--')) const endIndex = tmpEndIndex !== -1 ? tmpEndIndex : process.argv.length - flagIndex - 1 for (let i = flagIndex + 1; i < flagIndex + endIndex; i++) { - listenAddresses.push(multiaddr(process.argv[i + 1])) + listenAddresses.push(process.argv[i + 1]) } + } else if (process.env.LISTEN_MULTIADDRS) { + listenAddresses = process.env.LISTEN_MULTIADDRS.split(',') } return listenAddresses diff --git a/test/fixtures/peers.js b/test/fixtures/peers.js new file mode 100644 index 0000000..27150e5 --- /dev/null +++ b/test/fixtures/peers.js @@ -0,0 +1,27 @@ +'use strict' + +module.exports = [{ + id: '12D3KooWNvSZnPi3RrhrTwEY4LuuBeB6K6facKUCJcyWG1aoDd2p', + privKey: 'CAESYHyCgD+3HtEHm6kzPO6fuwP+BAr/PxfJKlvAOWhc/IqAwrZjCNn0jz93sSl81cP6R6x/g+iVYmR5Wxmn4ZtzJFnCtmMI2fSPP3exKXzVw/pHrH+D6JViZHlbGafhm3MkWQ==', + pubKey: 'CAESIMK2YwjZ9I8/d7EpfNXD+kesf4PolWJkeVsZp+GbcyRZ' +}, { + id: '12D3KooWLV3w42LqUb9MWE7oTzG7vwaFjPw9GvDqmsuDif5chTn9', + privKey: 'CAESYI44p8HiCHtCBhuUcetU9XdIEtWvon15a5ZLsfyssSj9nn3mt4oZI0t6wXTHOvIA0GSFWrYkdKp1338oFIambdKefea3ihkjS3rBdMc68gDQZIVatiR0qnXffygUhqZt0g==', + pubKey: 'CAESIJ595reKGSNLesF0xzryANBkhVq2JHSqdd9/KBSGpm3S' +}, { + id: '12D3KooWDRHe5x3tEQfZi4yirsdhA3zgqhE8awxAjET7zVF23JHB', + privKey: 'CAESYP+GrxgDqKnx79W5l4sgpCEYvNF9bBlCSVu3McENPluhNYVEwxo5KboVuOPnYO6ZOeeTmglqc2vcN8pldRF8lq41hUTDGjkpuhW44+dg7pk555OaCWpza9w3ymV1EXyWrg==', + pubKey: 'CAESIDWFRMMaOSm6Fbjj52DumTnnk5oJanNr3DfKZXURfJau' +}, { + id: '12D3KooWQJMnsoT7js35ZgkboxzUjXpVhfvG8cMqZnBJTP4XPuhU', + privKey: 'CAESYL1Fwm/+layh15V1ITWkK9tEQLuGeJFi16VkNDUU+GFs1y90DFs9vlkRziuJFZ/QtEIlYZWjFTsNRJxFA/etwCvXL3QMWz2+WRHOK4kVn9C0QiVhlaMVOw1EnEUD963AKw==', + pubKey: 'CAESINcvdAxbPb5ZEc4riRWf0LRCJWGVoxU7DUScRQP3rcAr' +}, { + id: '12D3KooWFYyvJysHGbbYiruVY8bgjKn7sYN9axgbnMxrWVkGXABF', + privKey: 'CAESYCtlyHA9SQ9F0yO6frmkrFFmboLCzGt8syr0ix8QkuTcVTVAp9JiBXb2xI1lzK6Fn2mRJUxtQIuuW+3V2mu3DZZVNUCn0mIFdvbEjWXMroWfaZElTG1Ai65b7dXaa7cNlg==', + pubKey: 'CAESIFU1QKfSYgV29sSNZcyuhZ9pkSVMbUCLrlvt1dprtw2W' +}, { + id: '12D3KooWHFKTMzwerBtsVmtz4ZZEQy2heafxzWw6wNn5PPYkBxJ5', + privKey: 'CAESYLU/qFxBHsdsQa63w3MrP8VvxJDyAk7rB7gLnIN01CyibmZCtQc7a1gIEDOGb10maUltL8wJxEdmOw3Bpjo7xrpuZkK1BztrWAgQM4ZvXSZpSW0vzAnER2Y7DcGmOjvGug==', + pubKey: 'CAESIG5mQrUHO2tYCBAzhm9dJmlJbS/MCcRHZjsNwaY6O8a6' +}] diff --git a/test/index.spec.js b/test/index.spec.js new file mode 100644 index 0000000..185a111 --- /dev/null +++ b/test/index.spec.js @@ -0,0 +1,66 @@ +'use strict' +/* eslint-env mocha */ + +const { expect } = require('aegir/utils/chai') +const PeerId = require('peer-id') + +const Peers = require('./fixtures/peers') + +const createRelayServer = require('../src') + +describe('Relay Hop Server', () => { + let relay + let peerId + + before(async () => { + peerId = await PeerId.createFromJSON(Peers[0]) + }) + + afterEach(async () => { + relay && await relay.stop() + }) + + it('fails to create a relay if no peerId is provided', async () => { + try { + relay = await createRelayServer({}) + } catch (err) { + expect(err).to.exist() + return + } + throw new Error('should fail to create a relay if no peerId is provided') + }) + + it('can create an start a relay with only a peerId', async () => { + relay = await createRelayServer({ peerId }) + + await relay.start() + }) + + it('can specify listenMultiaddrs for the relay', async () => { + const listenAddresses = ['/ip4/127.0.0.1/tcp/15002/ws', '/ip4/127.0.0.1/tcp/8000'] + + relay = await createRelayServer({ + peerId, + listenAddresses + }) + + await relay.start() + + expect(relay.multiaddrs).to.have.lengthOf(listenAddresses.length) + relay.multiaddrs.forEach((m) => listenAddresses.includes(m.toString())) + }) + + it('can specify announceMultiaddrs for the relay', async () => { + const announceAddresses = ['/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA'] + + relay = await createRelayServer({ + peerId, + announceAddresses + }) + + await relay.start() + + expect(relay.multiaddrs).to.have.lengthOf(announceAddresses.length) + relay.multiaddrs.forEach((m) => announceAddresses.includes(m.toString())) + }) +}) From 34df221298bea290971be618d97bc8c34169ffca Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Tue, 1 Dec 2020 10:32:47 +0100 Subject: [PATCH 04/10] chore: address review --- .env | 1 - README.md | 48 ++++++++++++++++++--------------------- package.json | 8 +++---- src/bin.js | 35 +++++++++-------------------- src/index.js | 50 +++++++++++++++-------------------------- src/utils.js | 56 ++++++++++++++++++++++++++++++++-------------- test/index.spec.js | 37 +++++++++++++++++++++--------- 7 files changed, 119 insertions(+), 116 deletions(-) delete mode 100644 .env diff --git a/.env b/.env deleted file mode 100644 index a6980aa..0000000 --- a/.env +++ /dev/null @@ -1 +0,0 @@ -LISTEN_MULTIADDRS='' \ No newline at end of file diff --git a/README.md b/README.md index 7cfd40e..080a9a3 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ -# js-libp2p-hop-relay-server +# js-libp2p-relay-server [![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://protocol.ai) [![](https://img.shields.io/badge/project-libp2p-yellow.svg?style=flat-square)](http://libp2p.io/) [![](https://img.shields.io/badge/freenode-%23libp2p-yellow.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23libp2p) [![](https://img.shields.io/discourse/https/discuss.libp2p.io/posts.svg)](https://discuss.libp2p.io) -[![GitHub Workflow Status](https://img.shields.io/github/workflow/status/libp2p/js-libp2p-hop-relay-server/ci?label=ci&style=flat-square)](https://github.com/libp2p/js-libp2p-hop-relay-server/actions?query=branch%3Amaster+workflow%3Aci+) +[![GitHub Workflow Status](https://img.shields.io/github/workflow/status/libp2p/js-libp2p-relay-server/ci?label=ci&style=flat-square)](https://github.com/libp2p/js-libp2p-relay-server/actions?query=branch%3Amaster+workflow%3Aci+) > An out of the box libp2p relay server with HOP @@ -37,17 +37,17 @@ You can read more in its [SPEC](https://github.com/libp2p/specs/tree/master/rela ### Install ```bash -> npm install --global libp2p-hop-relay-server +> npm install --global libp2p-relay-server ``` -Now you can use the cli command `libp2p-hop-relay-server` to spawn an out of the box libp2p hop relay server. +Now you can use the cli command `libp2p-relay-server` to spawn an out of the box libp2p relay server. ### CLI -After installing the relay server, you can use its binary. It accepts several arguments: `--peerId`, `--listenMultiaddrs`, `--announceMultiaddrs`, `--metricsMultiaddr`, `--disableMetrics`, `--delegateMultiaddr` and `--disableAdvertise`. +After installing the relay server, you can use its binary. It accepts several arguments: `--peerId`, `--listenMultiaddrs`, `--announceMultiaddrs`, `--metricsMultiaddr`, `--disableMetrics`, `--disablePubsubDiscovery` and `--discoveryTopics`. ```sh -libp2p-hop-relay-server [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] [--metricsMultiaddr ] [--disableMetrics] [--delegateMultiaddr ] [--disableAdvertise] +libp2p-relay-server [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] [--discoveryTopics ... ] ``` #### PeerId @@ -55,7 +55,7 @@ libp2p-hop-relay-server [--peerId ] [--listenMultiaddrs ... < You can create a [PeerId](https://github.com/libp2p/js-peer-id) via its [CLI](https://github.com/libp2p/js-peer-id#cli). ```sh -libp2p-hop-relay-server --peerId id.json +libp2p-relay-server --peerId id.json ``` #### Multiaddrs @@ -63,7 +63,7 @@ libp2p-hop-relay-server --peerId id.json You can specify the libp2p rendezvous server listen and announce multiaddrs. This server is configured with [libp2p-tcp](https://github.com/libp2p/js-libp2p-tcp) and [libp2p-websockets](https://github.com/libp2p/js-libp2p-websockets) and addresses with this transports should be used. It can always be modified via the API. ```sh -libp2p-hop-relay-server --peerId id.json --listenMultiaddrs '/ip4/127.0.0.1/tcp/15002/ws' '/ip4/127.0.0.1/tcp/8000' --announceMultiaddrs '/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' '/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' +libp2p-relay-server --peerId id.json --listenMultiaddrs '/ip4/127.0.0.1/tcp/15002/ws' '/ip4/127.0.0.1/tcp/8000' --announceMultiaddrs '/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' '/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' ``` By default it listens on `/ip4/127.0.0.1/tcp/15002/ws` and has no announce multiaddrs specified. @@ -73,36 +73,32 @@ By default it listens on `/ip4/127.0.0.1/tcp/15002/ws` and has no announce multi Metrics are enabled by default on `/ip4/127.0.0.1/tcp/8003` via Prometheus. This address can also be modified with: ```sh -libp2p-hop-relay-server --metricsMultiaddr '/ip4/127.0.0.1/tcp/8000' +libp2p-relay-server --metricsMultiaddr '/ip4/127.0.0.1/tcp/8000' ``` Moreover, metrics can also be disabled with: ```sh -libp2p-hop-relay-server --disableMetrics +libp2p-relay-server --disableMetrics ``` -#### Advertise +#### Discovery -The relay server will advertise its HOP capability by default, using a delegate node on `/dns4/node0.delegate.ipfs.io/tcp/443/https`. This is important for peers that will try to find HOP relays on the network to bind themselves. - -This advertise can be disabled with: +A discovery module [libp2p/js-libp2p-pubsub-peer-discovery](https://github.com/libp2p/js-libp2p-pubsub-peer-discovery) is configured and enabled by default. It can be disabled with: ```sh -libp2p-hop-relay-server --disableAdvertise +libp2p-relay-server --disablePubsubDiscovery ``` -You can also customize the delegate node to use with: +This discovery module uses `_peer-discovery._p2p._pubsub` as a topic for enabling peer discovery. However, other topics can be used with: ```sh -libp2p-hop-relay-server --delegateMultiaddr '/dns4/node1.delegate.ipfs.io/tcp/443/https' +libp2p-relay-server --discoveryTopics '_peer-discovery._app1._pubsub' '_peer_discovery._app2._pubsub' ``` -Note: In the future this will leverage libp2p's [DHT](https://github.com/libp2p/js-libp2p-kad-dht). - ### Docker -When running the hop relay server in Docker, you can configure the same parameters via environment variables, as follows: +When running the relay server in Docker, you can configure the same parameters via environment variables, as follows: ```sh PEER_ID='./id.json' @@ -110,15 +106,15 @@ LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' ANNOUNCE_MULTIADDRS='/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA,/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' METRICS_MULTIADDR='/ip4/127.0.0.1/tcp/8000' DISABLE_METRICS='true' -DELEGATE_MULTIADDR='/dns4/node1.delegate.ipfs.io/tcp/443/https' -DISABLE_ADVERTISE='true' +DISABLE_PUBSUB_DISCOVERY='true' +DISCOVERY_TOPICS='_peer-discovery._app1._pubsub,_peer_discovery._app2._pubsub' ``` Please note that you should expose expose the used ports with the docker run command. The default ports used are `8003` for the metrics and `150003` for the websockets listener ```sh -docker build NAME -t libp2p-hop-relay -docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -d libp2p-hop-relay +docker build NAME -t libp2p-relay +docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -d libp2p-relay ``` ### SSL @@ -133,7 +129,7 @@ You can debug the relay by setting the `DEBUG` environment variable. For instanc ## Contribute -Feel free to join in. All welcome. Open an [issue](https://github.com/libp2p/js-libp2p-hop-relay-server/issues)! +Feel free to join in. All welcome. Open an [issue](https://github.com/libp2p/js-libp2p-relay-server/issues)! This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). @@ -141,4 +137,4 @@ This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/c ## License -MIT - Protocol Labs 2020 \ No newline at end of file +MIT - Protocol Labs 2020 diff --git a/package.json b/package.json index 5c7ff1a..d62853d 100644 --- a/package.json +++ b/package.json @@ -1,11 +1,11 @@ { - "name": "libp2p-hop-relay-server", + "name": "libp2p-relay-server", "version": "0.0.0", "description": "A out of the box libp2p relay server with HOP", "leadMaintainer": "Vasco Santos ", "main": "src/index.js", "bin": { - "libp2p-hop-relay-server": "src/bin.js" + "libp2p-relay-server": "src/bin.js" }, "scripts": { "lint": "aegir lint", @@ -39,11 +39,11 @@ }, "homepage": "https://github.com/vasco-santos/js-libp2p-hop-relay-server#readme", "dependencies": { - "ipfs-http-client": "^48.1.2", "libp2p": "libp2p/js-libp2p#0.30.x", - "libp2p-delegated-content-routing": "^0.8.0", + "libp2p-gossipsub": "^0.7.0", "libp2p-mplex": "^0.10.1", "libp2p-noise": "^2.0.1", + "libp2p-pubsub-peer-discovery": "^3.0.0", "libp2p-tcp": "^0.15.1", "libp2p-websockets": "^0.14.0", "menoetius": "0.0.2", diff --git a/src/bin.js b/src/bin.js index d2d041c..4ce8c64 100644 --- a/src/bin.js +++ b/src/bin.js @@ -3,12 +3,12 @@ 'use strict' // Usage: $0 [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] -// [--metricsMultiaddr ] [--disableMetrics] [--delegateMultiaddr ] [--disableAdvertise] +// [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] [--discoveryTopics ... ] /* eslint-disable no-console */ const debug = require('debug') -const log = debug('libp2p:hop-relay:bin') +const log = debug('libp2p:relay:bin') const fs = require('fs') const http = require('http') @@ -18,7 +18,7 @@ const argv = require('minimist')(process.argv.slice(2)) const multiaddr = require('multiaddr') const PeerId = require('peer-id') -const { getAnnounceAddresses, getListenAddresses } = require('./utils') +const { getAnnounceAddresses, getListenAddresses, getDiscoveryTopics } = require('./utils') const createRelay = require('./index') async function main () { @@ -35,19 +35,9 @@ async function main () { log(`listenAddresses: ${listenAddresses.map((a) => a)}`) announceAddresses.length && log(`announceAddresses: ${announceAddresses.map((a) => a)}`) - // Should advertise - const shouldAdvertise = !(argv.disableAdvertise || process.env.DISABLE_ADVERTISE) - - // Delegate - let delegateOptions - if (argv.delegateMultiaddr || argv.dm || process.env.DELEGATE_MULTIADDR) { - const delegateAddr = multiaddr(argv.delegateMultiaddr || argv.dm || process.env.DELEGATE_MULTIADDR).toOptions() - delegateOptions = { - host: delegateAddr.host, - protocol: delegateAddr.port === '443' ? 'https' : 'http', - port: delegateAddr.port - } - } + // Discovery + const pubsubDiscoveryEnabled = !(argv.disablePubsubDiscovery || process.env.DISABLE_PUBSUB_DISCOVERY) + const pubsubDiscoveryTopics = getDiscoveryTopics(argv) // PeerId let peerId @@ -66,18 +56,13 @@ async function main () { peerId, listenAddresses, announceAddresses, - shouldAdvertise, - delegateOptions - }) - - relay.peerStore.on('change:multiaddrs', ({ peerId: changedPeerId, multiaddrs }) => { - if (peerId.equals(changedPeerId)) { - console.log('Relay server listening on:') - multiaddrs.forEach((m) => console.log(m)) - } + pubsubDiscoveryEnabled, + pubsubDiscoveryTopics }) await relay.start() + console.log('Relay server listening on:') + relay.multiaddrs.forEach((m) => console.log(m)) if (metrics) { log('enabling metrics') diff --git a/src/index.js b/src/index.js index 68328fa..af614ac 100644 --- a/src/index.js +++ b/src/index.js @@ -5,33 +5,20 @@ const TCP = require('libp2p-tcp') const Websockets = require('libp2p-websockets') const Muxer = require('libp2p-mplex') const { NOISE: Crypto } = require('libp2p-noise') -const DelegatedContentRouting = require('libp2p-delegated-content-routing') -const ipfsHttpClient = require('ipfs-http-client') +const GossipSub = require('libp2p-gossipsub') +const PubsubPeerDiscovery = require('libp2p-pubsub-peer-discovery') /** * @typedef {import('peer-id')} PeerId */ -/** - * @typedef {Object} DelegateOptions - * @property {string} host - * @property {string} protocol - * @property {number} port - */ - -const defaulDelegateOptions = { - host: 'node0.delegate.ipfs.io', - protocol: 'https', - port: 443 -} - /** * @typedef {Object} HopRelayOptions - * @property {PeerId} peerId - * @property {DelegateOptions} [delegateOptions = defaulDelegateOptions] + * @property {PeerId} [peerId] * @property {string[]} [listenAddresses = []] * @property {string[]} [announceAddresses = []] - * @property {boolean} [shouldAdvertise = true] + * @property {boolean} [pubsubDiscoveryEnabled = true] + * @property {string[]} [pubsubDiscoveryTopics = ['_peer-discovery._p2p._pubsub']] uses discovery default */ /** @@ -40,35 +27,34 @@ const defaulDelegateOptions = { * @param {HopRelayOptions} options * @returns {Promise} */ -function create ({ peerId, delegateOptions = defaulDelegateOptions, listenAddresses = [], announceAddresses = [], shouldAdvertise = true }) { - const contentRouting = [] - - if (shouldAdvertise) { - const httpClient = ipfsHttpClient(delegateOptions) - contentRouting.push(new DelegatedContentRouting(peerId, httpClient)) - } - +function create ({ peerId, listenAddresses = [], announceAddresses = [], pubsubDiscoveryEnabled = true, pubsubDiscoveryTopics = ['_peer-discovery._p2p._pubsub'] }) { return Libp2p.create({ peerId, modules: { transport: [Websockets, TCP], streamMuxer: [Muxer], connEncryption: [Crypto], - contentRouting + pubsub: GossipSub, + peerDiscovery: [PubsubPeerDiscovery] }, addresses: { listen: listenAddresses, announce: announceAddresses }, config: { + pubsub: { + enabled: pubsubDiscoveryEnabled + }, + peerDiscovery: { + [PubsubPeerDiscovery.tag]: { + topics: pubsubDiscoveryTopics, + enabled: pubsubDiscoveryEnabled + } + }, relay: { enabled: true, // Allows you to dial and accept relayed connections. Does not make you a relay. hop: { - enabled: true, // Allows you to be a relay for other peers - active: true // You will attempt to dial destination peers if you are not connected to them - }, - advertise: { - enabled: shouldAdvertise // Allows you to advertise the Hop service + enabled: true // Allows you to be a relay for other peers } } } diff --git a/src/utils.js b/src/utils.js index a380b59..587a221 100644 --- a/src/utils.js +++ b/src/utils.js @@ -1,5 +1,35 @@ 'use strict' +function getExtraParams (alias1, alias2) { + const params = [] + + const flagIndex = process.argv.findIndex((e) => e === alias1 || e === alias2) + const tmpEndIndex = process.argv.slice(flagIndex + 1).findIndex((e) => e.startsWith('--')) + const endIndex = tmpEndIndex !== -1 ? tmpEndIndex : process.argv.length - flagIndex - 1 + + for (let i = flagIndex + 1; i < flagIndex + endIndex; i++) { + params.push(process.argv[i + 1]) + } + + return params +} + +function getDiscoveryTopics (argv) { + let discoveryTopics = ['_peer-discovery._p2p._pubsub'] + + const argvTopic = argv.discoveryTopics || argv.dt + if (argvTopic) { + discoveryTopics = [argvTopic] + + const extraParams = getExtraParams('--discoveryTopics', '--dt') + extraParams.forEach((p) => discoveryTopics.push(p)) + } else if (process.env.DISCOVERY_TOPICS) { + discoveryTopics = process.env.DISCOVERY_TOPICS.split(',') + } + + return discoveryTopics +} + function getAnnounceAddresses (argv) { let announceAddresses = [] const argvAddr = argv.announceMultiaddrs || argv.am @@ -7,13 +37,8 @@ function getAnnounceAddresses (argv) { if (argvAddr) { announceAddresses = [argvAddr] - const flagIndex = process.argv.findIndex((e) => e === '--announceMultiaddrs' || e === '--am') - const tmpEndIndex = process.argv.slice(flagIndex + 1).findIndex((e) => e.startsWith('--')) - const endIndex = tmpEndIndex !== -1 ? tmpEndIndex : process.argv.length - flagIndex - 1 - - for (let i = flagIndex + 1; i < flagIndex + endIndex; i++) { - announceAddresses.push(process.argv[i + 1]) - } + const extraParams = getExtraParams('--announceMultiaddrs', '--am') + extraParams.forEach((p) => announceAddresses.push(p)) } else if (process.env.ANNOUNCE_MULTIADDRS) { announceAddresses = process.env.ANNOUNCE_MULTIADDRS.split(',') } @@ -21,8 +46,6 @@ function getAnnounceAddresses (argv) { return announceAddresses } -module.exports.getAnnounceAddresses = getAnnounceAddresses - function getListenAddresses (argv) { let listenAddresses = ['/ip4/127.0.0.1/tcp/15003/ws'] const argvAddr = argv.listenMultiaddrs || argv.lm @@ -30,13 +53,8 @@ function getListenAddresses (argv) { if (argvAddr) { listenAddresses = [argvAddr] - const flagIndex = process.argv.findIndex((e) => e === '--listenMultiaddrs' || e === '--lm') - const tmpEndIndex = process.argv.slice(flagIndex + 1).findIndex((e) => e.startsWith('--')) - const endIndex = tmpEndIndex !== -1 ? tmpEndIndex : process.argv.length - flagIndex - 1 - - for (let i = flagIndex + 1; i < flagIndex + endIndex; i++) { - listenAddresses.push(process.argv[i + 1]) - } + const extraParams = getExtraParams('--listenMultiaddrs', '--lm') + extraParams.forEach((p) => listenAddresses.push(p)) } else if (process.env.LISTEN_MULTIADDRS) { listenAddresses = process.env.LISTEN_MULTIADDRS.split(',') } @@ -44,4 +62,8 @@ function getListenAddresses (argv) { return listenAddresses } -module.exports.getListenAddresses = getListenAddresses +module.exports = { + getDiscoveryTopics, + getAnnounceAddresses, + getListenAddresses +} diff --git a/test/index.spec.js b/test/index.spec.js index 185a111..6b942f4 100644 --- a/test/index.spec.js +++ b/test/index.spec.js @@ -8,7 +8,7 @@ const Peers = require('./fixtures/peers') const createRelayServer = require('../src') -describe('Relay Hop Server', () => { +describe('Relay Server', () => { let relay let peerId @@ -20,16 +20,6 @@ describe('Relay Hop Server', () => { relay && await relay.stop() }) - it('fails to create a relay if no peerId is provided', async () => { - try { - relay = await createRelayServer({}) - } catch (err) { - expect(err).to.exist() - return - } - throw new Error('should fail to create a relay if no peerId is provided') - }) - it('can create an start a relay with only a peerId', async () => { relay = await createRelayServer({ peerId }) @@ -63,4 +53,29 @@ describe('Relay Hop Server', () => { expect(relay.multiaddrs).to.have.lengthOf(announceAddresses.length) relay.multiaddrs.forEach((m) => announceAddresses.includes(m.toString())) }) + + it('can specify discovery topics to discover', async () => { + const topics = ['_peer-discovery._app1._pubsub', '_peer_discovery._app2._pubsub'] + relay = await createRelayServer({ + peerId, + pubsubDiscoveryTopics: topics + }) + + await relay.start() + + const subsTopics = relay.pubsub.getTopics() + expect(subsTopics).to.have.lengthOf(topics.length) + subsTopics.forEach((m) => topics.includes(m.toString())) + }) + + it('can disable discovery', async () => { + relay = await createRelayServer({ + peerId, + pubsubDiscoveryEnabled: false + }) + + await relay.start() + + expect(relay._discovery.size).to.eql(0) + }) }) From b679196beff1e95eb343da60bad4fbc1e39b7624 Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Tue, 1 Dec 2020 15:12:54 +0100 Subject: [PATCH 05/10] chore: update docker node image and remove custom topics --- Dockerfile | 4 ++-- README.md | 16 ++++++---------- src/bin.js | 8 +++----- src/utils.js | 16 ---------------- 4 files changed, 11 insertions(+), 33 deletions(-) diff --git a/Dockerfile b/Dockerfile index fac0710..2bf03c7 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,7 +1,7 @@ -FROM node:lts-buster +FROM node:lts-alpine # Install deps -RUN apt-get update && apt-get install -y +RUN apk add --update git build-base python3 libressl-dev ca-certificates # Get dumb-init to allow quit running interactively RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 && chmod +x /usr/local/bin/dumb-init diff --git a/README.md b/README.md index 080a9a3..e7747a1 100644 --- a/README.md +++ b/README.md @@ -44,10 +44,10 @@ Now you can use the cli command `libp2p-relay-server` to spawn an out of the box ### CLI -After installing the relay server, you can use its binary. It accepts several arguments: `--peerId`, `--listenMultiaddrs`, `--announceMultiaddrs`, `--metricsMultiaddr`, `--disableMetrics`, `--disablePubsubDiscovery` and `--discoveryTopics`. +After installing the relay server, you can use its binary. It accepts several arguments: `--peerId`, `--listenMultiaddrs`, `--announceMultiaddrs`, `--metricsMultiaddr`, `--disableMetrics` and `--disablePubsubDiscovery`. ```sh -libp2p-relay-server [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] [--discoveryTopics ... ] +libp2p-relay-server [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] ``` #### PeerId @@ -66,7 +66,7 @@ You can specify the libp2p rendezvous server listen and announce multiaddrs. Thi libp2p-relay-server --peerId id.json --listenMultiaddrs '/ip4/127.0.0.1/tcp/15002/ws' '/ip4/127.0.0.1/tcp/8000' --announceMultiaddrs '/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' '/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' ``` -By default it listens on `/ip4/127.0.0.1/tcp/15002/ws` and has no announce multiaddrs specified. +By default it listens on `/ip4/127.0.0.1/tcp/15003/ws` and has no announce multiaddrs specified. #### Metrics @@ -90,12 +90,6 @@ A discovery module [libp2p/js-libp2p-pubsub-peer-discovery](https://github.com/l libp2p-relay-server --disablePubsubDiscovery ``` -This discovery module uses `_peer-discovery._p2p._pubsub` as a topic for enabling peer discovery. However, other topics can be used with: - -```sh -libp2p-relay-server --discoveryTopics '_peer-discovery._app1._pubsub' '_peer_discovery._app2._pubsub' -``` - ### Docker When running the relay server in Docker, you can configure the same parameters via environment variables, as follows: @@ -110,7 +104,9 @@ DISABLE_PUBSUB_DISCOVERY='true' DISCOVERY_TOPICS='_peer-discovery._app1._pubsub,_peer_discovery._app2._pubsub' ``` -Please note that you should expose expose the used ports with the docker run command. The default ports used are `8003` for the metrics and `150003` for the websockets listener +Please note that you should expose expose the used ports with the docker run command. The default ports used are `8003` for the metrics and `150003` for the websockets listener. + +Configuration example: ```sh docker build NAME -t libp2p-relay diff --git a/src/bin.js b/src/bin.js index 4ce8c64..14f44c3 100644 --- a/src/bin.js +++ b/src/bin.js @@ -3,7 +3,7 @@ 'use strict' // Usage: $0 [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] -// [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] [--discoveryTopics ... ] +// [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] /* eslint-disable no-console */ @@ -18,7 +18,7 @@ const argv = require('minimist')(process.argv.slice(2)) const multiaddr = require('multiaddr') const PeerId = require('peer-id') -const { getAnnounceAddresses, getListenAddresses, getDiscoveryTopics } = require('./utils') +const { getAnnounceAddresses, getListenAddresses } = require('./utils') const createRelay = require('./index') async function main () { @@ -37,7 +37,6 @@ async function main () { // Discovery const pubsubDiscoveryEnabled = !(argv.disablePubsubDiscovery || process.env.DISABLE_PUBSUB_DISCOVERY) - const pubsubDiscoveryTopics = getDiscoveryTopics(argv) // PeerId let peerId @@ -56,8 +55,7 @@ async function main () { peerId, listenAddresses, announceAddresses, - pubsubDiscoveryEnabled, - pubsubDiscoveryTopics + pubsubDiscoveryEnabled }) await relay.start() diff --git a/src/utils.js b/src/utils.js index 587a221..7bc3626 100644 --- a/src/utils.js +++ b/src/utils.js @@ -14,22 +14,6 @@ function getExtraParams (alias1, alias2) { return params } -function getDiscoveryTopics (argv) { - let discoveryTopics = ['_peer-discovery._p2p._pubsub'] - - const argvTopic = argv.discoveryTopics || argv.dt - if (argvTopic) { - discoveryTopics = [argvTopic] - - const extraParams = getExtraParams('--discoveryTopics', '--dt') - extraParams.forEach((p) => discoveryTopics.push(p)) - } else if (process.env.DISCOVERY_TOPICS) { - discoveryTopics = process.env.DISCOVERY_TOPICS.split(',') - } - - return discoveryTopics -} - function getAnnounceAddresses (argv) { let announceAddresses = [] const argvAddr = argv.announceMultiaddrs || argv.am From cdbc8f38df9dbbbb74938f34285ac90a632c5fce Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Tue, 1 Dec 2020 15:19:46 +0100 Subject: [PATCH 06/10] chore: change readme structure to divide between required/recommended cli and docker params and optional --- README.md | 55 ++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 40 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index e7747a1..908574c 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,10 @@ - [CLI](#cli) - [Docker](#docker) - [SSL](#ssl) +- [Configuration](#configuration) + - [Metrics](#metrics) + - [Discovery](#discovery) + - [Docker](#docker) - [Debug](#debug) - [Contribute](#contribute) - [License](#license) @@ -50,6 +54,8 @@ After installing the relay server, you can use its binary. It accepts several ar libp2p-relay-server [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] ``` +The main configuration you should focus are the `PeerId` and the `Multiaddrs`, which are detailed next. + #### PeerId You can create a [PeerId](https://github.com/libp2p/js-peer-id) via its [CLI](https://github.com/libp2p/js-peer-id#cli). @@ -68,7 +74,36 @@ libp2p-relay-server --peerId id.json --listenMultiaddrs '/ip4/127.0.0.1/tcp/1500 By default it listens on `/ip4/127.0.0.1/tcp/15003/ws` and has no announce multiaddrs specified. -#### Metrics +### Docker + +When running the relay server in Docker, you can configure the same parameters via environment variables, as follows: + +```sh +PEER_ID='./id.json' +LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' +ANNOUNCE_MULTIADDRS='/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA,/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' +``` + +Please note that you should expose expose the used ports with the docker run command. The default ports used are `8003` for the metrics and `150003` for the websockets listener. + +Example: + +```sh +docker build NAME -t libp2p-relay +docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -d libp2p-relay +``` + +### SSL + +You should setup an SSL certificate with nginx and proxy to the API. You can use a service that already offers an SSL certificate with the server and configure nginx, or you can create valid certificates with for example [Letsencrypt](https://certbot.eff.org/lets-encrypt/osx-nginx). Letsencrypt won’t give you a cert for an IP address (yet) so you need to connect via SSL to a domain name. + +With this, you should specify in your relay the announce multiaddrs for your listening transports. This is specially important for browser peers that will leverage this relay, as browser nodes can only dial peers behind a `DNS+WSS` multiaddr. + +## Configuration + +Besides the `PeerId` and `Multiaddrs`, this server can also have other configurations for fine tuning. + +### Metrics Metrics are enabled by default on `/ip4/127.0.0.1/tcp/8003` via Prometheus. This address can also be modified with: @@ -82,7 +117,7 @@ Moreover, metrics can also be disabled with: libp2p-relay-server --disableMetrics ``` -#### Discovery +### Discovery A discovery module [libp2p/js-libp2p-pubsub-peer-discovery](https://github.com/libp2p/js-libp2p-pubsub-peer-discovery) is configured and enabled by default. It can be disabled with: @@ -92,33 +127,23 @@ libp2p-relay-server --disablePubsubDiscovery ### Docker -When running the relay server in Docker, you can configure the same parameters via environment variables, as follows: +When running the relay server in Docker, you can configure other parameters via environment variables, as follows: ```sh -PEER_ID='./id.json' -LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -ANNOUNCE_MULTIADDRS='/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA,/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' METRICS_MULTIADDR='/ip4/127.0.0.1/tcp/8000' DISABLE_METRICS='true' DISABLE_PUBSUB_DISCOVERY='true' -DISCOVERY_TOPICS='_peer-discovery._app1._pubsub,_peer_discovery._app2._pubsub' ``` Please note that you should expose expose the used ports with the docker run command. The default ports used are `8003` for the metrics and `150003` for the websockets listener. -Configuration example: +Example: ```sh docker build NAME -t libp2p-relay -docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -d libp2p-relay +docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -e METRICS_MULTIADDR='/ip4/127.0.0.1/tcp/8000' -e DISABLE_PUBSUB_DISCOVERY='true' -d libp2p-relay ``` -### SSL - -You should setup an SSL certificate with nginx and proxy to the API. You can use a service that already offers an SSL certificate with the server and configure nginx, or you can create valid certificates with for example [Letsencrypt](https://certbot.eff.org/lets-encrypt/osx-nginx). Letsencrypt won’t give you a cert for an IP address (yet) so you need to connect via SSL to a domain name. - -With this, you should specify in your relay the announce multiaddrs for your listening transports. This is specially important for browser peers that will leverage this relay, as browser nodes can only dial peers behind a `DNS+WSS` multiaddr. - ### Debug You can debug the relay by setting the `DEBUG` environment variable. For instance, you can set it to `libp2p*`. From f99d5a95f2b54baf47183a92c081ea040feaf821 Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Tue, 1 Dec 2020 15:21:55 +0100 Subject: [PATCH 07/10] chore: fix ci removing custom topics remainings --- src/utils.js | 1 - test/index.spec.js | 14 -------------- 2 files changed, 15 deletions(-) diff --git a/src/utils.js b/src/utils.js index 7bc3626..bf41ce2 100644 --- a/src/utils.js +++ b/src/utils.js @@ -47,7 +47,6 @@ function getListenAddresses (argv) { } module.exports = { - getDiscoveryTopics, getAnnounceAddresses, getListenAddresses } diff --git a/test/index.spec.js b/test/index.spec.js index 6b942f4..a9e1eae 100644 --- a/test/index.spec.js +++ b/test/index.spec.js @@ -54,20 +54,6 @@ describe('Relay Server', () => { relay.multiaddrs.forEach((m) => announceAddresses.includes(m.toString())) }) - it('can specify discovery topics to discover', async () => { - const topics = ['_peer-discovery._app1._pubsub', '_peer_discovery._app2._pubsub'] - relay = await createRelayServer({ - peerId, - pubsubDiscoveryTopics: topics - }) - - await relay.start() - - const subsTopics = relay.pubsub.getTopics() - expect(subsTopics).to.have.lengthOf(topics.length) - subsTopics.forEach((m) => topics.includes(m.toString())) - }) - it('can disable discovery', async () => { relay = await createRelayServer({ peerId, From a482dce766ebec86b8f54d1fa6f99b682e898bc7 Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Tue, 1 Dec 2020 15:36:30 +0100 Subject: [PATCH 08/10] chore: remove unecessary docker deps install --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 2bf03c7..71d83dd 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,7 +1,7 @@ FROM node:lts-alpine # Install deps -RUN apk add --update git build-base python3 libressl-dev ca-certificates +RUN apk add --update git build-base python3 # Get dumb-init to allow quit running interactively RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 && chmod +x /usr/local/bin/dumb-init From 6ab04d4f1c4b6794ec4d320ad46c3af495179997 Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Fri, 4 Dec 2020 13:56:41 +0100 Subject: [PATCH 09/10] chore: apply suggestions from code review Co-authored-by: Jacob Heun --- README.md | 53 +++++++++++++++++++++++----------------------- src/bin.js | 10 ++++----- src/index.js | 8 +++---- src/utils.js | 2 +- test/index.spec.js | 10 ++++++--- 5 files changed, 43 insertions(+), 40 deletions(-) diff --git a/README.md b/README.md index 908574c..01fed27 100644 --- a/README.md +++ b/README.md @@ -48,49 +48,57 @@ Now you can use the cli command `libp2p-relay-server` to spawn an out of the box ### CLI -After installing the relay server, you can use its binary. It accepts several arguments: `--peerId`, `--listenMultiaddrs`, `--announceMultiaddrs`, `--metricsMultiaddr`, `--disableMetrics` and `--disablePubsubDiscovery`. +After installing the relay server, you can use its binary. It accepts several arguments: `--peerId`, `--listenMultiaddrs`, `--announceMultiaddrs`, `--metricsPort`, `--disableMetrics` and `--disablePubsubDiscovery`. ```sh -libp2p-relay-server [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] +libp2p-relay-server [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] [--metricsPort ] [--disableMetrics] [--disablePubsubDiscovery] ``` -The main configuration you should focus are the `PeerId` and the `Multiaddrs`, which are detailed next. +The main configuration you *should* include are the `PeerId` and `Multiaddrs`, which are detailed next. Using a consistent PeerId will ensure your node's identity is consistent across restarts, and the Multiaddrs will allow you to appropriate bind your local and external addresses so that other peers can connect to you. #### PeerId You can create a [PeerId](https://github.com/libp2p/js-peer-id) via its [CLI](https://github.com/libp2p/js-peer-id#cli). +Once you have a generated PeerId json file, you can start the relay with that PeerId by specifying its path via the `--peerId` flag: ```sh -libp2p-relay-server --peerId id.json +libp2p-relay-server --peerId /path/to/peer-id.json ``` #### Multiaddrs -You can specify the libp2p rendezvous server listen and announce multiaddrs. This server is configured with [libp2p-tcp](https://github.com/libp2p/js-libp2p-tcp) and [libp2p-websockets](https://github.com/libp2p/js-libp2p-websockets) and addresses with this transports should be used. It can always be modified via the API. +You can specify the libp2p rendezvous server listen and announce multiaddrs. This server is configured with [libp2p-tcp](https://github.com/libp2p/js-libp2p-tcp) and [libp2p-websockets](https://github.com/libp2p/js-libp2p-websockets), so you will only be able to specify addresses for these transports. ```sh -libp2p-relay-server --peerId id.json --listenMultiaddrs '/ip4/127.0.0.1/tcp/15002/ws' '/ip4/127.0.0.1/tcp/8000' --announceMultiaddrs '/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' '/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' +libp2p-relay-server --peerId /path/to/peer-id.json --listenMultiaddrs '/ip4/127.0.0.1/tcp/15002/ws' '/ip4/127.0.0.1/tcp/8001' --announceMultiaddrs '/dns4/test.io/tcp' '/dns4/test.io/tcp/443/wss' ``` -By default it listens on `/ip4/127.0.0.1/tcp/15003/ws` and has no announce multiaddrs specified. +By default it listens on `/ip4/127.0.0.1/tcp/8000` and `/ip4/127.0.0.1/tcp/15003/ws`. It has no announce multiaddrs specified. ### Docker When running the relay server in Docker, you can configure the same parameters via environment variables, as follows: ```sh -PEER_ID='./id.json' -LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -ANNOUNCE_MULTIADDRS='/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA,/dns6/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA' +PEER_ID='/var/local/myapp/id.json' +LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8001' +ANNOUNCE_MULTIADDRS='/dns4/test.io/tcp/443/wss,/dns6/test.io/tcp/443/wss' ``` -Please note that you should expose expose the used ports with the docker run command. The default ports used are `8003` for the metrics and `150003` for the websockets listener. +Docker does not provide a clean way of adding a file using the RUN command. As a file should be added before running the server, the easiest way is to create a `Dockerfile` that extends this Docker image and copy a file to a path that you can reference. + +``` +FROM libp2p/libp2p-relay-server +COPY file /id.json /var/local/myapp/ +``` + +Please note that you should expose the listening ports with the docker run command. The default ports used are `8003` for the metrics, `8000` for the tcp listener and `150003` for the websockets listener. Example: ```sh -docker build NAME -t libp2p-relay -docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -d libp2p-relay +docker build . -t libp2p-relay +docker run -p 8003:8003 -p 15002:15002 -p 8001:8001 -e PEER_ID='/var/local/myapp/id.json' -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8001' -e ANNOUNCE_MULTIADDRS='/dns4/localhost/tcp/8000,/dns4/localhost/tcp/15002/ws' -d libp2p-relay ``` ### SSL @@ -105,10 +113,10 @@ Besides the `PeerId` and `Multiaddrs`, this server can also have other configura ### Metrics -Metrics are enabled by default on `/ip4/127.0.0.1/tcp/8003` via Prometheus. This address can also be modified with: +Metrics are enabled by default on `/ip4/127.0.0.1/tcp/8003` via Prometheus. This port can also be modified with: ```sh -libp2p-relay-server --metricsMultiaddr '/ip4/127.0.0.1/tcp/8000' +libp2p-relay-server --metricsPort '8008' ``` Moreover, metrics can also be disabled with: @@ -127,26 +135,19 @@ libp2p-relay-server --disablePubsubDiscovery ### Docker -When running the relay server in Docker, you can configure other parameters via environment variables, as follows: +On docker you can also specify the configurations above with the following environment variables: ```sh -METRICS_MULTIADDR='/ip4/127.0.0.1/tcp/8000' +METRICS_PORT='8008' DISABLE_METRICS='true' DISABLE_PUBSUB_DISCOVERY='true' ``` -Please note that you should expose expose the used ports with the docker run command. The default ports used are `8003` for the metrics and `150003` for the websockets listener. - -Example: - -```sh -docker build NAME -t libp2p-relay -docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8000' -e METRICS_MULTIADDR='/ip4/127.0.0.1/tcp/8000' -e DISABLE_PUBSUB_DISCOVERY='true' -d libp2p-relay -``` +Please note that you should expose expose the used ports with the docker run command. ### Debug -You can debug the relay by setting the `DEBUG` environment variable. For instance, you can set it to `libp2p*`. +You can debug the relay by setting the `DEBUG` environment variable. For instance, you can set it to `libp2p*`. These logs can be noisy so you may wish to tune the namespaces that are logging, see the [Debug module](https://github.com/visionmedia/debug) for detailed usage. ## Contribute diff --git a/src/bin.js b/src/bin.js index 14f44c3..6713ea4 100644 --- a/src/bin.js +++ b/src/bin.js @@ -3,7 +3,7 @@ 'use strict' // Usage: $0 [--peerId ] [--listenMultiaddrs ... ] [--announceMultiaddrs ... ] -// [--metricsMultiaddr ] [--disableMetrics] [--disablePubsubDiscovery] +// [--metricsPort ] [--disableMetrics] [--disablePubsubDiscovery] /* eslint-disable no-console */ @@ -15,7 +15,6 @@ const http = require('http') const menoetius = require('menoetius') const argv = require('minimist')(process.argv.slice(2)) -const multiaddr = require('multiaddr') const PeerId = require('peer-id') const { getAnnounceAddresses, getListenAddresses } = require('./utils') @@ -25,8 +24,7 @@ async function main () { // Metrics let metricsServer const metrics = !(argv.disableMetrics || process.env.DISABLE_METRICS) - const metricsMa = multiaddr(argv.metricsMultiaddr || argv.ma || process.env.METRICS_MULTIADDR || '/ip4/0.0.0.0/tcp/8003') - const metricsAddr = metricsMa.nodeAddress() + const metricsPort = argv.metricsPort || argv.mp || process.env.METRICS_PORT || '8003' // multiaddrs const listenAddresses = getListenAddresses(argv) @@ -73,8 +71,8 @@ async function main () { menoetius.instrument(metricsServer) - metricsServer.listen(metricsAddr.port, metricsAddr.address, () => { - console.log(`metrics server listening on ${metricsAddr.port}`) + metricsServer.listen(metricsPort, '0.0.0.0', () => { + console.log(`metrics server listening on ${metricsPort}`) }) } diff --git a/src/index.js b/src/index.js index af614ac..b39f2ec 100644 --- a/src/index.js +++ b/src/index.js @@ -3,8 +3,8 @@ const Libp2p = require('libp2p') const TCP = require('libp2p-tcp') const Websockets = require('libp2p-websockets') -const Muxer = require('libp2p-mplex') -const { NOISE: Crypto } = require('libp2p-noise') +const MPLEX = require('libp2p-mplex') +const { NOISE } = require('libp2p-noise') const GossipSub = require('libp2p-gossipsub') const PubsubPeerDiscovery = require('libp2p-pubsub-peer-discovery') @@ -32,8 +32,8 @@ function create ({ peerId, listenAddresses = [], announceAddresses = [], pubsubD peerId, modules: { transport: [Websockets, TCP], - streamMuxer: [Muxer], - connEncryption: [Crypto], + streamMuxer: [MPLEX], + connEncryption: [NOISE], pubsub: GossipSub, peerDiscovery: [PubsubPeerDiscovery] }, diff --git a/src/utils.js b/src/utils.js index bf41ce2..0d5dfbd 100644 --- a/src/utils.js +++ b/src/utils.js @@ -31,7 +31,7 @@ function getAnnounceAddresses (argv) { } function getListenAddresses (argv) { - let listenAddresses = ['/ip4/127.0.0.1/tcp/15003/ws'] + let listenAddresses = ['/ip4/127.0.0.1/tcp/15003/ws', '/ip4/127.0.0.1/tcp/8000'] const argvAddr = argv.listenMultiaddrs || argv.lm if (argvAddr) { diff --git a/test/index.spec.js b/test/index.spec.js index a9e1eae..bc4583d 100644 --- a/test/index.spec.js +++ b/test/index.spec.js @@ -37,11 +37,13 @@ describe('Relay Server', () => { await relay.start() expect(relay.multiaddrs).to.have.lengthOf(listenAddresses.length) - relay.multiaddrs.forEach((m) => listenAddresses.includes(m.toString())) + relay.multiaddrs.forEach((m) => { + expect(listenAddresses.includes(m.toString())).to.eql(true) + }) }) it('can specify announceMultiaddrs for the relay', async () => { - const announceAddresses = ['/dns4/test.io/tcp/443/wss/p2p/12D3KooWAuEpJKhCAfNcHycKcZCv9Qy69utLAJ3MobjKpsoKbrGA'] + const announceAddresses = ['/dns4/test.io/tcp/443/wss'] relay = await createRelayServer({ peerId, @@ -51,7 +53,9 @@ describe('Relay Server', () => { await relay.start() expect(relay.multiaddrs).to.have.lengthOf(announceAddresses.length) - relay.multiaddrs.forEach((m) => announceAddresses.includes(m.toString())) + relay.multiaddrs.forEach((m) => { + expect(announceAddresses.includes(m.toString())).to.eql(true) + }) }) it('can disable discovery', async () => { From 0b6238db90ea05c0b24d3e2030fc04c289e6d3de Mon Sep 17 00:00:00 2001 From: Vasco Santos Date: Mon, 7 Dec 2020 10:10:15 +0100 Subject: [PATCH 10/10] chore: improve peer id usage with docker example --- README.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 01fed27..256068c 100644 --- a/README.md +++ b/README.md @@ -61,8 +61,10 @@ The main configuration you *should* include are the `PeerId` and `Multiaddrs`, w You can create a [PeerId](https://github.com/libp2p/js-peer-id) via its [CLI](https://github.com/libp2p/js-peer-id#cli). Once you have a generated PeerId json file, you can start the relay with that PeerId by specifying its path via the `--peerId` flag: + ```sh -libp2p-relay-server --peerId /path/to/peer-id.json +peer-id --type=ed25519 > id.json +libp2p-relay-server --peerId ./id.json ``` #### Multiaddrs @@ -80,25 +82,24 @@ By default it listens on `/ip4/127.0.0.1/tcp/8000` and `/ip4/127.0.0.1/tcp/15003 When running the relay server in Docker, you can configure the same parameters via environment variables, as follows: ```sh -PEER_ID='/var/local/myapp/id.json' +PEER_ID='/etc/opt/relay/id.json' LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8001' ANNOUNCE_MULTIADDRS='/dns4/test.io/tcp/443/wss,/dns6/test.io/tcp/443/wss' ``` -Docker does not provide a clean way of adding a file using the RUN command. As a file should be added before running the server, the easiest way is to create a `Dockerfile` that extends this Docker image and copy a file to a path that you can reference. - -``` -FROM libp2p/libp2p-relay-server -COPY file /id.json /var/local/myapp/ -``` - Please note that you should expose the listening ports with the docker run command. The default ports used are `8003` for the metrics, `8000` for the tcp listener and `150003` for the websockets listener. Example: ```sh +peer-id --type=ed25519 > id.json docker build . -t libp2p-relay -docker run -p 8003:8003 -p 15002:15002 -p 8001:8001 -e PEER_ID='/var/local/myapp/id.json' -e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/15002/ws,/ip4/127.0.0.1/tcp/8001' -e ANNOUNCE_MULTIADDRS='/dns4/localhost/tcp/8000,/dns4/localhost/tcp/15002/ws' -d libp2p-relay +docker run -p 8003:8003 -p 15002:15002 -p 8000:8000 \ +-e LISTEN_MULTIADDRS='/ip4/127.0.0.1/tcp/8000,/ip4/127.0.0.1/tcp/15002/ws' \ +-e ANNOUNCE_MULTIADDRS='/dns4/localhost/tcp/8000,/dns4/localhost/tcp/15002/ws' \ +-e PEER_ID='/etc/opt/relay/id.json' \ +-v $PWD/id.json:/etc/opt/relay/id.json \ +-d libp2p-relay ``` ### SSL