From 34a069ef077a408d742353d6b77888924a2aff27 Mon Sep 17 00:00:00 2001 From: achingbrain Date: Thu, 20 Jul 2023 14:49:05 +0100 Subject: [PATCH 1/5] docs: add migration guide for libp2p@0.46.x Adds migration guide detailing breaking changes in libp2p@0.46.x --- doc/migrations/v0.45-v0.46.md | 119 ++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+) create mode 100644 doc/migrations/v0.45-v0.46.md diff --git a/doc/migrations/v0.45-v0.46.md b/doc/migrations/v0.45-v0.46.md new file mode 100644 index 0000000000..8f6077f5b7 --- /dev/null +++ b/doc/migrations/v0.45-v0.46.md @@ -0,0 +1,119 @@ +# Migrating to libp2p@46 + +A migration guide for refactoring your application code from libp2p v__ to v__. + +## Table of Contents + +- [Graceful stream closing](#graceful-stream-closing) +- [Stream/Connection stat properties](#streamconnection-stat-properties) +- [Interface module consolidation](#interface-module-consolidation) + +## Graceful stream closing + +Streams can either be closed gracefully, where we wait for any unsent data to be sent, or aborted in which case any unsent data is discarded and a reset message is sent, notifying the remote of the abnormal termination. + +To close a stream gracefully we call the `.close` method (or `.closeRead`/`.closeWrite` for when we want half-closed streams). To abort a stream we call `.abort` and pass an error object. + +In previous versions the `.close` method was synchronous which meant it could not wait for existing data to be sent which made nodes behave unpredictably. + +From `0.46.x` the `.close`/`.closeRead`/`.closeWrite` methods on the Stream interface are now asynchronous. `.abort` is a synchronous method that accepts an Error object. + +Similarly the Connection interface now has asynchronous `.close` and synchronous `.abort` methods. + +The `.reset` method has been removed from the Stream interface as it is only to be invoked internally by stream multiplexers when a remote stream reset has occurred. + +**Before** + +```js +const stream = await libp2p.dialProtocol(multiaddr, '/my-protocol/1.0.0') + +// send some data +await stream.sink([data]) + +// close the stream - previously this may not have waited for the data to be sent +stream.close() + +// alternatively cause the stream to error on the remote +stream.abort(new Error('Oh no!')) +``` + +**After** + +```js +const stream = await libp2p.dialProtocol(multiaddr, '/my-protocol/1.0.0') + +// send some data +await stream.sink([data]) + +// close the stream - this method is now async +await stream.close() + +// alternatively cause the stream to error on the remote +stream.abort(new Error('Oh no!')) +``` + +## Stream/Connection stat properties + +The properties on the `stream.stat` and `connection.stat` objects are now stored on the stream/connection itself. + +**Before** + +```js +// stream.stat properties +console.info(stream.stat.direction) +console.info(stream.stat.timeline) +console.info(stream.stat.protocol) + +// connection.stat properties +console.info(connection.stat.direction) +console.info(connection.stat.timeline) +console.info(connection.stat.multiplexer) +console.info(connection.stat.encryption) +console.info(connection.stat.status) +``` + +**After** + +```js +// stream.stat properties +console.info(stream.direction) +console.info(stream.timeline) +console.info(stream.protocol) + +// connection.stat properties +console.info(connection.direction) +console.info(connection.timeline) +console.info(connection.multiplexer) +console.info(connection.encryption) +console.info(connection.status) +``` + +## Interface module consolidation + +In an effort to prevent breaking changes affecting unrelated modules, libp2p prior to 0.46.x had a large number of single-issue interface modules for internal and external types - `@libp2p/address-manager`, `@libp2p/connection-gater`, `@libp2p/connection-manager` and so on. + +This meant that although we could release a new version of the address manager interface without impacting modules that only depended on the connection manager, releasing any change became a multiple-step process during which there was a time window sometimes lasting several days when the latest versions of modules would be incompatible with each other. + +Adding new methods and types to interfaces also became a breaking change since the existing released implementations of those interfaces would not implement the new methods which complicated matters further. + +Since [libp2p/js-libp2p#1792](https://github.com/libp2p/js-libp2p/pull/1792) converted libp2p into a monorepo project, a lot of these problems have gone away since we can now release multiple libp2p modules simultaneously. + +The urgency that required multiple interface modules has also subsided somewhat so now all libp2p interfaces are collected into two modules - `@lib2p2p/interface` for public-facing APIs and `@libp2p/interface-internal` for APIs designed to be consumed by libp2p components. + +**Before** + +```js +import type { Libp2p } from '@libp2p/interface-libp2p' +import type { AddressManager } from '@libp2p/interface-address-manager' +import type { ConnectionManager } from '@libp2p/interface-connection-manager' +// etc +``` + +**After** + +```js +import type { Libp2p } from '@libp2p/interface' +import type { AddressManager } from '@libp2p/interface-internal/address-manager' +import type { ConnectionManager } from '@libp2p/interface-internal/connection-manager' +// etc +``` From b826b4ce05bc56341acfa2b25e9df4a6c05120d3 Mon Sep 17 00:00:00 2001 From: achingbrain Date: Thu, 27 Jul 2023 10:04:02 +0100 Subject: [PATCH 2/5] docs: update migration guide --- doc/migrations/v0.45-v0.46.md | 127 ++++++++++++++++++++++++++++++++-- 1 file changed, 121 insertions(+), 6 deletions(-) diff --git a/doc/migrations/v0.45-v0.46.md b/doc/migrations/v0.45-v0.46.md index 8f6077f5b7..7985e823e0 100644 --- a/doc/migrations/v0.45-v0.46.md +++ b/doc/migrations/v0.45-v0.46.md @@ -4,11 +4,126 @@ A migration guide for refactoring your application code from libp2p v__ to v__. ## Table of Contents -- [Graceful stream closing](#graceful-stream-closing) -- [Stream/Connection stat properties](#streamconnection-stat-properties) -- [Interface module consolidation](#interface-module-consolidation) +- [New features](#new-features) + - [Manual identify](#manual-identify) + - [Transient connections](#transient-connections) +- [Breaking changes](#breaking-changes) + - [Graceful stream closing](#graceful-stream-closing) + - [Stream/Connection stat properties](#streamconnection-stat-properties) + - [Interface module consolidation](#interface-module-consolidation) + +## New features + +### Manual identify + +The [identify protocol](https://github.com/libp2p/specs/blob/master/identify/README.md) is used by libp2p to discover which protocols a remote peer supports. By default it runs on every connection after it opens. + +Applications interested in peers that support certain protocols can register [topology callbacks]() to be notified when network peers that support those protocols connect or disconnect. + +`libp2p@0.46.x` adds the ability for the user to fine-tune their identify usage and to run the identify protocol manually: + +```ts +import { createLibp2p } from 'libp2p' +import { identifyService } from 'libp2p/identify' + +const node = await createLibp2p({ + services: { + identify: identifyService({ + // identify has stream limits so to prevent remote peers from closing + // streams due to too many identify streams being opened in parallel, + // so use this setting to disable running identify automatically. + // + // Note that this means you will need to run identify manually for + // every connection that opens in order for topologies to work. + // + // Some modules such as KAD-DHT and Circuit Relay rely on this being + // the case. + runOnConnectionOpen: false + }) + } +}) + +const conn = await node.dial('/ip4/123.123...') +const identifyResult = await node.services.identify.identify(conn) +``` + +Note that this is an advanced option and is not necessary for the vast majority of users. + +Most users will want to configure a topology instead to be notified when new peers are discovered that support a given protocol: + +```ts +import { createLibp2p } from 'libp2p' +import { identifyService } from 'libp2p/identify' + +const node = await createLibp2p({ + services: { + identify: identifyService() + } +}) + +node.register('/my/protocol', { + onConnect (peer, connection) { + // this is called after identify has completed and the peer has confirmed + // which protocols it supports + }, + onDisconnect (peer, connection) { + // handle disconnect + } +}) +``` + +### Transient connections + +Some connections have limits applied to them by the remote. For example, as part of the [Circuit Relay v2 protocol](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md) relay servers are allowed to limit the amount of data transferred over a relayed connection and for how long. + +These connections are not expected to be long-lived and must be treated as slightly fragile as the remote may close them at any time. To detect this, these types of connections have a boolean `.transient` property set to `true`. + +```ts +import { createLibp2p } from 'libp2p' +import { identifyService } from 'libp2p/identify' + +const node = await createLibp2p({ /* ... */ }) + +// make a direct connection to a peer +const conn1 = await node.dial('/ip4/123.123.123.123/tcp/123') +console.info(conn1.transient) // false + +// make a connection to a peer via a relay server +const conn2 = await node.dial('/ip4/.../p2p-circuit/...') +console.info(conn2.transient) // true +``` + +By default no protocols may run over a transient connection - to allow this protocols must explicitly opt-in to being run. This is in order to prevent high-bandwidth protocols from accidentally causing the remote to close the connection. + +```ts +import { createLibp2p } from 'libp2p' +import { identifyService } from 'libp2p/identify' + +const node = await createLibp2p({ + // config here +}) + +// register an incoming stream handler for a protocol that is allowed to run over +// transient connections +await node.register('/my/protocol', () => {}, { + runOnTransientConnection: true +}) + +// open a stream and allow the protocol to run over a transient connection +const stream = await node.dialProtocol('/ip4/.../p2p-circuit/...', '/my/protocol', { + runOnTransientConnection: true +}) + +// the same flag can be passed to the `newStream` method on the connection itself +const conn = await node.dial() +conn.newStream('/my/protocol', { + runOnTransientConnection: true +}) +``` + +## Breaking changes -## Graceful stream closing +### Graceful stream closing Streams can either be closed gracefully, where we wait for any unsent data to be sent, or aborted in which case any unsent data is discarded and a reset message is sent, notifying the remote of the abnormal termination. @@ -52,7 +167,7 @@ await stream.close() stream.abort(new Error('Oh no!')) ``` -## Stream/Connection stat properties +### Stream/Connection stat properties The properties on the `stream.stat` and `connection.stat` objects are now stored on the stream/connection itself. @@ -88,7 +203,7 @@ console.info(connection.encryption) console.info(connection.status) ``` -## Interface module consolidation +### Interface module consolidation In an effort to prevent breaking changes affecting unrelated modules, libp2p prior to 0.46.x had a large number of single-issue interface modules for internal and external types - `@libp2p/address-manager`, `@libp2p/connection-gater`, `@libp2p/connection-manager` and so on. From b1a5e9aa581123b33161efcb5a50caaab3f51d9d Mon Sep 17 00:00:00 2001 From: Alex Potsides Date: Fri, 28 Jul 2023 14:26:16 +0100 Subject: [PATCH 3/5] chore: Apply suggestions from code review Co-authored-by: Chad Nehemiah --- doc/migrations/v0.45-v0.46.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/migrations/v0.45-v0.46.md b/doc/migrations/v0.45-v0.46.md index 7985e823e0..c70ac1ea0e 100644 --- a/doc/migrations/v0.45-v0.46.md +++ b/doc/migrations/v0.45-v0.46.md @@ -1,6 +1,6 @@ # Migrating to libp2p@46 -A migration guide for refactoring your application code from libp2p v__ to v__. +A migration guide for refactoring your application code from libp2p `v0.45` to `v0.46`. ## Table of Contents From 0a2122b790bc4aef7ea72d025bb8ba99a7a575bc Mon Sep 17 00:00:00 2001 From: achingbrain Date: Fri, 28 Jul 2023 15:02:03 +0100 Subject: [PATCH 4/5] chore: add topology link --- doc/migrations/v0.45-v0.46.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/migrations/v0.45-v0.46.md b/doc/migrations/v0.45-v0.46.md index c70ac1ea0e..fc6b8d7df7 100644 --- a/doc/migrations/v0.45-v0.46.md +++ b/doc/migrations/v0.45-v0.46.md @@ -18,7 +18,7 @@ A migration guide for refactoring your application code from libp2p `v0.45` to ` The [identify protocol](https://github.com/libp2p/specs/blob/master/identify/README.md) is used by libp2p to discover which protocols a remote peer supports. By default it runs on every connection after it opens. -Applications interested in peers that support certain protocols can register [topology callbacks]() to be notified when network peers that support those protocols connect or disconnect. +Applications interested in peers that support certain protocols can register [topology callbacks](https://libp2p.github.io/js-libp2p/interfaces/_libp2p_interface.index.unknown.Topology.html) to be notified when network peers that support those protocols connect or disconnect. `libp2p@0.46.x` adds the ability for the user to fine-tune their identify usage and to run the identify protocol manually: From 2a4554e21300731f102bd8a3c4b906c2dc7d6494 Mon Sep 17 00:00:00 2001 From: achingbrain Date: Fri, 28 Jul 2023 17:10:35 +0100 Subject: [PATCH 5/5] chore: fix interop --- interop/BrowserDockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/interop/BrowserDockerfile b/interop/BrowserDockerfile index bc8702348a..55601230e9 100644 --- a/interop/BrowserDockerfile +++ b/interop/BrowserDockerfile @@ -11,4 +11,4 @@ ENV BROWSER=$BROWSER ENV CI true # manually specify runner until https://github.com/hugomrdias/playwright-test/issues/572 is resolved -ENTRYPOINT npm run test:interop:multidim -- --build false --types false -t browser -- --browser $BROWSER --runner mocha +ENTRYPOINT npm run test:interop:multidim -- --build false --types false -t browser -- --browser $BROWSER