go-libp2p v0.25

[marten-seemann]

Timeline

🗺 What's left for release

Swarm Metrics:

• swarm: add a basic metrics tracer #1973
• transports: expose the name of the transport in the ConnectionState #1911
• expose the security protocol on the ConnectionState #1907

WebTransport:

• Tracking issue: Finish WebTransport Support #1827
• add WebTransport to the list of default transports #1915

OpenSSL:

• Tracking Issue: drop support for OpenSSL #1954

Dealing with Deprecations:

• Migrate off github.com/gorilla/websocket #1970
• switch to Google's Protobuf library, make protobufs compile with go generate #1979

Identify Refactoring:

• identify: remove old code targeting Go 1.17 #1964
• identify: remove support Delta #1974
• identify: refactor sending of Identify pushes #1984
• identify: add some basic metrics #2069

QUIC:

• update quic-go and webtransport-go to their new location in quic-go: use quic-go and webtransport-go from quic-go organization #2040
• quic: disable sending of Version Negotiation packets #2015
• chore: update webtransport-go to v0.5.1 #2072

Resource Management:

• rcmgr: *: Always close connscope #2037
• rcmgr: Use prometheus for metrics (OpenCensus generates garbage) #1955
• feat!: rcmgr: Change LimitConfig to use LimitVal type #2000
• rcmgr: use libp2p prefix for Prometheus metrics #2045

Misc:

• calling Connect to a peerID not in the peerstore panics #2005
• quic, tcp: only register Prometheus counters when metrics are enabled #1971
• autorelay: refactor libp2p.EnableAutoRelay option #1866
• set names for eventbus event subscriptions #2057

🔦 Highlights

< top highlights for this release notes >

Metrics

We've started instrumenting the entire stack. In this release, we're adding metrics for:

• the swarm: tracking incoming and outgoing connections, transports, security protocols and stream multiplexers in sue: (TODO: add versioned link to the dashboard JSON after the release is cut)
• the event bus: tracking how different events are propagated through the stack and to external consumers (TODO: add versioned link to the dashboard JSON after the release is cut)

Our metrics effort is still ongoing, see #1356 for progress. We'll add metrics and dashboards for more libp2p components in a future release.

Switching to Google's official Protobuf compiler

So far, we were using GoGo Protobuf to compile our Protobuf definitions to Go code. However, this library was deprecated in October last year: https://twitter.com/awalterschulze/status/1584553056100057088. We benchmarked serialization and deserialization, and found that it's (only) 20% slower than GoGo. Since the vast majority of go-libp2p's CPU time is spent in code paths other than Protobuf handling, switching to the official compiler seemed like a worthwhile tradeoff.

Removal of OpenSSL

Before this release, go-libp2p had an option to use OpenSSL bindings for certain cryptographic primitives, mostly to speed up the generation of signatures and their verification. When building go-libp2p using go build, we'd use the standard library crypto packages. OpenSSL was only used when passing in a build tag: go build -tags openssl.
Maintaining our own fork of the long unmaintained go-openssl package has proven to place a larger than expected maintenance burden on the libp2p stewards, and when we recently discovered a range of new bugs (this and this and this), we decided to re-evaluate if this code path is really worth it. The results surprised us, it turns out that:

• The Go standard library is faster than OpenSSL for all key types that are not RSA.
• Verifying RSA signatures is as fast as Ed25519 signatures using the Go standard library, and even faster in OpenSSL.
• Generating RSA signatures is painfully slow, both using Go standard library crypto and using OpenSSL (but even slower using Go standard library).

Now the good news is, that if your node is not using an RSA key, it will never create any RSA signatures (it might need to verify them though, when it connects to a node that uses RSA keys). If you're concerned about CPU performance, it's a good idea to avoid RSA keys (the same applies to bandwidth, RSA keys are huge!). Even for nodes using RSA keys, it turns out that generating the signatures is not a significant part of their CPU load, as verified by profiling one of Kubo's bootstrap nodes.

We therefore concluded that it's safe to drop this code path altogether, and thereby reduce our maintenance burden.

Other Breaking Changes

We've cleaned up our API to consistently use protocol.ID for libp2p and application protocols. Specifically, this means that the peer store now uses protocol.IDs, and the host's SetStreamHandler as well.

Changelog

< changelog generated by scripts/mkreleaselog >

✅ Release Checklist

• Stage 0 - Finishing Touches
  • Go through relevant libp2p repos looking for unreleased changes that should make it into the release. If you find any, cut releases.
  • Run go get -u ./... to see if there are any out-of-date deps that look important. If there are, bubble them. Try to avoid directly updating indirect deps in go-libp2p's go.mod when possible.
• Stage 1 - Release
  • Publish the release through the GitHub UI, adding the release notes. Some users rely on this to receive notifications of new releases.
  • Announce the release on the discuss.libp2p.io.
• Stage 2 - Update Upstream
  • Update the examples to the final release
  • Update the upstream dependencies to the final release and create PRs.
    • filecoin-project/lotus
    • ipfs/kubo
• Make required changes to the release process.

[p-shahi]

Is it possible to include #1988 in this release if not too heavy of a lift?

[marten-seemann]

Is it possible to include #1988 in this release if not too heavy of a lift?

I don't think we should do that. I don't really see a lot of value in fixing this issue.

[BigLep]

Seems like in the changelog highlights there is also a good opportunity to highlight the maintenance decisions we're making (e.g., using well supported libraries, removing functionality like identify delta that aren't worth it, etc.)

[marten-seemann]

Seems like in the changelog highlights there is also a good opportunity to highlight the maintenance decisions we're making (e.g., using well supported libraries, removing functionality like identify delta that aren't worth it, etc.)

That was the plan. Just haven't gotten around to actually drafting release notes yet, except for the OpenSSL part.

[p-shahi]

Is it possible to include #1988 in this release if not too heavy of a lift?

I don't think we should do that. I don't really see a lot of value in fixing this issue.

reg. wondertan's latest comment, he makes a convincing point, it seems like it would be a welcome ergonomic change for the Celestia folks. Are you sure it shouldn't be included?

[marten-seemann]

Is it possible to include #1988 in this release if not too heavy of a lift?

I don't think we should do that. I don't really see a lot of value in fixing this issue.

reg. wondertan's latest comment, he makes a convincing point, it seems like it would be a welcome ergonomic change for the Celestia folks. Are you sure it shouldn't be included?

I'd be happy to review a PR, this seems like an easy and self-contained change. It's definitely an improvement, but I don't think this has high enough priority to be handled by the core team. If the PR lands by the time we cut the release, we'll include it.

[marten-seemann]

Timeline update: We expect this release to ship at some time in the week starting Jan 23.

[p-shahi]

Aiming to release around Feb 1st

[marten-seemann]

quic-go will cut a new release as soon as Go 1.20 is out (see quic-go/quic-go#3686), which will apparently happen on Feb 2nd. I suggest we release after that to include the latest quic-go version.

[marten-seemann]

quic-go will cut a new release as soon as Go 1.20 is out (see quic-go/quic-go#3686), which will apparently happen on Feb 2nd. I suggest we release after that to include the latest quic-go version.

quic-go v0.32.0 is out and here's the PR that updates it in go-libp2p: #2040. This is no longer a blocker.