Proposal: deprecate pnet / PSK

[marten-seemann]

What is pnet / PSK?

Users can configure a PSK to create a “private network”. This works by first encrypting the underlying TCP connection using Salsa20, and then running a libp2p TCP connection (i.e. TCP transport or WebSocket transport) on top of that.

Specification: https://github.com/libp2p/specs/blob/master/pnet/Private-Networks-PSK-V1.md

What is it intended to achieve?

The spec says that nodes in different “private networks” should not be able to connect to each other.

This statement has been interpreted in different ways, see for example the discussion around QUIC and pnet: libp2p/go-libp2p#1432. One property that one could aim for is that if a node doesn’t know the PSK, it is not able to learn that another node speaks libp2p (the way pnet works on TCP has that property). Another interpretation would be that we only care about the handshake not completing successfully,.

A lot of times, pnet is used to make sure that nodes don’t advertise to the public IPFS DHT.

What’s the problem with this?

1. It double-encrypts all data sent on TCP. This is slow.
2. Using a PSK for access control to any network (larger than a tiny number of nodes) has a lot of unappealing properties: There’s no way to revoke access, there’s no way to handle compromise of the PSK (which is virtually guaranteed given a large enough number of nodes), there’s no way to roll to a new key, etc.
3. It only works on TCP and WebSocket. It doesn’t work on QUIC, WebTransport and WebRTC (see issue linked above for our failed attempts to make it work). QUIC is the transport we’re optimizing for, and it already handles > 75% of the traffic on the IPFS network.

What are the alternatives?

• For building a private network that hides the fact that libp2p is spoken: use a VPN. VPNs are quite literally built for this.
• For making sure that the IPFS DHT is not used: use a different protocol name for Kademlia.

[marten-seemann]

cc @Stebalien @hsanjuan @mxinden @MarcoPolo Did I miss anything (more use cases?) here?

[mxinden]

I am not opposed to deprecating pnet. That said, do we know who is still using it?

// CC @rkuhn as I see you using libp2p-pnet in https://github.com/ipfs-rust/ipfs-embed/blob/master/src/net/mod.rs.

[hsanjuan]

IPFS Cluster uses it. People use it for private Kubo storage which they expose via gateways, for example, or simply to not have their networks inadvertently intermingle with others.

Putting the burden on the users to "just run a VPN" is a huge requirement (you know very well that a lot of users exposed to our tech don't even know how to open ports on their routers). I suspect for some people PNET is more important than QUIC. It would be good to get IPFS Stewards view too.

[p-shahi]

This may merit asking on discuss.libp2p.io (where I've seen users ask about pnet), #libp2p-implementers, and other related forums. I'm happy to share this proposal and get request for feedback

[Winterhuman]

Perhaps moving to a system similar to how https://github.com/hyprspace/hyprspace handles it, where it's just a PeerID allowlist, would be a better way? (or maybe extending it to be a separate DHT protocol as suggested in the original issue)

Definitely in agreement with @hsanjuan that it shouldn't be outright removed without a capable replacement, trying to sandbox IPFS so it only advertises VPN multiaddresses can be quite tricky even for experienced users familiar with networking

[lidel]

+1, agree with @hsanjuan

Another interpretation would be that we only care about the handshake not completing successfully,.

This is how IPFS uses it. We can't remove it without a replacement and migration plan.

IPFS ecosystem is using this feature not only to avoid announcing on public DHT, but to also guard block sniffing and exchange via bitswap (or any other libp2p protocol we add in the future).

IPFS docs nudge users to use pnet for setting up private swarms:

• https://docs.ipfs.tech/concepts/privacy-and-encryption/#creating-a-private-network
• https://docs.ipfs.tech/how-to/run-ipfs-inside-docker/#private-swarms-inside-docker

There is an unknown number of dark swarms created over the years with this, quick search reveals many examples and very recent engagement on forums:

• https://docs.ipfs.tech/concepts/case-study-morpheus/#how-morpheus-network-uses-ipfs
• https://github.com/Azure-Samples/blockchain/tree/master/ledger/template/ipfs
• https://www.reddit.com/r/ipfs/comments/uiznk5/how_do_i_make_file_added_to_my_cluster_private/
• https://discuss.ipfs.tech/t/set-up-private-ipfs/14138/12
• etc..

cc @autonome @2color – probably big chunk of our collabs / partners uses this too

Managing PeerID list is not a replacement – requires changing config of every node.
VPN is not a replacement – it introduces external dependency with configuration footguns which are not acceptable for many use cases.

If we want to deprecate PNET, we need a drop-in replacement that works with /webtransport and causes minimal pain to migrate existing swarms.

[2color]

As mentioned above, it's hard to know how many are using this, but I specifically know several users relying on this. We need to come up with a migration path for them.

[Winterhuman]

It sounds like ipfs/kubo#6932 would be required for allowing a separate DHT, I'd guess being based losely of the "LAN" DHT system, to have it's own DHT records advertised only to it. There's other ways of doing this without 6932 probably, however, it'd be ideal to have that issue solved as well for the public and LAN DHTs if reasonable to do

[marten-seemann]

Putting the burden on the users to "just run a VPN" is a huge requirement (you know very well that a lot of users exposed to our tech don't even know how to open ports on their routers).

@hsanjuan As I've written in the original post, this is the suggestion for the fringe use case of "I want to hide the fact that I'm running libp2p". I don't think we need to optimize our protocols for this use case. This is not the suggestion for how we should prevent accidental spillover of nodes between different DHTs.

IPFS ecosystem is using this feature not only to avoid announcing on public DHT, but to also guard block sniffing and exchange via bitswap (or any other libp2p protocol we add in the future).

@lidel, can you explain what you mean by guard block sniffing?

It seems like we're in a situation where pnet has been adopted for multiple use cases, not because it's the right tool to use, but because it was easier than building a proper solution for that use case.

• Separating DHTs: Definitely a valid and valuable use case, but this shouldn't require an extra layer of encryption, and doesn't need to live at (or below) the transport layer. There might be value in being able to communicate with other libp2p nodes outside of that DHT, which is not possible with the current pnet design.
• Limiting network membership: This is a complicated topic, as there are many ways of doing this, some more and some less decentralized, and suitable for use cases (degree of decentralization, size of network, etc.). Using a PSK that is eternally valid for any node is one way of doing this, but only gives you any security properties in a super small networks (it's unrealistic to assume that you can keep the PSK secret forever if you distribute it to a network of dozens or hundreds of nodes) and some protection against accidental misconfiguration in larger networks. Arguably, this would be better and more generally solved by a post-(transport)-handshake auth protocol.

[hsanjuan]

Using a PSK that is eternally valid for any node is one way of doing this, but only gives you any security properties in a super small networks (it's unrealistic to assume that you can keep the PSK secret forever if you distribute it to a network of dozens or hundreds of nodes)

This assumes multi-tenancy, but that's usually not how pnets are used, because as you say, it doesn't really have a point as authorization mechanism then. However, single operators could actually isolate and authenticate networks of hundreds of libp2p nodes using pnets (and that is fine when a single person controls all nodes). This is a perfectly valid usecase.

Other than that, it is not about using separate DHTs in particular, it is about nodes not exposing any libp2p services etc. and have the piece of mind that whatever I run on a libp2p network cannot somehow leak or interact with a different one (dht, pubsub, relays, identify and any custom protocol) by mistake. I agree this problem can be addressed differently, but in some cases both go hand-in-hand.

[lidel]

@lidel, can you explain what you mean by guard block sniffing?

@marten-seemann I meant bitswap: random peers connecting to my "private" node and using bitswap to proble if I have specific CIDs. This can be used for inferring browsing history, tracking me across networks etc.

Limiting network membership: [..] not exposing any libp2p services etc. and have the piece of mind

+1, this is the gist / core feature that we should always have a "batteries included" solution.

A common production pattern i've seen is org/team to orchestrate a private cluster of nodes sharing the same PSK (single owner, so leaking PSK is not a real concern).

[rkuhn]

First off: is there a reason for removing it? The OP only lists reasons why Marten doesn’t want to use it, but that is fully possible without any changes today. When adding a feature or protocol there needs to be sufficient justification — the same goes for removing a feature or protocol that has been present for many releases. As far as I am aware the maintenance burden on pnet has been insignificant hitherto (please tell me if this is incorrect).

---

Independent of whether there are reasons for the removal, here’s what we’re using it for:

• peer-to-peer deployments of limited scope (network-wise as well as organisational)
• all devices are under the control of the same commercial entity and used for business purposes
• the shared secret acts as a security gate, admitting only authorised devices into the swarm
• roll-over is done by switching all devices to a new PSK
• performance is not a concern as processing the data is way more costly than its encryption

None of the proposed alternatives are applicable to our use-case, for the reasons stated above by @hsanjuan and @lidel.

---

I agree that the double-encryption — even though inconsequential for us — is a wart in this system. OTOH the application of salsa20 is so simple and obvious that it took very little effort to prove to commercial users that this complies with their security requirements. If there were a new idea on how to support the above use-case that retains the nice properties (i.e. no N² configuration explosion, easily verifiable, clear delineation of which devices can join the swam) then we might even switch to it.

One thing I realised only through this thread (thanks!) is that the still hatching back-pressure story of libp2p should not be founded on the QUIC transport alone, at least not until there is a replacement for pnet.

[rklaehn]

I am fine with replacing pnet with another mechanism. But just removing it entirely without replacement is not good.

There are a lot of use cases for a simple mechanism to provide some isolation from the public ipfs swarm. Due to the pre shared secret it is not a true security mechanism, so "private swarms" is maybe not a good name. For the use cases that I have seen and used it myself, the purpose is more an "isolated swarm".

Using a VPN is not a good solution, since it just requires a single misconfiguration for your node to connect to the public ipfs then. Ipfs has various mechanisms to discover peers. I have seen it many times that a node without bootstrap peers has somehow found a way to connect to the public ipfs via mdns.

[RobotSail]

So I'm not sure what the status surrounding this is, but I'm in agreement that some form of creating private networks is necessary.

As @marten-seemann points out, a limitation of PSKs are that symmetric-key encryption simply doesn't scale, and the amount of nodes existing within a swarm is potentially unbounded. A few use-cases that we have for IPFS and running a private swarm involves running within a multi-cluster environment, which opens up the risk of the key becoming compromised at some point.

I also agree that it'd be nice if there was some form of VPN-ed environment that could be bootstrapped for the network nodes to exist in. Could something like Apex be of any help in getting a private IPFS swarm running?

[rkuhn]

@RobotSail Not all swarms are unbounded, and thus pnet doesn’t need to scale. The existence of one non-served use-case does not invalidate the existence of use-cases served perfectly well.

I don’t want to keep anyone from adding a better way of doing private swarms, but please everyone keep in mind that so far no argument has been presented for removing pnet.