Can't connect /p2p-circuit/ peers

[phillmac]

Version information:

Broken versions:

$ ipfs version --all
go-ipfs version: 0.6.0-rc1-aa16952
Repo version: 9
System version: amd64/linux
Golang version: go1.14.2

$ ipfs version --all
go-ipfs version: 0.5.1-8431e2e
Repo version: 9
System version: amd64/linux
Golang version: go1.13.10

$ ipfs version --all
go-ipfs version: 0.5.0-36789ea
Repo version: 9
System version: amd64/linux
Golang version: go1.13.10

Working versions:

$ ipfs version --all
go-ipfs version: 0.4.23-6ce9a35
Repo version: 7
System version: amd64/linux
Golang version: go1.12.16

Description:

Trying to connect to peers using /p2p-circuit/ipfs/Qm...etc notation fails with an error:

$ ipfs swarm connect /p2p-circuit/ipfs/QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF
Error: connect QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF failure: failed to dial QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF: all dials failed
  * [/p2p-circuit] can't dial a p2p-circuit without specifying a relay: /p2p-circuit

In the broken versions this happens regardless if Swarm.EnableRelayHop is set to true or false
In versions 0.4.23 and before this works perfectly fine:

$ ipfs swarm connect /p2p-circuit/ipfs/QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF
connect QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF success
$ ipfs ping /p2p-circuit/ipfs/QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF
PING QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF.
Pong received: time=399.81 ms
Pong received: time=372.97 ms
Pong received: time=373.37 ms
Pong received: time=375.93 ms
Pong received: time=373.44 ms
Pong received: time=373.99 ms
Pong received: time=387.63 ms
Pong received: time=376.67 ms
Pong received: time=376.86 ms
Pong received: time=373.26 ms
Average latency: 378.39ms

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[Stebalien]

This feature was intentionally removed as it was useless outside of demos. It would try connecting to a specific peer over random relays until it found a relay already connected to that peer.

If you want to use relays, either:

1. Specify the relay when connecting: ipfs swarm connect /p2p/QmRelayAddress/p2p-circuit/p2p/QmTargetNode.
2. Explicitly advertise relay addresses on the target node using Addresses.Announce.
3. Enable the autorelay feature on the target node.

Also, please do not enable EnableRelayHop unless you want to be a relay for other peers. Especially do not enable EnableRelayHop and autorelay at the same time; your node will start advertising itself as a public relay.

[phillmac]

I take great offence to it being called a 'useless' feature. However thanks for pointing out a method for fixing the situation.
If this is so useless, can you give me a detailed explaination of how a default go-node is supposed to connect to a js-ipfs node running in the browser with only websocket or webrtc star available? This is a serious blocker to moving away from v0.4.23 as some apps rely on direct conection and comunication via pubsub.
I seriously request you re-evaluate this.

[Stebalien]

I take great offence to it being called a 'useless' feature.

Please keep this discussion technical.

---

The circuit-relay feature still works just fine. Instead of specifying /p2p-circuit/p2p/QmTargetNode ("connect to QmTargetNode by any relay"), you need to tell go-ipfs which relay to use.

Unfortunately, ipfs swarm connect /p2p-circuit/p2p/QmTarget was really inefficient, bad for the network, and flat out unreliable. It worked by:

1. Asking every node we saw to "are you willing to relay for me?" and recording the answer.
2. Whenever the user ran ipfs swarm connect /p2p-circuit/p2p/QmTargetNode , the node would:
   1. Iterate through every relay ever seen.
   2. For each relay, connect to it and ask it to open a relay connection to QmTargetNode. This would only work if the target relay happened to be connected to QmTargetNode.

It was designed for demoing the relay feature.

---

In your case:

1. In the browser, connect to some relays. For example:

   /ip4/147.75.80.110/tcp/4001/p2p/QmbFgm5zan8P6eWWmeyfncR5feYEMPbht5b1FW1C37aQ7y
   /ip4/147.75.195.153/tcp/4001/p2p/QmW9m57aiBDHAkKj9nmFSEn7ZqrcF1fZS4bipsTCHburei
   /ip4/147.75.70.221/tcp/4001/p2p/Qme8g49gm3q4Acp7xWBKg3nAa9fxZ1YmyDJdyGgoG6LsXh
   

2. In your go-ipfs node, connect to <some-relay-address>/p2p-circuit/p2p/Qm..etc (where Qm..etc is the peer ID of your browser node and <some-relay-address> is your chosen relay). For example, ipfs swarm connect /ip4/147.75.80.110/tcp/4001/p2p/QmbFgm5zan8P6eWWmeyfncR5feYEMPbht5b1FW1C37aQ7y/p2p-circuit/p2p/Qm..etc.

[phillmac]

Even after enabling auto relay and restarting the daemon this is still broken.

relevant config is as follows:

  "Swarm": {
    "AddrFilters": null,
    "ConnMgr": {
      "GracePeriod": "20s",
      "HighWater": 900,
      "LowWater": 600,
      "Type": "basic"
    },
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "DisableRelay": false,
    "EnableAutoRelay": true,
    "EnableRelayHop": false
  }

phill@charon:~$ ipfs version --all
go-ipfs version: 0.5.1-8431e2e
Repo version: 9
System version: amd64/linux
Golang version: go1.13.10

phill@charon:~$ ipfs config --json "Swarm.EnableAutoRelay"
true

phill@charon:~$ ipfs swarm connect /p2p-circuit/ipfs/QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF
Error: connect QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF failure: failed to dial QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF: all dials failed
  * [/p2p-circuit] can't dial a p2p-circuit without specifying a relay: /p2p-circuit

So I rescind my thanks, as the problem is by no means solved

[phillmac]

With regards to step one in the browser, this is these are the problems:
1.a) I'm not able to do any connecting / dialing specific relays as most apps are static bundles that don't expose the internal ipfs node or dial features e.g. https://ipfs.io/ipfs/Qma4jzYSZkzUt4Tz4Pxr7KGupHCViaFNZh13bsEhovPkZp
1.b) What about other apps that use the same data but not the same source code? How can we interoperate in a distributed fashion when we all have to mutually agree on a fixed set of relay nodes?

2). I think it's bad design to hard code a list of static ip addresses. I'm sure you've heard this argument before so I won't rehash it.

The upshot is it's just not viable, there needs to be a functioning, automatic discovery system with no hard coded relays

[phillmac]

2. Explicitly advertise relay addresses on the target node using Addresses.Announce.

If ipfs dht findpeer results in Error: routing: not found unless I've already connected via /p2p-circuit/ how can we use the announce function?

[RubenKelevra]

Hey @phillmac,

I understand your frustration, but the functionality you relied on wasn't really supposed to be used this way. You actually relied on a "fixed subset of relay nodes" before, and that's why it's not working anymore now.

Let's try to move on and find a solution that works for you.

Your browser client should connect to the network and use a random relay (if it detects that it needs one) and announce the peer address.

Once connected to the network you should be able to do this on other clients to find the remotes:

ipfs dht findpeer QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF

And then connect to a remote which is published for this public key.

This doesn't require any static relay, a relay per se.

If you want to avoid this findpeer step, just use ipfs swarm connect /ipfs/QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF.

[phillmac]

@RubenKelevra Ok I'm willing to try to work through this as it's major pain point having to continually try to solve this downstream when issues are posted about techniques that work in the past that keep breaking.

If you want to avoid this findpeer step, just use ipfs swarm connect /ipfs/QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF.

phill@charon:~$ ipfs swarm connect /ipfs/QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF
Error: connect QmegNdg2BRG7ujQrtra8aZ1U8YYXbS8GED7dAc1P6REXoF failure: routing: not found

what I'm really interested in is this bit

Your browser client should connect to the network and use a random relay (if it detects that it needs one) and announce the peer address.

I'm going to make some assumptions here, rightly or wrongly.

1). I haven't seen a working example of how to get rendezvous working esp. since the most recent refactor to js-ipfs moved to proper async/await instead of callbacks, so it's not really on the table yet

2). With the old websocket-star and the more current webrtc-star there no way that it can 'use a random relay', because the config defines the star nodes that its allowed to connect to, thus the browser node can't arbitrarily pick some node to relay thru, much less join the dht and announce its presence.
What's the flaw with my assumptions? Is there some third point I haven't considered?

[phillmac]

I'm not altogether against relying on "fixed subset of relay nodes". I'm against having to hard code that specifically in my app, because over time it will almost certainly become outdated and incorrect if it's not constantly updated; which incurs a heavy maintenance cost let alone if the app is no longer receiving updates. The point is any configuration of relay nodes should be handled in ipfs to ensure reliability and compatibility between various apps

[Stebalien]

Unfortunately, this isn't something you can just "fix" through a config tweak. In my first response, I assumed you were using go-ipfs everywhere (no mention of js-ipfs). My second response presented the only available solution for js-ipfs.

To be clear, addresses of the form /p2p-circuit/ipfs/QmTargetNode will not work and that's not going to change. This "feature" tried random relays until something worked and was never intended for production use. I'm surprised it even worked as well as it did.

In your case, the main issue is that js-ipfs doesn't support the DHT when run in the the browser. Usually, a (go) IPFS node will find another IPFS node by looking up the other IPFS node's peer ID (QmFooBar) in the DHT, resolving it to a set of addresses (/ip4/.../tcp/4001/ipfs/QmFooBar, /ipfs/QmRelay/ipfs/QmFooBar, etc.). That's how an IPFS node can automatically determine the correct relay for a target node. However, for this to work, the QmFooBar node needs to join the DHT to announce this information to the DHT.

Go-ipfs will use relays as follows (when Swarm.EnableAutoRelay is set to true):

1. The node (A) that needs the relay (the node that can't otherwise be dialed) will pick a relay from a preset list of relays (we found that random relays on the network were too unreliable to be useful).
2. Node A will then advertise this choice via the DHT.
3. Then, when another node (B) wants to connect to node A, it will:
   1. Search the DHT for node A.
   2. Find these advertised relay addresses.
   3. Connect to the relay.
   4. Connect to node B through the relay.

Unfortunately, that's clearly no going to work in this case because node A can't advertise itself in the DHT without DHT support.

I'm against having to hard code that specifically in my app, because over time it will almost certainly become outdated and incorrect if it's not constantly updated; which incurs a heavy maintenance cost let alone if the app is no longer receiving updates. The point is any configuration of relay nodes should be handled in ipfs to ensure reliability and compatibility between various apps

You're absolutely right that this should "just work". However, the fact of the matter is that it doesn't "just work".

---

Now, putting my moderator hat on, I understand that you're frustrated but your behavior is unacceptable. We've done our best to answer your every question and have been met with nothing but hostility.

I take great offence to it being called a 'useless' feature.

Me calling the feature "useless" has nothing to do with you.

So I rescind my thanks

What about a thanks for trying to help? For responding promptly? For maintaining a hellishly complex project you depend on? This kind of comment really hurts.

Ok I'm willing to try to work through this as it's major pain point having to continually try to solve this downstream when issues are posted about techniques that work in the past that keep breaking.

This is the first issue you've filed in this repo and the first issue you've commented on. Have we been missing something?

[phillmac]

This is the first issue you've filed in this repo and the first issue you've commented on. Have we been missing something?

I'm going to try to avoid link spam here so please bear with me. I've inserted an x to trip up the auto mentioner
https://github.com/orbitdbx/orbit-db/issues/790
https://github.com/orbitdbx/orbit-db/issues/789
https://github.com/orbitdbx/orbit-db/issues/761
https://github.com/orbitdbx/orbit-db/issues/755

The majority of issues here are browser based js-ipfs connectivity issues,
and for a long time my go-to fix has been to try to get connectivity working via p2p-circuit,
but now I feel as if that rug has been pulled out from under my feet with no viable alternative.

My apologies for coming off like an asshat, and yes I do appreciate the work put into this, especially the help given with issues. I guess I'm just a bit raw over the fact that everything keeps breaking and I have no ability to flash forward 10 years to when ipfs is a stable mature technology with a well defined feature set; and no ability to contribute to improvements (working with go as a language is even more frustrating imho)

[Stebalien]

Ah, yeah, I'd be frustrated as well. And I'd love to fast forward 10 years.

In terms of short-term solutions, I can't think of anything simpler than hard coding a few relays.

A medium term solution would be to bring the DHT to browsers. Unfortunately, proper DHT support generally requires creating too many connections to be usable in the browser. However, it should be possible to partially implement DHT support through delegated routing (partially implemented already but not fully usable).

1. Have the browser ask a "delegated router" to find the closest DHT nodes to the requesting node.
2. Have the browser then connect to these nodes over a relay.

The browser node would then be findable in the DHT.

Unfortunately, these browser nodes would still need a hard-coded list of delegated routers.

In terms of long term solutions, there has been grumblings of proper WebRTC support in go and js, but I haven't seen anything significant land yet. That would let us skip the relays entirely (but we'd still need some way to find browser nodes, either through a DHT or some other system).

[phillmac]

@Stebalien Just a final follow up on this, I know there's no hope this misstep will ever be rectified, but It's been a massive pain point to me to the extent that I'm having to roll back all my nodes to v0.4.23, which has been a horrible experience 😞

[Stebalien]

Unfortunately, neither downgrading to 0.4.23 nor re-adding the "try every relay till one works" feature are viable, long-term solutions.

Thinking through this a bit, I think the right solution for now is to have js-libp2p nodes use "delegated routing" (like https://github.com/libp2p/js-libp2p-delegated-peer-routing). However, that library is focused on finding peers. In this case, we need js-libp2p nodes to advertise themselves.

1. The js-libp2p node in question will need an equivalent of go-ipfs's AutoRelay feature (https://github.com/libp2p/go-libp2p/tree/master/p2p/host/relay). Specifically, the js-libp2p node will need to advertise it's chosen relays via the identify protocol (along with all of its other addresses). Alternatively, each js-libp2p node can configure go-libp2p with a specific set of relays on start.
2. The js-libp2p node will need to connect to its closest DHT peers. It can do this by:
3. Looking up its own peer ID in the DHT via a delegated router (http://delegated.router/api/v0/dht/query/MyPeerID)`.
4. Connecting to its closest peers via the circuit-relay transport.
5. Once the js-libp2p node connects to its closest peers in the DHT, these peers will learn the js-libp2p's node's addresses.

If you do this, any peer on the network should be able to find your js-libp2p node via the DHT.

cc @jacobheun are there any plans related to this?

[jacobheun]

Yes, although I think we will need to prioritize AutoRelay to help with this.

• js-ipfs is going to be turning on delegates by default which will help with out of the box support feat: turn on delegate nodes by default js-ipfs#3148 for routing.
• We're looking at improving support for the DHT in js-ipfs in general. Starting with getting support for running a client by default and looking at updating the js DHT to include the latest updates from Go, but this won't be until Q4. We can use delegates in the interim.

This is a good, tangible use case we can work on resolving on the JS side. I'll create a tracking issues in js-libp2p for this, I think we can get some mitigations in place relatively soon to improve this. I've stubbed out an issue for this in js-libp2p, I will flush out the details today, libp2p/js-libp2p#699.