Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BOLT 7: add gossip address descriptor type DNS hostname #911

Merged
merged 1 commit into from
Aug 2, 2022

Conversation

m-schmoock
Copy link
Collaborator

@m-schmoock m-schmoock commented Sep 13, 2021

This introduces a new gossip address descriptor type used for DNS hostnames.
This is particular useful for dynamic DNS users that want to use their home
ISP connection with changing IP addresses without relying only on TOR.

The len field is deliberately encoded with just a byte (u8) since
DNS hostnames do not exceed 253 bytes in total (255 on POSIX) and we want
to avoid a field that can be used for arbitrary spam.

Since current implementations SHOULD ignore the first 'address descriptor' that does NOT match the types defined above, this can be phased in as an optional addition.

Questions:

  • Currently none

Alternatives:

07-routing-gossip.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@t-bast t-bast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If your node changes its IP address, your peers won't be able to connect to you automagically, that's for sure. However in that case you should be the one initiating connections, which gives them the opportunity to save your new IP address. You should also broadcast a new node_announcement with your new IP address, and your peers may pick that up and be able to connect back to you (if for some reason you didn't initiate the connection).

So it seems the only case where you're screwed is if both you and all of your peers changed IP addresses at the same time, which seems very unlikely.

I'm not completely against this proposal, but would like to better understand if the underlying issue is rather bugs/holes in current implementations: if that's the case, we should instead fix implementations and if everything works fine then, no need to add anything new?

07-routing-gossip.md Outdated Show resolved Hide resolved
@m-schmoock
Copy link
Collaborator Author

Hi @t-bast ,
I know that the current way would be to broadcast an node_announcement update via gossip when the IP address has changed. However, there are multiple reasons to also support DNS hostnames at the protocol level:

  • Separation of duty: Detection of IP address change is often already handled better by a local router/box or by system services like ddclient.
  • The users usually does not have to care if their ISP supports IPv6 or IPv4 or a mix of those, this is handled by the router/box or ddclient for that matter.
  • For IPv4 addresses it is usually not trivial to detect the correct public address (NAT etc), because the software would then have to query external services and implement the stuff that the router/box or ddclient already knows better.
  • For IPv6 the situation can be better if the network is setup correctly.
  • DNS is sometimes used for handing redundancy in connectivity, i.e. where a node operator has a main and a backup line (slower or more expensive to use) that he would like to switch over in-case of failure.
  • Finally, broadcasting a node_announcement and propagation is likely slower than a dynamic DNS update.

Not sure if I missed something :D

@m-schmoock
Copy link
Collaborator Author

m-schmoock commented Sep 24, 2021

Also my motivation was to enable users a well known alternative to using TOR, whose broad usage just for bypassing firewall issues has become a pain for the network.

@TheBlueMatt
Copy link
Collaborator

TheBlueMatt commented Sep 24, 2021

Without taking a position on the rest:

For IPv4 addresses it is usually not trivial to detect the correct public address (NAT etc), because the software would then have to query external services and implement the stuff that the router/box or ddclient already knows better.

This is better handled by asking your peers what your IP is. Your router doesn't know your public IP address any better than any other software running locally does - CGNAT is becoming more of a reality, so you either ask a public centralized service or you ask your peers if they can connect back to you.

DNS is sometimes used for handing redundancy in connectivity, i.e. where a node operator has a main and a backup line (slower or more expensive to use) that he would like to switch over in-case of failure.

This is better handled by making it explicit (I know we talked about it, dunno if we ever did it) that IP addresses in gossip messages are in "preferred order", and connectors should consider it when connecting.

Finally, broadcasting a node_announcement and propagation is likely slower than a dynamic DNS update.

This isn't really a useful comparison, though - the connector needs to poll for the update, so in practice it may be quicker to just see a node_announcement, at least that can happen via "push".

@m-schmoock
Copy link
Collaborator Author

m-schmoock commented Sep 25, 2021

This isn't really a useful comparison, though - the connector needs to poll for the update, so in practice it may be quicker to just see a node_announcement, at least that can happen via "push".

True, the operating system will do that when the daemon decides to connect. To be precise dynamic DNS providers usually use a DNS TTL of 60 seconds, so that also has to be accounted for when comparing this to node_annoucement propagation.

I think the main advantage is to not re-implement dynamic DNS that is already running on your router. The router knows the correct IPv4 address for sure, a daemon running in userspace needs to ask external services and hope to get a reliable answer.

@m-schmoock
Copy link
Collaborator Author

m-schmoock commented Sep 25, 2021

Another not so clean but less centralized option to get 'the correct' IP(v4) address is to add the address another node connected from to the init message after connection establishment. In this case no external service would be required. However, the user needs to open a connection to discover that, but he likely got disconnected anyway before.

When it discovers that multiple peers return 'the same new address' it can decide to send an updated node_annoucement.

Just a thoght.

Update: This would be a plug and play solution because the user would not need to hassle to setup dynamic DNS. But the router still needs to have UPNP or port forwarding enabled. To pros and cons, but not elegant imho.

@t-bast
Copy link
Collaborator

t-bast commented Sep 27, 2021

However, there are multiple reasons to also support DNS hostnames at the protocol level:

Sure, I'm not arguing that it will not make some people's lives easier.
But it's a dependency on a centralized service, which is why it wasn't provided.
If DNS providers decide to stop resolving your DNS name, your peers won't be able to find you so you're effectively DoS-ed.
It's unlikely that it will happen for "small" nodes using this, but it should be clear that nodes with a lot at stake should strongly avoid using DNS whenever possible (IMO).

@m-schmoock
Copy link
Collaborator Author

@t-bast
I see, what your thought about the approach to add the IP address a node connected from to the init message of the responder?
This way any implementation can choose when/how to use that information or not with no external service required.

@t-bast
Copy link
Collaborator

t-bast commented Sep 27, 2021

I see, what your thought about the approach to add the IP address a node connected from to the init message of the responder?
This way any implementation can choose when/how to use that information or not with no external service required.

I think it's interesting, but I would be wary of automatically generating node_announcements from it. If you have only a few peers, they can easily collude to send you the same invalid IP address, and if you broadcast that to the network then no-one will be able to connect to you, and your peers have effectively eclipsed you.

This is something that the node operator could detect (either manually or by using some kind of watchdog polling other external services to discover its real IP address) and then could force-close / ban these peers, but it's tedious.

I think it's useful information to have though. The way I would use it would look like:

  • my node receives IP addresses from my peers' init
  • if the IP address my peers report is different from my last node_announcement, add a line into a special log file
  • the node operator reads that special log file regularly, can see its IP address may have changed, verifies that with other services, and if it looks reasonable uses the node API to broadcast a new node_announcement with this IP address

I've been thinking about having this "special log file" for curated messages that the node operator should read because they may require action (there are already a few other cases where I'd like to write to such a log file), we may ship something like this in eclair soon.

@m-schmoock
Copy link
Collaborator Author

they can easily collude to send you the same invalid IP address, and if you broadcast that to the network then no-one will be able to connect to you, and your peers have effectively eclipsed you.

Well, in a scenario where a node also has a TOR connection as fallback there is no big deal in using this information when a couple of peers or a certain fraction of them report the same public address.

@m-schmoock
Copy link
Collaborator Author

m-schmoock commented Sep 27, 2021

Also if I'm not mistaken, the node itself can make the check if the address is correct by 'connecting to himself' using the reported address, right?

Update: Also a node can check several random peers he knows via gossip to check that information for correctness.

@t-bast
Copy link
Collaborator

t-bast commented Sep 27, 2021

Also if I'm not mistaken, the node itself can make the check if the address is correct by 'connecting to himself' using the reported address, right?

Good idea, I haven't tested it though, it's worth prototyping.

Update: Also a node can check several random peers he knows via gossip to check that information for correctness.

True, if we do that lightning would really turn into a decentralized "what's my ip" service (which is interesting).

@m-schmoock
Copy link
Collaborator Author

@t-bast added #917 as a potential alternative to discuss

@TheBlueMatt
Copy link
Collaborator

TheBlueMatt commented Sep 27, 2021

I think it's interesting, but I would be wary of automatically generating node_announcements from it. If you have only a few peers, they can easily collude to send you the same invalid IP address, and if you broadcast that to the network then no-one will be able to connect to you, and your peers have effectively eclipsed you.

I dunno that this is so cut-and-dry - if you have a dynamic IP and your IP changes, your peers aren't going to be able to connect to you. If you get lied to and you broadcast a new bogus IP, you still can't be connected to, and as long as your node private key hasn't leaked, you haven't changed anything. Obviously if you have accepted incoming connection(s) to the previous IP you shouldn't update your IP in the node announcements, but as long as you can connect outbound to your peers it may not be the end of the world.

Edit: note that I suppose mitm is a concern for timing attacks - we don't want an attacker to be able to observe payment flows by looking at TCP traffic. Of course we should also solve this via other means because network-level attackers are a concern whether they are MITM'ing you or not, but still.

@TheBlueMatt
Copy link
Collaborator

The router knows the correct IPv4 address for sure

This is absolutely not the case - this is the case in large western ISPs that have a surplus of IPv4 addresses. This is not the case for many smaller ISPs or, increasingly, non-western ISPs that do not provide public IP addresses to their users. Hell, T-Mobile US even assigns "public" IP addresses in the 11/8 block to its clients, except of course those aren't T-Mobile's IP addresses nor publicly routable, they're just in the publicly routable blocks of v4. Of course in most of those cases you can't accept incoming connections anyway, but do not assume that you have a publicly routable IP in any case where some third party hasn't managed to connect inbound to you.

Also if I'm not mistaken, the node itself can make the check if the address is correct by 'connecting to himself' using the reported address, right?

Generally not - while this may work sometimes, there's plenty of cases where NAT routing does not work properly and you cannot connect back to yourself even though others can connect to you.

In general I'd rather just announce every unverified IP any peer that I have a channel with has told me they connected to me on and hope at least one works. Peers trying to connect inbound will cycle through them all and test until they find one that matches.

@m-schmoock
Copy link
Collaborator Author

Generally not - while this may work sometimes, there's plenty of cases where NAT routing does not work properly and you cannot connect back to yourself even though others can connect to you.

Sure, port forwarding or upnp are issues that the user and or the software can take care of. Enabling forwarding is something most users capable of setting up a LN node should be able to do.

@TheBlueMatt
Copy link
Collaborator

Sure, port forwarding or upnp are issues that the user and or the software can take care of. Enabling forwarding is something most users capable of setting up a LN node should be able to do.

Sorry, to clarify, I did not mean that it won't work because of lack of port forwarding (though that can be the case), I'm more speaking to cases where an IP/port is connectable, but due to asymmetric routing, a VPN, a bug in port mapping internal->intern on a NAT, etc, you may not be able to connect "to yourself", but others can connect to you. Its not entirely uncommon in my experience that a NAT system will fail to properly remap source addresses when you connect internal->external IP.

wvanlint added a commit to wvanlint/rust-lightning that referenced this pull request Jun 21, 2022
Supports Bolt 7 DNS hostnames specified by lightning/bolts#911.
wvanlint added a commit to wvanlint/rust-lightning that referenced this pull request Jun 21, 2022
Supports Bolt 7 DNS hostnames specified by lightning/bolts#911.
wvanlint added a commit to wvanlint/rust-lightning that referenced this pull request Jun 21, 2022
Supports Bolt 7 DNS hostnames specified by lightning/bolts#911.
wvanlint added a commit to wvanlint/rust-lightning that referenced this pull request Jun 21, 2022
Supports Bolt 7 DNS hostnames specified by lightning/bolts#911.
@TheBlueMatt
Copy link
Collaborator

We landed a PR to add this to LDK today 🎉 . Is there a node announcing these today that I can glance at to make sure its working?

@m-schmoock
Copy link
Collaborator Author

@TheBlueMatt
Nice! I just updated and configured my node to announce a DNS name on mainnaet. My ID: 02a4aae2e021b0785f2f2a7c9dc9089361995085298598992bf69f11302a5dbd2c

It should announce
orangenight.dynv6.net

@TheBlueMatt
Copy link
Collaborator

Oops I spoke to soon, sorry. I still need lightningdevkit/rust-lightning#1519 to upgrade my node and actually go test on mainnet.

@m-schmoock
Copy link
Collaborator Author

@TheBlueMatt
Hey, I also just fixed an issue with our code that was announcing my DNS address on port 0 which was likely filtered out or not usable by your node because of that... Should be working now... at least when my dyn DNS provider ( https://dynv6.com ) get its issues fixed as well (likely start of the week). Needless to say that explorers won't show this annoucement yet as its not part of the protocol yet.

{
   "id": "02a4aae2e021b0785f2f2a7c9dc9089361995085298598992bf69f11302a5dbd2c",
   "address": [
      {
         "type": "ipv6",
         "address": "2001:16b8:11f9:5c00:e65f:1ff:fe2a:8aa9",
         "port": 9735
      },
      {
         "type": "torv3",
         "address": "rs3ujxl6k43qltjjbozj6zble3nby6xaswrywd2r7ejx74ptbxzylpqd.onion",
         "port": 9735
      },
      {
         "type": "dns",
         "address": "orangenight.dynv6.net",
         "port": 9735
      }
   ]
}

Copy link
Collaborator

@TheBlueMatt TheBlueMatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They *really* have addr Hostname { hostname: Hostname("orangenight.dynv6.net"), port: 9735 }

Looks like this is ready to land! Let's land it at the spec meeting monday (or before then, even?)

@TheBlueMatt TheBlueMatt removed the Awaiting Interop Approved, pending 2 or more impl interoperating label Jul 26, 2022
Copy link
Collaborator

@Roasbeef Roasbeef left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🥕

Copy link
Collaborator

@t-bast t-bast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 088ac9d

@TheBlueMatt TheBlueMatt merged commit 38a2b1d into lightning:master Aug 2, 2022
@m-schmoock m-schmoock deleted the bolt7-hostnames branch August 2, 2022 17:44
t-bast pushed a commit to ACINQ/eclair that referenced this pull request Aug 16, 2022
It is now possible to specify a DNS host name as one of your
`server.public-ips` addresses.

DNS host names will not be resolved until eclair attempts to
connect to the peer.

See lightning/bolts#911
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants