Adapt max HTLC amount to balance

[thomash-acinq]

Some channels have only a few sats available to send but are not currently disabled, leading other peers to try to use them and fail. We now disable channels lower the maximum HTLC amount when the balance goes below a configurable threshold. This should reduce the number of failed attempts and benefit the network.

[t-bast]

I learned today that we are the only implementation that emits a channel_update with the disable bit set when we're low on liquidity: all other implementations only use the disable bit to advertise that their peer is offline. I think that makes sense, because that way we can simplify our onion messages path-finding and ignore edges that are disabled.

When we're low on liquidity, we should advertise that by emitting a new channel_update with either:

• a very large htlc_minimum_msat
• a very low htlc_maximum_msat
• high relay fees

I'm not sure yet which of these options is best. We also discussed adding a new failure message to the spec, to be used when we cannot relay a payment because we lack liquidity.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 80.00000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 85.86%. Comparing base (3e1cc9d) to head (2a82788).
Report is 131 commits behind head on master.

Files with missing lines	Patch %	Lines
...in/scala/fr/acinq/eclair/channel/fsm/Channel.scala	71.42%	6 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2703      +/-   ##
==========================================
+ Coverage   85.83%   85.86%   +0.03%     
==========================================
  Files         216      216              
  Lines       18077    18102      +25     
  Branches      750      762      +12     
==========================================
+ Hits        15516    15543      +27     
+ Misses       2561     2559       -2     

Files with missing lines	Coverage Δ
...r-core/src/main/scala/fr/acinq/eclair/Eclair.scala	55.69% <100.00%> (ø)
...re/src/main/scala/fr/acinq/eclair/NodeParams.scala	93.41% <100.00%> (+0.08%)	⬆️
...in/scala/fr/acinq/eclair/channel/ChannelData.scala	100.00% <ø> (ø)
...c/main/scala/fr/acinq/eclair/channel/Helpers.scala	94.52% <100.00%> (+0.01%)	⬆️
...inq/eclair/channel/fsm/CommonFundingHandlers.scala	90.69% <100.00%> (ø)
...n/scala/fr/acinq/eclair/router/Announcements.scala	100.00% <ø> (ø)
...in/scala/fr/acinq/eclair/channel/fsm/Channel.scala	85.96% <71.42%> (+0.04%)	⬆️

... and 11 files with indirect coverage changes

[thomash-acinq]

I chose the option of using htlc_maximum_msat. It is configurable and it's possible to set several thresholds, the more you use the more precisely you leak your balance (which is partly a good thing as it help others route their payments).

[t-bast]

Looks mostly good, the code is nice and concise, but this has a lot of subtle side-effects! I'm curious to see how it behaves in practice.

[rorp]

When we're low on liquidity, we should advertise that by emitting a new channel_update with either:

• a very large htlc_minimum_msat
• a very low htlc_maximum_msat
• high relay fees

I wanted to create a plugin that could change htlc_max and htlc_min in various ways based on channel capacity, liquidity, fees, etc and also that could enable/disable channels at will. But it turned out that there's absolutely no way to programmatically issue an arbitrary channel update. One can only update the routing fees...

I'm not sure yet which of these options is best.

Yeah, that's why I wanted a plugin(s) for that.

[t-bast]

I wanted to create a plugin that could change htlc_max and htlc_min in various ways based on channel capacity, liquidity, fees, etc and also that could enable/disable channels at will.

This is quite dangerous: for example you mention "enable/disable channels at will", but that's not something that should be exposed to the user, because it must only be done when the peer is offline. There are a lot of ways users would shoot themselves in the foot if we expose too many hooks here, that's why we haven't done so. Also, if we wanted to allow plugins to do that, we would need to defer almost everything channel-update related to plugins, because otherwise eclair's internals would most likely end up undoing what the plugin wants to achieve (and the plugin would undo what eclair's internals do). We'd need to think very carefully about which fields we allow plugins to override and how to handle rate-limiting. It could be worth investigating at some point, but not a high priority right now.

[rorp]

I wanted to create a plugin that could change htlc_max and htlc_min in various ways based on channel capacity, liquidity, fees, etc and also that could enable/disable channels at will.

This is quite dangerous: for example you mention "enable/disable channels at will", but that's not something that should be exposed to the user, because it must only be done when the peer is offline. There are a lot of ways users would shoot themselves in the foot if we expose too many hooks here, that's why we haven't done so. Also, if we wanted to allow plugins to do that, we would need to defer almost everything channel-update related to plugins, because otherwise eclair's internals would most likely end up undoing what the plugin wants to achieve (and the plugin would undo what eclair's internals do). We'd need to think very carefully about which fields we allow plugins to override and how to handle rate-limiting. It could be worth investigating at some point, but not a high priority right now.

I don't find it dangerous to enable/disable channels at will. What I do find dangerous though is the ability to set the base fee to 0 and inability to adjust htlc-min accordingly to avoid free routing.

IMHO it's not necessary to be that protective because the users know better want kind of plugins their want to install (or not) on their nodes.

[t-bast]

I don't find it dangerous to enable/disable channels at will.

But that's exactly why we want to hide it from users, you don't know the side-effects that it actually has! Even we thought this would only disable the channel in one direction, while all other implementations will ignore it in both directions when receiving such a channel_update, which means that you'll never receive incoming payments to bring the channel back to a balanced state.

IMHO it's not necessary to be that protective because the users know better want kind of plugins their want to install (or not) on their nodes.

The example above is a very good example of the opposite. Users often want something, but they don't realize the unwanted side effects this will have (and plugin writers generally don't realize it either). I believe it's our job to avoid exposing hooks like those when there's too much of a risk that they'll be misused and end up bringing more drawbacks than benefits.

[t-bast]

LGTM 👍

[rorp]

I don't find it dangerous to enable/disable channels at will.

But that's exactly why we want to hide it from users, you don't know the side-effects that it actually has! Even we thought this would only disable the channel in one direction, while all other implementations will ignore it in both directions when receiving such a channel_update, which means that you'll never receive incoming payments to bring the channel back to a balanced state.

For example, I don’t think that an average user of LND is way more knowledgeable that an average user of Eclair. Just check the PlebNet Telegram groups out, and you’ll see that by yourself. Yet...

$ ./lncli help updatechanstatus
NAME:
   lncli updatechanstatus - Set the status of an existing channel on the network.

USAGE:
   lncli updatechanstatus [command options] funding_txid [output_index] action

CATEGORY:
   Channels

DESCRIPTION:
   
  Set the status of an existing channel on the network. The actions can
  be "enable", "disable", or "auto". If the action changes the status, a
  message will be broadcast over the network.
  …

You can do this kind of stuff from the CLI not even from a plugin! Why so?

IMHO it's not necessary to be that protective because the users know better want kind of plugins their want to install (or not) on their nodes.

The example above is a very good example of the opposite. Users often want something, but they don't realize the unwanted side effects this will have (and plugin writers generally don't realize it either). I believe it's our job to avoid exposing hooks like those when there's too much of a risk that they'll be misused and end up bringing more drawbacks than benefits.

That’s a very good example, indeed. Everybody including the PlebNet knew about the issue of 0 base fees and htlc-min for a long time. Unfortunately, Eclair doesn’t provide any help to deal with it…

Seriously, I don’t really understand where this premise that you guys have an obligation to nanny the users comes from.

What is the target audience of Eclair? The users of custodial wallets or the builder of the custodial wallets? I believe it’s the latter (say “hello” from me to @DerEwige :) ). They know what they need and what kind of unwanted side effect they can have very well.

[DerEwige]

I would really like the ability to have more control over my channels.

I can see the potential danger for the cli, but think a “safety switch” in the config could help with that.
Like a parameter “enable_expert_cli” that needs to be set to be able to use those cli commands.

But I don’t really see an issue for plugins.

I currently do have several use cases, where I would need the ability to have better control.
Right now I use ugly hacks with much more sever side effects to fulfill my need.

1.) LND still not able to do spec compliant mutual close
lightningnetwork/lnd#6039

To prevent FCs when closing channels to LND I implemented a save close feature.
It blocks all traffic from my side (by setting the fees to 1’000 sats base and 10’000 ppm) and waits for all HTLC to clear bevor sending the close command to the channel.

The option to disable the channel would make that much saver/easier.

2.) I monitor the health of my peers (ping response time, HTLC processing time, number of stuck HTLC on the channel, …).
This helps me detect potential peer failures before they happen.
When I detect an issue, I block all traffic from my side (side (by setting the fees to 1’000 sats base and 10’000 ppm) until the problem is gone.

The option to disable the channel would make that much saver/easier.

3.) I use dynamic max_htlc for a year now to help with traffic flow and minimize local failures.
Because eclair does not allow me to persistently set that value, I have to generate gossip announcements for my channel by hand (that have different values than eclair has internally)
But eclair sends out a “wrong” channel announcement each time I update the fees which I have to catch and overwrite each time….

4.) Because I use zero base fee, I have adaptive min_htlc that make sure any forward will always pay 1 milli Satoshi in fee (freeloader protection)

Again the same issue as with max_htlc.

I would really love to have full control over at least those 3 channel stats.

[t-bast]

For example, I don’t think that an average user of LND is way more knowledgeable that an average user of Eclair. Just check the PlebNet Telegram groups out, and you’ll see that by yourself. Yet...

I've never said the opposite? But exposing all those hooks is one of the reason the main instabilities of the network come from LND nodes, because it's often a faulty plugin that creates incompatibilities and degrade payment reliability and compatibility.

Everybody including the PlebNet knew about the issue of 0 base fees and htlc-min for a long time.

I'm not sure what you're thinking of here? Which issue about 0 base fees? This isn't what I was referring to.

Seriously, I don’t really understand where this premise that you guys have an obligation to nanny the users comes from.

Lightning is extremely complex: we can't expect people to understand the side effects of some of the things they want to achieve. And lightning isn't bitcoin: running a faulty node actually harms the network and real users. Our goal is to give the right set of knobs to ensure people can build useful tools, while protecting them from shooting themselves in the foot. If you think this isn't our job, then we disagree, and that's ok!

[t-bast]

1.) LND still not able to do spec compliant mutual close
lightningnetwork/lnd#6039

It shouldn't be our responsibility to maintain additional code to work around other implementations bugs. We're not working on short-term timelines here: those bugs will be fixed in LND which will resolve this issue.

The option to disable the channel would make that much saver/easier.

But you simply can't: as explained above, using the disable bit here would violate the spec. And again, this is a case of a faulty peer: the right thing to do here is to get them to fix their issues, not to add bandages on top of faulty software.

But eclair sends out a “wrong” channel announcement each time I update the fees which I have to catch and overwrite each time….

That's exactly what I mentioned earlier: giving plugins the ability to manage channel_updates is hard to do correctly. This is something that we'll probably allow in the future, but this isn't high priority for us right now (remember that we're a small team and don't have the resources to implement everything we'd like). But if you feel strongly about it, you're free to open a PR to enable that mechanism, this is open-source software!