Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LSPS4: Continuous JIT Channels. #37

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ZmnSCPxj-jr
Copy link
Contributor

@ZmnSCPxj-jr ZmnSCPxj-jr commented Apr 20, 2023

Alternative to #22

Closes: #25

Closes: #26

Much more complex than the currrent LSPS2 spec.

Note: we really should not be clearing state based on disconnections of the peer, because CLN, LND, and LDK have race conditions between connection/disconnection notification and receiving and sending of custom messages. This is why this is unnecessarily complex as both sides kinda need to persist information across connection sessions for robustness against this race condition.

Advantages:

  • Continuous JIT: keep opening new channels as your inbound liquidity depletes, with the client not having to keep track of this inbound liquidity (which it might not be able to manage allocation, since e.g. asynchronous receives are intended to be spontaneous and not require a single invoice from the receiver). Client continuously checks the LSP opening feerates (which can change over time depending on mining fee estimates) as needed.
  • Privacy with BOLT11: Can issue multiple ephemeral SCIDs, the client can generate multiple "node ID" aliases of itself so each BOLT11 invoice it issues has a different SCID and payee node ID.
  • Onion oracle: Support MPP and var-amount invoices. Even better privacy when PTLCs come into the picture, as the LSP cannot even know if you are receiving a single MPP or multiple unrelated single payments, it only knows the total aggregate.
  • Aggregate multiple unrelated payments: If you receive a lot of small payments in a short span of time, each of which is unable to pay for a single channel open, you can aggregate them and receive them all at once to pay for the channel open.
    • This would be significantly improved by asynchronous receives, which we already prepare for in the API.

The major disadvantage is a big increase in complexity in both LSP and client sides. Not even sure this is implementable with current node software LOL.

Copy link
Contributor

@JssDWt JssDWt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a process like this with the pending payment parts would be great. It allows for several things

  • multipart payments
  • 0 amount invoices or overpayment in general
  • wallets that send probes before the actual payment

All in all this seamlessly integrates with most functionality available in the lighting network today.

Comments below on how to structure the messaging. I think the queue structure could be removed if the client is notified of every ppp.

LSPS2/README.md Outdated Show resolved Hide resolved
LSPS2/README.md Outdated Show resolved Hide resolved
LSPS2/README.md Outdated Show resolved Hide resolved
LSPS2/README.md Outdated Show resolved Hide resolved
@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: Always PPP-hold all HTLCs, change retryppp to forwardppps (note plural), replay notifications at reconnection instead of maintaining a queue of PPP identifiers, renumber to LSPS4.

@ZmnSCPxj-jr ZmnSCPxj-jr changed the title LSPS2: Continuous JIT Channels. LSPS4: Continuous JIT Channels. May 2, 2023
Copy link

@TonyGiorgio TonyGiorgio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really enjoy the considerations made here for privacy, MPP, eventual async/ptlc, and long-lasting relationships with clients. Seems like it solves several issues I have considered with LSPS2.

Is the justification between the two specs understood to be that LSPS2 is the simple, first pass approach that LSPs will implement first while working to support LSPS4? Or is LSPS4 supposed to replace LSPS2 (or not even merge LSPS2 at all)?

LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
@ZmnSCPxj-jr
Copy link
Contributor Author

ZmnSCPxj-jr commented May 8, 2023

Is the justification between the two specs understood to be that LSPS2 is the simple, first pass approach that LSPs will implement first while working to support LSPS4? Or is LSPS4 supposed to replace LSPS2 (or not even merge LSPS2 at all)?

That (LSPS2 is the simple first-pass, LSPS4 later) is the current plan inside Block at least.

@ZmnSCPxj-jr ZmnSCPxj-jr force-pushed the alt-jit-channels branch 2 times, most recently from a759fa9 to 8a3ad1d Compare May 9, 2023 10:41
@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: change API version negotiation scheme, start copying the bits from LSPS2, clarify about hash for HTLC-type PPPs.

@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: explain why all HTLCs need to be put into PPPs even if the client already has inbound capacity

LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: fill in details from LSPS2, mark as ready for review, lots of changes.

@ZmnSCPxj-jr ZmnSCPxj-jr marked this pull request as ready for review July 31, 2023 10:40
@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: Move some text blocks around, explain permanent SCIDs a bit better, remove duplicated paragraph.

Copy link
Collaborator

@SeverinAlexB SeverinAlexB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First review pass (will need more passes in the future). In general, I like the approach. It's definitely more complicated than LSPS2. It's important we are aware of how Route Blinding works to find the right balance between privacy and features.

Thx Zmn for this proposal.

LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Show resolved Hide resolved
LSPS4/README.md Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: use names instead of one-character codes for types (e.g. "htlc" instead of "h" ), rename ppp / ppps to ppp_id / ppp_ids , recommend how to make the SCIDs, allow LSP to change feerate and CLTV-delta of permanent SCIDs.

@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: Added via_scid field to PPP notifications and a corresponding via_scid TLV to update_add_htlc when forwarding those PPPs, for use with private stateless BOLT11 invoices. @tnull

@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: Remove ephemeral SCIDs.

Copy link
Collaborator

@SeverinAlexB SeverinAlexB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All in all, the spec is already pretty solid IMO.

Comment on lines +125 to +126
* The client requests for an SCID it can use in an invoice via
`lsps4.reserve_scid`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* The client requests for an SCID it can use in an invoice via
`lsps4.reserve_scid`.
* The client requests an SCID it can use in an invoice via
`lsps4.reserve_scid`.

Comment on lines +333 to +359
`expiry_time` is the expiry time for the reserved SCID
[<LSPS0.datetime>][].
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we just use 1 permanent SCID we don't need an expiry_time? In general, we could simplify the whole scid part of this spec enormously in case of only 1 permanent scid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do still need an expiry, for example if the client is trying to DoS and waste the storage of the LSP by reserving and then disconnecting forever. The LSP and the client need to agree on how long that period would be before the LSP can safely forget the client if the client never comes back again.

Copy link
Collaborator

@SeverinAlexB SeverinAlexB Aug 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to expire the permanent SCID? Can't we just use the same SCID for a node pubkey all the time?
In this case, I don't see how the client can DDoS the LSP as there is the same SCID returned every time.

In theory, the LSP can even derive the permanent SCID from the node pubkey. Of course, there needs to be a secret included so others can't guess the SCID.

Permanent SCID = hmac(client pubkey + lsp pubkey + randomLspSecret)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, I don't see how the client can DDoS the LSP as there is the same SCID returned every time.

A client can always generate a new node pubkey, you can literally treat the private key as a large 256-bit number and increment it each time, no need to even use a CSPRNG if you are just using throwaway keypairs for the purpose of DoS. With e.g. Tor or a botnet the IP address is not the same too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I can follow. Let's discuss it on the call today.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not resolved in previous call.

@SeverinAlexB proposes that an actual LSP implementation can use a 63-bit hash of the client node ID, so that the permanent SCID does not need to be remembered.

However, some software such as LDK has its own way of doing SCID interception. LDK gets "first dibs" at the data so LSPs built on top of LDK cannot change the way that LDK creates interception SCIDs.

In addition, something like a hash is a one-way function. So if the LSP receives an interception SCID, it has to iterate through all connected peers and hash each of their node IDs to get the SCID, then find the correct SCID. The LSP can cache this. However, how about disconnected peers? Ideally if the LSP receives an SCID for a peer that is disconnected, it would wake it up (e.g. via LSPS5). So LSP needs to remember the SCID->client mapping, and I think we need to nail down how long the mapping is retained by the LSP after the peer disconnects without making a channel.

LSPS4/README.md Outdated Show resolved Hide resolved
LSPS4/README.md Outdated Show resolved Hide resolved
@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: Remove recommendation on SCID generation, re-add ephemeral SCIDs as an extension.

@ZmnSCPxj-jr
Copy link
Contributor Author

Updated: Remove details.type from lsps4.fail_ppp, revise rationale for the +2 margin in min_final_cltv_delta_expiry.

Copy link
Contributor

@JssDWt JssDWt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on this proposal!
I've left comments here and there, but the general flow of events is exactly what I would expect from a system like this. Also, the considerations are very thorough.

I like the way it is set up for future protocol improvements, like ptlcs. Also, I was blown away by the (future) asyncpay flow and the option to batch open channels.

* The LSP node ID.
* The LSP-provided `cltv_expiry_delta`.
* The expiry for the payment.
* Whether it can use MPP or not.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Whether it can use MPP or not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this? Payer can only use MPP if payee (client in this context) supports it. In theory a payee can be simplified to the point that it does not support MPP at all, so it should signal whether it supports it or not. LSPs are forced to always consider the possibility of MPP, as this lets us also consider the case of multiple unrelated tiny payments by using all of them to fund a single channel open.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I thought this was a remnant of the LSPS2 spec. If the client doesn't support MPP, he can totally put that in the invoice, but I didn't think it was relevant for this spec, because it works regardless of MPP signaling or not.

* MAY have any format, provided:
* MUST be encodable in an ASCII encoded JSON string without any
string escapes needed.
* MUST not be longer than 24 bytes (not including the `"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be 'characters'. Why 24? Assuming a sha256 hash is a good identifier, you would get 64 hex characters here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anything finite is fine. Could go 64. Characters is fine too, as we specify ASCII and in ASCII 1 char = 1 byte.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make it 64 then, to have the hash option.

* Starts a hold timer for that incoming payment, duration depending on
its PPP type.
* Does not respond to the inbound until after the hold timer expires,
or the client has handled the payment fully.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
or the client has handled the payment fully.
or the client has handled the part fully.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this text talks about parts, not payments.

Comment on lines +744 to +750
* Adds the PPP to a called the "PPP Map", which maps the PPP identifier
to the information the LSP needs to forward, handle, or fail the
underlying payment, as well as the information that the client needs to
process for that PPP.
* If the hold timer expires before the client has decided to forward,
fail, or use the PPP for opening a new channel, should also remove
it from the PPP Map.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PPP Map is an implementation detail. I think everything in the spec can be described with a PPP that is being held. In what data structure it is being held is not relevant to the spec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Map" is intended to be a very generic term, not a specific structure. It is just some mapping from PPP identifier to whatever internal details structure your node software needs.

Comment on lines +759 to +761
However, the LSP MAY choose not to persist the PPP Map.
In this case, if the LSP crashes and restarts, it MUST fail all HTLCs that
were held in PPPs with `temporary_channel_failure`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this important?
If the LSP crashes (or restarts), and the PPP identifier would be based on the internal htlc identifier of the lightning node, the flow could continue? The only downside I see is that LSP might send duplicate notifications for the same PPP.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the LSP is able to persist the PPP Map somehow.

In LDK, on restart any intercepted HTLCs are failed automatically with no way for the LSP code to re-send to the client. So we need to specify this (LDK-based LSPs cannot actually store the PPP Map).

(This is the main reason why we talk about the PPP Map, some software can recover on restart, some (LDK) cannot)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In LND the htlcs are replayed when reconnecting to the grpc stream.
In lspd (breez lsp implementation), we replay htlcs from CLN as well when lspd reconnects.

This allows for a 'stateless' handling of these htlcs. When the node restarts, lspd still has the same state, and ignores replayed htlcs. When lspd restarts, the htlcs are replayed and lspd can continue without remembering the previous state.

So maybe just change MUST to MAY here, in case the LSP can recover after crash.

Comment on lines +1798 to +1815
The client MUST NOT use the `alias` SCID sent in the 0-conf
channel open (unless it is equal to the LSPS4 SCID, which
is **not** assured), or the "real" SCID of the channel once
confirmed.
The client MUST use only the SCID issued via `lsps4.reserve_scid`
for all invoices and offers it creates.

The LSP SHOULD fail an incoming HTLC if it uses the `alias` or
"real" SCID of the channel as the next hop, with the failure
`unknown_next_peer`, if the `alias` is not the same as the
LSPS4 SCID.

> **Rationale** Some node software implementations do not have
> an API to set the `alias` of the channel.
> The LSPS4 SCID, in any case, represents all the channels
> with the client, not any specific channel.
> It also represents a possible future channel if there is no
> actual channel between the LSP and the client yet.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be useful for the client to have the option to bypass the LSPS4 spec by issuing invoices with the real alias scid. In that case it is just a normal forward, without the PPP overhead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client can be simpler if it always just uses the LSPS4 SCID. It can issue invoices with 0 regard for current inbound liquidity, or if it issues multiple invoices (some of which may or may not be paid, e.g. merchant application) and does not want to have to allocate inbound liquidity to invoices.

For example, suppose the client currently has 1 unit of inbound. Then it issues two invoices of 1 unit each, with no assurance of whether or not it is going to get paid. If it used the "real" SCID then if both invoices are paid, then the second one will fail due to lack of capacity.

An LSP can ameliorate this by putting the second one in a PPP even though it did not use the LSPS4 SCID. This is equivalent to saying that "the real and/or alias SCID of a channel with a client is an alias of the LSPS4 SCID". But it is much much simpler to just say "client always uses the LSPS4 SCID" for implementation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Say the client selects a new LSP to get its future channels from, it doesn't want new channels from the old LSP. It can still use the channel with the old LSP, by including a 'normal' hop hint in the invoice, and perhaps another hop hint for the new LSP.

Copy link
Contributor Author

@ZmnSCPxj-jr ZmnSCPxj-jr Nov 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Say the client selects a new LSP to get its future channels from, it doesn't want new channels from the old LSP. It can still use the channel with the old LSP, by including a 'normal' hop hint in the invoice, and perhaps another hop hint for the new LSP.

And the "normal" routehint in this case would be the result from lsps4.reserve_scid.

The LSP MUST ensure that all parts forwarded sum up to at least
`payment_size_msat - opening_fee`.

The LSP MUST include an `extra_fee` (type 65536) TLV to the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next to the extra_fee TLV, every lsps4 update_add_htlc probably needs a ppp_id TLV, to correlate the actual HTLC with the PPP.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why though? The PPP exists only as a way to implement onion oracles (where the oracle is the client). In the end an HTLC is an HTLC.

next hop.
When payment arrives at the LSP, it creates pending payment parts
(PPPs, a fancy invented term for HODL HTLCs) for it.
* LSP notifies the client with `lsps4.you_have_new_ppp`, indicating that
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(boring) name suggestion:lsps4.new_ppp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that is so boring.... "You have new mail!!!"

be changed by the LSP at any time, with a grace period during which
the LSP MUST accept both the previous and current settings.
The client is notified of this change via the
`lsps4.i_changed_fee_rates` notification, to be
Copy link
Contributor

@JssDWt JssDWt Sep 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(boring) name suggestion: lsps4.fee_update

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BOOOOOOOOOOORING :P

`lsps4.reserve_scid` has the following defined errors (error `code`
in parenthesis):

* `expiry_too_long` (2) - The `expiry_time` requested by the client
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* `expiry_too_long` (2) - The `expiry_time` requested by the client
* `expiry_too_far` (2) - The `expiry_time` requested by the client

@ZmnSCPxj-jr
Copy link
Contributor Author

TODO: In 2023-11-09 meeting, it was agreed that LSPS4 will specifically state that the SCID returned by lsps4.reserve_scid would be different from any SCIDs issued in LSPS2, but without specifying any specific bit-patterns or versioning bits in the SCID.

check the result of `lsps4.select_version`:

* The extension name is a key of the result object.
* The value of that key is the JSON Boolean `true`.
Copy link

@dzdidi dzdidi Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not able to find nd answer in LSPS0 so just asking here.

What is motivation behind

The value of that key is the JSON Boolean true.

instead of storing extension names in extensions array?

so that response will look like:

{
  "api_version": 1,
   "extensions": ["ephemeral_scids"]
}

or to allow extensions to have some parameterized "nuances", then something like:

{
  "api_version": 1,
   "extensions": [ { "name": "ephemeral_scids" }]
}

The later will allow for more extensibility while keeping the same data format also some more things can be added to lsps4.select_version if necessary.

For example in cases with different expiry_time (518400000 - 6 days), or updated fee rates

{
  "api_version": 1,
   "extensions": [ { 
        "name": "ephemeral_scids", 
         "expiry_time": 518400000,
         "ln_base_fee_msat": 10, 
         "ln_prop_fee": 1,
          "cltv_expiry_delta": 121
 }]
}

Also not sure where the mechanism for notification in case of https://github.com/BitcoinAndLightningLayerSpecs/lsp/blob/db0e99e8b65641a18c2d9b9fdf22c2dbae2af32d/LSPS4/README.md#lightning-forwarding-feerate-change are described

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants