-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LSPS4: Continuous JIT Channels. #37
base: main
Are you sure you want to change the base?
LSPS4: Continuous JIT Channels. #37
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think a process like this with the pending payment parts would be great. It allows for several things
- multipart payments
- 0 amount invoices or overpayment in general
- wallets that send probes before the actual payment
All in all this seamlessly integrates with most functionality available in the lighting network today.
Comments below on how to structure the messaging. I think the queue structure could be removed if the client is notified of every ppp.
21fa770
to
4abc8dd
Compare
Updated: Always PPP-hold all HTLCs, change |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really enjoy the considerations made here for privacy, MPP, eventual async/ptlc, and long-lasting relationships with clients. Seems like it solves several issues I have considered with LSPS2.
Is the justification between the two specs understood to be that LSPS2 is the simple, first pass approach that LSPs will implement first while working to support LSPS4? Or is LSPS4 supposed to replace LSPS2 (or not even merge LSPS2 at all)?
That (LSPS2 is the simple first-pass, LSPS4 later) is the current plan inside Block at least. |
a759fa9
to
8a3ad1d
Compare
8a3ad1d
to
716b66b
Compare
Updated: change API version negotiation scheme, start copying the bits from LSPS2, clarify about |
716b66b
to
d084455
Compare
Updated: explain why all HTLCs need to be put into PPPs even if the client already has inbound capacity |
d084455
to
f742ad5
Compare
f742ad5
to
2501240
Compare
Updated: fill in details from LSPS2, mark as ready for review, lots of changes. |
2501240
to
ac5a7ed
Compare
Updated: Move some text blocks around, explain permanent SCIDs a bit better, remove duplicated paragraph. |
3619031
to
87728f9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First review pass (will need more passes in the future). In general, I like the approach. It's definitely more complicated than LSPS2. It's important we are aware of how Route Blinding works to find the right balance between privacy and features.
Thx Zmn for this proposal.
87728f9
to
710dbf4
Compare
Updated: use names instead of one-character codes for types (e.g. |
710dbf4
to
eee4ba8
Compare
Updated: Added |
eee4ba8
to
de7304b
Compare
Updated: Remove |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All in all, the spec is already pretty solid IMO.
* The client requests for an SCID it can use in an invoice via | ||
`lsps4.reserve_scid`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* The client requests for an SCID it can use in an invoice via | |
`lsps4.reserve_scid`. | |
* The client requests an SCID it can use in an invoice via | |
`lsps4.reserve_scid`. |
`expiry_time` is the expiry time for the reserved SCID | ||
[<LSPS0.datetime>][]. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we just use 1 permanent SCID we don't need an expiry_time
? In general, we could simplify the whole scid part of this spec enormously in case of only 1 permanent scid.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do still need an expiry, for example if the client is trying to DoS and waste the storage of the LSP by reserving and then disconnecting forever. The LSP and the client need to agree on how long that period would be before the LSP can safely forget the client if the client never comes back again.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to expire the permanent SCID? Can't we just use the same SCID for a node pubkey all the time?
In this case, I don't see how the client can DDoS the LSP as there is the same SCID returned every time.
In theory, the LSP can even derive the permanent SCID from the node pubkey. Of course, there needs to be a secret included so others can't guess the SCID.
Permanent SCID = hmac(client pubkey + lsp pubkey + randomLspSecret)
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case, I don't see how the client can DDoS the LSP as there is the same SCID returned every time.
A client can always generate a new node pubkey, you can literally treat the private key as a large 256-bit number and increment it each time, no need to even use a CSPRNG if you are just using throwaway keypairs for the purpose of DoS. With e.g. Tor or a botnet the IP address is not the same too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure I can follow. Let's discuss it on the call today.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not resolved in previous call.
@SeverinAlexB proposes that an actual LSP implementation can use a 63-bit hash of the client node ID, so that the permanent SCID does not need to be remembered.
However, some software such as LDK has its own way of doing SCID interception. LDK gets "first dibs" at the data so LSPs built on top of LDK cannot change the way that LDK creates interception SCIDs.
In addition, something like a hash is a one-way function. So if the LSP receives an interception SCID, it has to iterate through all connected peers and hash each of their node IDs to get the SCID, then find the correct SCID. The LSP can cache this. However, how about disconnected peers? Ideally if the LSP receives an SCID for a peer that is disconnected, it would wake it up (e.g. via LSPS5). So LSP needs to remember the SCID->client mapping, and I think we need to nail down how long the mapping is retained by the LSP after the peer disconnects without making a channel.
de7304b
to
8501e9d
Compare
Updated: Remove recommendation on SCID generation, re-add ephemeral SCIDs as an extension. |
8501e9d
to
db0e99e
Compare
Updated: Remove |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work on this proposal!
I've left comments here and there, but the general flow of events is exactly what I would expect from a system like this. Also, the considerations are very thorough.
I like the way it is set up for future protocol improvements, like ptlcs. Also, I was blown away by the (future) asyncpay flow and the option to batch open channels.
* The LSP node ID. | ||
* The LSP-provided `cltv_expiry_delta`. | ||
* The expiry for the payment. | ||
* Whether it can use MPP or not. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* Whether it can use MPP or not. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why remove this? Payer can only use MPP if payee (client in this context) supports it. In theory a payee can be simplified to the point that it does not support MPP at all, so it should signal whether it supports it or not. LSPs are forced to always consider the possibility of MPP, as this lets us also consider the case of multiple unrelated tiny payments by using all of them to fund a single channel open.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I thought this was a remnant of the LSPS2 spec. If the client doesn't support MPP, he can totally put that in the invoice, but I didn't think it was relevant for this spec, because it works regardless of MPP signaling or not.
* MAY have any format, provided: | ||
* MUST be encodable in an ASCII encoded JSON string without any | ||
string escapes needed. | ||
* MUST not be longer than 24 bytes (not including the `"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should be 'characters'. Why 24? Assuming a sha256 hash is a good identifier, you would get 64 hex characters here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Anything finite is fine. Could go 64. Characters is fine too, as we specify ASCII and in ASCII 1 char = 1 byte.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's make it 64 then, to have the hash option.
* Starts a hold timer for that incoming payment, duration depending on | ||
its PPP type. | ||
* Does not respond to the inbound until after the hold timer expires, | ||
or the client has handled the payment fully. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or the client has handled the payment fully. | |
or the client has handled the part fully. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah this text talks about parts, not payments.
* Adds the PPP to a called the "PPP Map", which maps the PPP identifier | ||
to the information the LSP needs to forward, handle, or fail the | ||
underlying payment, as well as the information that the client needs to | ||
process for that PPP. | ||
* If the hold timer expires before the client has decided to forward, | ||
fail, or use the PPP for opening a new channel, should also remove | ||
it from the PPP Map. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PPP Map is an implementation detail. I think everything in the spec can be described with a PPP
that is being held. In what data structure it is being held is not relevant to the spec.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Map" is intended to be a very generic term, not a specific structure. It is just some mapping from PPP identifier to whatever internal details structure your node software needs.
However, the LSP MAY choose not to persist the PPP Map. | ||
In this case, if the LSP crashes and restarts, it MUST fail all HTLCs that | ||
were held in PPPs with `temporary_channel_failure`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this important?
If the LSP crashes (or restarts), and the PPP identifier would be based on the internal htlc identifier of the lightning node, the flow could continue? The only downside I see is that LSP might send duplicate notifications for the same PPP.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assuming the LSP is able to persist the PPP Map somehow.
In LDK, on restart any intercepted HTLCs are failed automatically with no way for the LSP code to re-send to the client. So we need to specify this (LDK-based LSPs cannot actually store the PPP Map).
(This is the main reason why we talk about the PPP Map, some software can recover on restart, some (LDK) cannot)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In LND the htlcs are replayed when reconnecting to the grpc stream.
In lspd (breez lsp implementation), we replay htlcs from CLN as well when lspd reconnects.
This allows for a 'stateless' handling of these htlcs. When the node restarts, lspd still has the same state, and ignores replayed htlcs. When lspd restarts, the htlcs are replayed and lspd can continue without remembering the previous state.
So maybe just change MUST to MAY here, in case the LSP can recover after crash.
The client MUST NOT use the `alias` SCID sent in the 0-conf | ||
channel open (unless it is equal to the LSPS4 SCID, which | ||
is **not** assured), or the "real" SCID of the channel once | ||
confirmed. | ||
The client MUST use only the SCID issued via `lsps4.reserve_scid` | ||
for all invoices and offers it creates. | ||
|
||
The LSP SHOULD fail an incoming HTLC if it uses the `alias` or | ||
"real" SCID of the channel as the next hop, with the failure | ||
`unknown_next_peer`, if the `alias` is not the same as the | ||
LSPS4 SCID. | ||
|
||
> **Rationale** Some node software implementations do not have | ||
> an API to set the `alias` of the channel. | ||
> The LSPS4 SCID, in any case, represents all the channels | ||
> with the client, not any specific channel. | ||
> It also represents a possible future channel if there is no | ||
> actual channel between the LSP and the client yet. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It could be useful for the client to have the option to bypass the LSPS4 spec by issuing invoices with the real alias scid. In that case it is just a normal forward, without the PPP overhead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Client can be simpler if it always just uses the LSPS4 SCID. It can issue invoices with 0 regard for current inbound liquidity, or if it issues multiple invoices (some of which may or may not be paid, e.g. merchant application) and does not want to have to allocate inbound liquidity to invoices.
For example, suppose the client currently has 1 unit of inbound. Then it issues two invoices of 1 unit each, with no assurance of whether or not it is going to get paid. If it used the "real" SCID then if both invoices are paid, then the second one will fail due to lack of capacity.
An LSP can ameliorate this by putting the second one in a PPP even though it did not use the LSPS4 SCID. This is equivalent to saying that "the real and/or alias SCID of a channel with a client is an alias of the LSPS4 SCID". But it is much much simpler to just say "client always uses the LSPS4 SCID" for implementation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Say the client selects a new LSP to get its future channels from, it doesn't want new channels from the old LSP. It can still use the channel with the old LSP, by including a 'normal' hop hint in the invoice, and perhaps another hop hint for the new LSP.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Say the client selects a new LSP to get its future channels from, it doesn't want new channels from the old LSP. It can still use the channel with the old LSP, by including a 'normal' hop hint in the invoice, and perhaps another hop hint for the new LSP.
And the "normal" routehint in this case would be the result from lsps4.reserve_scid
.
The LSP MUST ensure that all parts forwarded sum up to at least | ||
`payment_size_msat - opening_fee`. | ||
|
||
The LSP MUST include an `extra_fee` (type 65536) TLV to the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Next to the extra_fee
TLV, every lsps4 update_add_htlc
probably needs a ppp_id
TLV, to correlate the actual HTLC with the PPP.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why though? The PPP exists only as a way to implement onion oracles (where the oracle is the client). In the end an HTLC is an HTLC.
next hop. | ||
When payment arrives at the LSP, it creates pending payment parts | ||
(PPPs, a fancy invented term for HODL HTLCs) for it. | ||
* LSP notifies the client with `lsps4.you_have_new_ppp`, indicating that |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(boring) name suggestion:lsps4.new_ppp
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But that is so boring.... "You have new mail!!!"
be changed by the LSP at any time, with a grace period during which | ||
the LSP MUST accept both the previous and current settings. | ||
The client is notified of this change via the | ||
`lsps4.i_changed_fee_rates` notification, to be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(boring) name suggestion: lsps4.fee_update
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BOOOOOOOOOOORING :P
`lsps4.reserve_scid` has the following defined errors (error `code` | ||
in parenthesis): | ||
|
||
* `expiry_too_long` (2) - The `expiry_time` requested by the client |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* `expiry_too_long` (2) - The `expiry_time` requested by the client | |
* `expiry_too_far` (2) - The `expiry_time` requested by the client |
TODO: In 2023-11-09 meeting, it was agreed that LSPS4 will specifically state that the SCID returned by |
check the result of `lsps4.select_version`: | ||
|
||
* The extension name is a key of the result object. | ||
* The value of that key is the JSON Boolean `true`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was not able to find nd answer in LSPS0
so just asking here.
What is motivation behind
The value of that key is the JSON Boolean
true
.
instead of storing extension names in extensions
array?
so that response will look like:
{
"api_version": 1,
"extensions": ["ephemeral_scids"]
}
or to allow extensions to have some parameterized "nuances", then something like:
{
"api_version": 1,
"extensions": [ { "name": "ephemeral_scids" }]
}
The later will allow for more extensibility while keeping the same data format also some more things can be added to lsps4.select_version
if necessary.
For example in cases with different expiry_time
(518400000 - 6 days), or updated fee rates
{
"api_version": 1,
"extensions": [ {
"name": "ephemeral_scids",
"expiry_time": 518400000,
"ln_base_fee_msat": 10,
"ln_prop_fee": 1,
"cltv_expiry_delta": 121
}]
}
Also not sure where the mechanism for notification in case of https://github.com/BitcoinAndLightningLayerSpecs/lsp/blob/db0e99e8b65641a18c2d9b9fdf22c2dbae2af32d/LSPS4/README.md#lightning-forwarding-feerate-change are described
Alternative to #22
Closes: #25
Closes: #26
Much more complex than the currrent LSPS2 spec.
Note: we really should not be clearing state based on disconnections of the peer, because CLN, LND, and LDK have race conditions between connection/disconnection notification and receiving and sending of custom messages. This is why this is unnecessarily complex as both sides kinda need to persist information across connection sessions for robustness against this race condition.
Advantages:
The major disadvantage is a big increase in complexity in both LSP and client sides. Not even sure this is implementable with current node software LOL.