lncli+invoices: experimental key send mode #3795

joostjager · 2019-12-05T11:59:11Z

This PR implements the key send functionality that is described in #2455 in lnd as it is today.

Throughout the year 2019, many PRs have been merged that together created a solid foundation to implement key send on. This is reflected in the size of this PR. The core changes to the invoice registry are relatively small and non-intrusive.

Key send isn't part of the Lightning spec yet. Therefore the preimage is sent as a custom record (in the type range >= 65536):

KeySendRecord uint64 = 5482373484

It is likely that this custom record will be deprecated eventually and replaced by either spontaneous sending as an AMP payment or a an official key send payload record.

What this means is that applications using the functionality as it is introduced in this PR may break when either the sender or receiver upgrades to a newer version of lnd.

As before, intermediate hops will support key send with any version of the software.

To enable receiving key send spontaneous payments, start lnd with the --accept-key-send flag. Without this flag set, spontaneous payments will be rejected.

Sending a spontaneous payment can be done via lncli:

lncli sendpayment -d <dest> -a <amount> --final_cltv_delta 40 --key_send

This PR is also the final step in enabling mainline lnd to send and receive chat messages (Whatsat).

Roasbeef

The ~~Phoenix~~, ~~Sphinx~~, keysend rises!

Roasbeef · 2019-12-11T01:14:04Z

lnrpc/invoicesrpc/utils.go

+
+	paymentRequest := string(invoice.PaymentRequest)
+	if paymentRequest == "" {
+		preimage := invoice.Terms.PaymentPreimage


Do we also want to manually set the feature bit we'll use to signal this on the graph level (in the NodeAnnouncement)?

This would be another way to differentiate this payment from other payments when looking at the set of received payments.

My idea was to keep it minimal as long as it is experimental. The node announcement isn't very critical. You can just try to key send and see if it gets pulled.

The payment can be differentiated because there is no pay req, but I think that can also happen with very old payments. It was already an optional field before. Would you want to add a new (experimental) field on the invoices returns by for example ListInvoices?

The can also be recognized by the custom record that is present and exposed on invoices.

The differentation part is solved now with a is_key_send rpc bool

invoices/invoiceregistry.go

invoices/invoiceregistry_test.go

cmd/lncli/commands.go

Roasbeef · 2019-12-11T01:27:42Z

cmd/lncli/commands.go

+		}
+
+		req.DestCustomRecords = map[uint64][]byte{
+			invoices.KeySendRecord: preimage[:],


Re payload what about also adding a set of optional 32 bytes? This kind of nudges users towards an intended use cases of specifying some sort of account/payment ID which is to be looked up on the backend to accept/deny the payments.

People can do that easily through the rpc interface if they want. I am not sure if we should make assumptions like that they require 32 bytes in lncli. Also, if we always use the same record id for that, different uses may interfere with each other.

invoices/invoiceregistry.go

invoices/invoiceregistry_test.go

cmd/lncli/commands.go

invoices/invoiceregistry.go

lnrpc/invoicesrpc/utils.go

joostjager · 2019-12-11T14:50:20Z

Main open question is at what level(s) we want to support key send. My philosophy for this pr was to keep it as non-intrusive as possible. Postpone adding any new rpc fields until we reconcile with amp. What is in now, is enough for people to start building off of it without requiring a fork of lnd. I think that is good enough given the experimental status of it.

config.go

invoices/invoiceregistry.go

cmd/lncli/commands.go

invoices/invoiceregistry.go

lntest/itest/lnd_single_hop_invoice_test.go

joostjager · 2019-12-20T08:44:08Z

Nil dest_features pr is merged, key send is operational again

cfromknecht

LGTM 🔥

joostjager · 2019-12-22T08:12:36Z

Itest was failing because I happened to choose the same preimage for the key send test that was used in a later test for a hodl invoice. Fixed now.

Previously 5 seconds was used, which made the otherwise fast registry tests relatively slow.

This field isn't optional. It was introduced in 5ed31b1 which was part of release 0.4.0. This release contained breaking database changes, so it is safe to assume that from there on the field is always populated. If there would still be empty payment request fields in the wild, users would also experience issues with ListInvoices. ListInvoices decodes the payment request.

Debug send has been removed some time ago. This is a left over.

Prepares for spontaneous key send payments that do not have a payment request.

Move in separate commit to make next commit more readable.

This commit adds handling code for the key send custom record. If this record is present and its hash matches the payment hash, invoice registry will insert a new invoice into the database "just in time". The subsequent settle flow is unchanged. The newly inserted invoice is picked up and settled. Notifications will be broadcast as usual.

Extend lncli with a flag that indicates the intention to send a spontaneous payment. lncli will generate a preimage/hash combination and send the preimage as a custom record to the recipient.

Roasbeef

LGTM 🛤

Works as advertised on the tin:

   lncli --network=simnet sendpayment --key_send --dest=03d645f08fec4abe8ce02e17c25c3f2ed2b7c40b6aebb725e86bccc92665980054 --amt=100000 --final_cltv_delta=40
{
    "payment_error": "",
    "payment_preimage": "7e85e6bb1fdccd04c1085f1724ee11ebbe32ab6ff9411173fe53032635f37562",
    "payment_route": {
        "total_time_lock": 1704,
        "total_fees": "0",
        "total_amt": "100000",
        "hops": [
            {
                "chan_id": "1820791255662592",
                "chan_capacity": "1000000",
                "amt_to_forward": "100000",
                "fee": "0",
                "expiry": 1704,
                "amt_to_forward_msat": "100000000",
                "fee_msat": "0",
                "pub_key": "03d645f08fec4abe8ce02e17c25c3f2ed2b7c40b6aebb725e86bccc92665980054",
                "tlv_payload": true,
                "mpp_record": null,
                "custom_records": {
                    "5482373484": "7e85e6bb1fdccd04c1085f1724ee11ebbe32ab6ff9411173fe53032635f37562"
                }
            }
        ],
        "total_fees_msat": "0",
        "total_amt_msat": "100000000"
    },
    "payment_hash": "972f041144a5d97d081360f2e8cb84f3bd569d93a1256a4cf721566e553f5ab0"
}

One follow up that I think we should consider is exposing setting custom records on the command line as well, while merging that with the keysend feature if the flag is set. This makes the entire suite a bit more useful, as users can then specify other custom information along with the core keysend record.

joostjager requested a review from Roasbeef as a code owner December 5, 2019 11:59

joostjager force-pushed the keysend branch from bdc7c7a to b84ab5d Compare December 5, 2019 12:00

joostjager added the payments Related to invoices/payments label Dec 5, 2019

joostjager force-pushed the keysend branch 6 times, most recently from 1516edf to ce4ed09 Compare December 5, 2019 19:59

joostjager requested a review from cfromknecht as a code owner December 5, 2019 19:59

joostjager force-pushed the keysend branch from ce4ed09 to 0694843 Compare December 6, 2019 10:09

joostjager self-assigned this Dec 6, 2019

joostjager added tlv v0.9.0 labels Dec 6, 2019

joostjager added this to the 0.9.0 milestone Dec 6, 2019

joostjager force-pushed the keysend branch 4 times, most recently from b0107e9 to 8442b69 Compare December 10, 2019 10:26

Roasbeef requested changes Dec 11, 2019

View reviewed changes

cmd/lncli/commands.go Show resolved Hide resolved

invoices/invoiceregistry.go Show resolved Hide resolved

lnrpc/invoicesrpc/utils.go Show resolved Hide resolved

joostjager force-pushed the keysend branch from 8442b69 to 6a8b54a Compare December 11, 2019 14:47

joostjager requested a review from Roasbeef December 11, 2019 14:48

joostjager force-pushed the keysend branch from 6a8b54a to f6b4844 Compare December 11, 2019 18:34

cfromknecht reviewed Dec 12, 2019

View reviewed changes

config.go Outdated Show resolved Hide resolved

invoices/invoiceregistry.go Outdated Show resolved Hide resolved

invoices/invoiceregistry.go Outdated Show resolved Hide resolved

invoices/invoiceregistry.go Outdated Show resolved Hide resolved

cmd/lncli/commands.go Show resolved Hide resolved

joostjager force-pushed the keysend branch 2 times, most recently from 5d0872f to b133385 Compare December 12, 2019 07:26

joostjager force-pushed the keysend branch from 5906b2d to a7ecc72 Compare December 19, 2019 21:37

cfromknecht reviewed Dec 20, 2019

View reviewed changes

invoices/invoiceregistry.go Show resolved Hide resolved

lntest/itest/lnd_single_hop_invoice_test.go Show resolved Hide resolved

joostjager force-pushed the keysend branch from a7ecc72 to f62b798 Compare December 20, 2019 08:41

joostjager requested a review from cfromknecht December 20, 2019 08:44

cfromknecht approved these changes Dec 20, 2019

View reviewed changes

joostjager force-pushed the keysend branch 5 times, most recently from b246cc6 to 7aad2b6 Compare December 22, 2019 08:08

joostjager added 10 commits December 23, 2019 21:51

invoices/test: lower test delay

c89f0dc

Previously 5 seconds was used, which made the otherwise fast registry tests relatively slow.

lncli: remove debug send

15eedd2

Debug send has been removed some time ago. This is a left over.

lnrpc/invoicesrpc: support empty payment request fields

6063dc3

Prepares for spontaneous key send payments that do not have a payment request.

invoices: move lock acquisition down

202b7c2

Move in separate commit to make next commit more readable.

lnrpc/invoicesrpc: add is_key_send flag to rpc invoice

c08865f

lncli: send key

b1206f6

Extend lncli with a flag that indicates the intention to send a spontaneous payment. lncli will generate a preimage/hash combination and send the preimage as a custom record to the recipient.

lntest/itest: move single hop invoice test to separate file

bb1e2af

lntest/itest: add key send test

4273bc0

joostjager force-pushed the keysend branch from 7aad2b6 to 4273bc0 Compare December 23, 2019 20:53

Roasbeef approved these changes Dec 24, 2019

View reviewed changes

Roasbeef merged commit f289a39 into lightningnetwork:master Dec 24, 2019

Roasbeef mentioned this pull request Dec 25, 2019

[WIP] multi: add new draft key send mode for spontaneous payments #2455

Closed

landgenoot mentioned this pull request Jan 20, 2020

Meeting Notes / Project Diary lightning-sprinkle/master-thesis#1

Open

openoms mentioned this pull request Mar 23, 2020

Keysend -- spontanous payments raspiblitz/raspiblitz#1000

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lncli+invoices: experimental key send mode #3795

lncli+invoices: experimental key send mode #3795

joostjager commented Dec 5, 2019 •

edited

Loading

Roasbeef left a comment

Roasbeef Dec 11, 2019

Roasbeef Dec 11, 2019

joostjager Dec 11, 2019

joostjager Dec 11, 2019

joostjager Dec 12, 2019

Roasbeef Dec 11, 2019

joostjager Dec 11, 2019

joostjager commented Dec 11, 2019

joostjager commented Dec 20, 2019

cfromknecht left a comment

joostjager commented Dec 22, 2019 •

edited

Loading

Roasbeef left a comment

lncli+invoices: experimental key send mode #3795

lncli+invoices: experimental key send mode #3795

Conversation

joostjager commented Dec 5, 2019 • edited Loading

Roasbeef left a comment

Choose a reason for hiding this comment

Roasbeef Dec 11, 2019

Choose a reason for hiding this comment

Roasbeef Dec 11, 2019

Choose a reason for hiding this comment

joostjager Dec 11, 2019

Choose a reason for hiding this comment

joostjager Dec 11, 2019

Choose a reason for hiding this comment

joostjager Dec 12, 2019

Choose a reason for hiding this comment

Roasbeef Dec 11, 2019

Choose a reason for hiding this comment

joostjager Dec 11, 2019

Choose a reason for hiding this comment

joostjager commented Dec 11, 2019

joostjager commented Dec 20, 2019

cfromknecht left a comment

Choose a reason for hiding this comment

joostjager commented Dec 22, 2019 • edited Loading

Roasbeef left a comment

Choose a reason for hiding this comment

joostjager commented Dec 5, 2019 •

edited

Loading

joostjager commented Dec 22, 2019 •

edited

Loading