C-lightning on the RaspiBlitz

[openoms]

Will share the development of the c-lightning implementation here.

The first functional install script is in:
https://github.com/openoms/raspiblitz/blob/cblitz/home.admin/config.scripts/cln.install.sh

The aim is to have c-lightning first as an optional service and later as a full featured alternative of LND.

The development is helped by being able to run signet and testnet parallel as in: #2290

[openoms]

Many new things are functional now:

• CLN menu for basic lightning actions like
• FUNDING
• PEERING
• CHANNEL
• PAY
• REQUEST
• Sparko plugin - providing a web interface and connection to Zeus / Spark Wallet
• C-Lightning-REST for RTL
• further improvements of running the chains and LND/CLN services parallel

Working in the branch https://github.com/openoms/raspiblitz/tree/cblitz which can be synchronised for testing from UPDATE.
Don't use mainnet funds yet please.

[openoms]

The status screen and parallel networks are functional.

If syncing from
https://github.com/openoms/raspiblitz/tree/cblitz
or
https://github.com/rootzoll/raspiblitz/tree/pre-1.7.1
make the new scripts executable with:
chmod +x *.sh
and a restart is necessary.

For C-lightning the keys will be generated from 24 words which will be created by the BIP39 reference implementation: https://github.com/trezor/python-mnemonic#python-mnemonic. This way the on-chain wallet backup will be similar to LND and other wallets.

Automating the seed creation, encryption, decryption and auto-unlock is depending on the solution of ElementsProject/lightning#4570.

[openoms]

Next up:

• Summary plugin for channel stats in menu (https://github.com/lightningd/plugins/tree/master/summary)
• Sparko TLS cert, Hidden Service and menu
• Zeus pairing with QR to CLN-REST
• framework for plugins openoms@1c9fd09 . Resource: https://github.com/lightningd/plugins
• restore and backup-compact with the backup plugin
  https://lightning.readthedocs.io/BACKUP.html#backup-plugin-and-remote-nfs-mount
  https://lightning.readthedocs.io/BACKUP.html#database-file-backups

Note on recovery: Most funds in channels (with the feature: option_static_remotekey) can be recovered after the peer force closes the channel unilaterraly: https://lightning.readthedocs.io/FAQ.html?highlight=option_static_remotekey#database-corruption-channel-state-lost

[openoms]

The latest state is in: https://github.com/rootzoll/raspiblitz/tree/pre-1.7.1

• A round of testing is due after some heavy refactoring.

Outstanding:

• Zeus pairing in menu - config.scripts/cln-plugin.sparko.sh connect testnet
• encrypt / decrypt / autounlock the hsm_secret with the hsmtool
• Suez Suez - Tool for pretty printing and optimizing Lightning Network channels for LND and CLN #2366
• backup the lightningd.sqlite3 to the SDcard with cln-plugin.backup.sh

[openoms]

52d71f7 adds the CLBOSS plugin - The C-Lightning Node Manager
CLBOSS is an automated manager for C-Lightning forwarding nodes.
more info: ttps://github.com/ZmnSCPxj/clboss#operating

install with:
config.scripts/cln-plugin.clboss.sh on

The CLBOSS data is stored in the .lightning dir with all the other node speficic files:

[openoms]

Screenshots from Zeus v0.5.2:

C-lightningREST connection screen:

Spark connection screen (no option to scan a QR):

Managed to connect through both using LAN and Tor, but pasting the strings was manual after scanning them, one by one.

[rootzoll]

@openoms do you think a QR code connection with Zeus for c-lightning is possible within the next 2 weeks or should we target for the v1.7.1 release a workaround - like a short text describing the user how to connect manually?

[openoms]

@rootzoll I think the best we can do is to coordinate with the Zeus team in ZeusLN/zeus#515 to ask them to implement a QRcode parsing the way it will be implemented in the RaspiBlitz.

Spark-wallet already does use a QRcode format, but it needs a CA signed certificate (or importing the certificate in the phone which is far from easily atm).
Should try if it can be used with LetsEncrypt HTTPS Domain subscription.

[rootzoll]

@openoms On a test build of the modified sd build script with c-lightning I ran into this that made the build stop:

  inflating: clightning-v0.10.1/wire/wire_sync.h
- Configuring EXPERIMENTAL_FEATURES enabled
Compiling ccan/tools/configurator/configurator.../usr/bin/ld: cannot open output file ccan/tools/configurator/configurator: Permission denied
collect2: error: ld returned 1 exit status
- Building from source C-lightning
File config.vars not found: you must run ./configure before running make.
make: *** No rule to make target 'config.vars', needed by 'show-flags'.  Stop.
- Install to /usr/local/bin/
File config.vars not found: you must run ./configure before running make.
mkdir -p /bin
mkdir -p /libexec/c-lightning
mkdir -p /libexec/c-lightning/plugins
mkdir -p /share/man/man1
mkdir -p /share/man/man5
mkdir -p /share/man/man7
mkdir -p /share/man/man8
mkdir -p /share/doc/c-lightning
make: *** No rule to make target 'config.vars', needed by 'ccan/config.h'.  Stop.

This seems where the problem is:

pi@raspberrypi:/home/admin/download/cln/clightning-v0.10.1 $ sudo -u admin ./configure --enable-experimental-features
Compiling ccan/tools/configurator/configurator.../usr/bin/ld: cannot open output file ccan/tools/configurator/configurator: Permission denied
collect2: error: ld returned 1 exit status

[rootzoll]

It runs when using just sudo @openoms is there a specific reason why building it with sudo -u admin

[openoms]

@rootzoll wanted to avoid to run the compilation with root, but does not make much difference really.
I have tested the script from the admin user, not pi, which could be the difference.

fixed in 656be6d (tested by running with sudo from the user pi)

[openoms]

@openoms do you think a QR code connection with Zeus for c-lightning is possible within the next 2 weeks or should we target for the v1.7.1 release a workaround - like a short text describing the user how to connect manually?

I think since the QR format is still in flux we should show the access-key or macaroon in itself in a QR and put the explanation to paste bit manually to Zeus.

Can have two steps switching the QRcode on the LCD after enter:
first would just show the access-key or macaroon
second the onion address

so they can be scanned individually and pasted in the app manually.

[rootzoll]

@openoms sounds good as an temp workaround. Will you work on that?

[openoms]

@openoms sounds good as an temp workaround. Will you work on that?

sure will sort it

[rootzoll]

This will be contained in v1.7.1rc1 for further intense testing.

[openoms]

Added a CLNREPAIR menu to cover the functions of cln.hsmtool.sh in the dialogs

The hsm_secret password is equivalent to the Password_C for LND.
By default the hsm_secret is stored in cleartext on the disk.
If encrypted the hsm_secret password is kept in memory between restarts.
The AUTOUNLOCK means that the hsm_secret password is stored on the disk in cleartext.

Creating a new C-ligtning wallet will now create a seedwords.info beneath the hsm_secret with the contents:

# This file was placed by cln.hsmtool.sh .
# Contains the seed words from which the hsm_secret in the same directory was generated
seedwords='${seedwords}'
seedwords6x4='${seedwords6x4}'
# Will be removed safely when the hsm_secret is encrypted.

# This file is placed by cln.hsmtool.sh .
# The seed words from which the hsm_secret in the same directory was generated
# were deleted when the hsm_secret was encrypted.
# The words cannot be generated from the hsm_secret (one way function).
# If you don't have the words the hsm_secret can be still backed up in hex:
# https://lightning.readthedocs.io/BACKUP.html#hsm-secret 

[openoms]

Some simpler tasks to do next:

• CLBOSS - Automatic channel management for C-lightning

• SPARK - Spark Wallet to manage the C-Lightning wallet (possible replacement for Sparko)

• Update https://github.com/Ride-The-Lightning/c-lightning-REST/releases/

• Add a custom binary as a C-ligthning plugin - this likely will be a manual guide to start with due to the lack of standards as now

• List closed channels with details - waiting for a plugin do this other wise if is a potentially harmful direct (external) sqllite query

[rootzoll]

OK added "display-seed" option to cln.install.shthat now will be called if user activated the secondary/parallel cln wallet.

[openoms]

Future integrations will be recorded for the v1.8 milestone

• LNBits backed by CLN - [CLN] LNBits backed by C-lightning #2556

[rootzoll]

@openoms for the CLN LNbits best would be to open a ticket on v1.8.0 milestone.

[openoms]

Final tests:

• CL BACKUP
• CL SEEDRESTORE
• CL FILERESTORE

[rootzoll]

I switched an LND node to CL default and the deactivated runBehindTor .. CL seems not to start up in that configuration:

*** C-LIGHTNING (MAINNET) SYSTEMD STATUS ***
● lightningd.service - c-lightning daemon on mainnet
   Loaded: loaded (/etc/systemd/system/lightningd.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Mon 2021-10-11 15:53:30 BST; 3s ago
  Process: 25871 ExecStartPre=/home/admin/config.scripts/cl.check.sh prestart mainnet (code=exited, status=0/SUCCESS)
  Process: 25880 ExecStart=/bin/sh -c /usr/local/bin/lightningd --conf=/home/bitcoin/.lightning/config  (code=exited, status=1/FAILURE)
 Main PID: 25880 (code=exited, status=1/FAILURE)

*** LAST 30 C-LIGHTNING (MAINNET) INFO LOGS ***
sudo tail -n 30 /mnt/hdd/lnd/logs/bitcoin/mainnet/lnd.log
2021-10-11T14:52:24.958Z UNUSUAL plugin-backup.py:   File \"/home/bitcoin/cl-plugins-enabled/backup.py\", line 135, in <module>
2021-10-11T14:52:24.958Z UNUSUAL plugin-backup.py:     kill(\"Could not find backup.lock in the lightning-dir\")
2021-10-11T14:52:24.959Z UNUSUAL plugin-backup.py:   File \"/home/bitcoin/cl-plugins-enabled/backup.py\", line 110, in kill
2021-10-11T14:52:24.959Z UNUSUAL plugin-backup.py:     procs = [p for p in psutil.Process(os.getpid()).parents()]
2021-10-11T14:52:24.959Z UNUSUAL plugin-backup.py: AttributeError: 'Process' object has no attribute 'parents'
2021-10-11T14:52:25.006Z INFO    plugin-backup.py: Killing plugin: exited before replying to getmanifest

[rootzoll]

@openoms the problem seems to be

2021-10-11T14:56:10.765Z INFO    plugin-backup.py: Could not find backup.lock in the lightning-dir

[openoms]

will,need to run:
config.scripts/cl-plugin.backup.sh init
if that lock file is not present I think. Will have a look.

[openoms]

@rootzoll to be clear I did not test C-lightning without Tor so this combination will not work with v1.7.1.

Otherwise all the SWITCHLN option does is changes to ligthning=cl in the raspiblitz.conf so the menus and scripts can accommodate.

[rootzoll]

We need to refactor the clearnet mode for v1.8.0 anyway. So its OK for now.
@openoms let me know when you have the cl-plugin.backup.sh init fix ready .. then I think the issue is ready to close for v1.7.1 release.

[openoms]

The backup plugin is already always reinitialized when cl-plugin.backup.sh on is run
During:

• recovery with FILERESTORE
• new wallet creation new or from seed
• and installation
  https://github.com/rootzoll/raspiblitz/blob/1812e0625860abeb1ad3560e98d480dcbde1a8d7/home.admin/config.scripts/cl-plugin.backup.sh#L67

Could not reproduce the issue just by switching to the CL node to be default.

[rootzoll]

OK .. so lets render this as an edge case for now and see what reports come in on this after final release. If there is something we didnt see we can always do a quick 1.7.2 before 1.8.0 - closing this issue.