Chantools derives the wrong identity key

[cryptosharks131]

While trying to assist in the recovery of funds described in #41, I was presented with an error that the derived identity key was not found in the match file during step [1/3] using the zombierecovery prepare keys command. Further investigation by running derivekey --neuter --identity yielded the same identity which is not the expected node identity for this seed phrase.

At first, I thought the seed phrase was wrong for the node (big problem!) but I had verified it previously with the lnd recovery process, so I setup another test to verify my previous results. I installed a fresh copy of bitcoind/lnd along side the chantools to compare. When loading a wallet with lncli create and copy/pasting the seed used with chantools, I am indeed able to recover the correct identity of my node (good news, hopefully?).

[guggero]

Hmm... I've used this command many times and it always worked for me.
How exactly are you passing the seed to chantools? Are you pasting it in when it ask you for it on the terminal or through the --rootkey flag? Or an env variable?

[guggero]

Can you check if you get the correct identity if you use https://guggero.github.io/cryptography-toolkit/#!/aezeed, take the xprv and use that as --rootkey for chantools?

[cryptosharks131]

The seed phrase was used directly in all cases. I was able to use chantools with a different seed phrase to recover a working testnet3 instance without any issue by including the -t flag with the derivekey command.

In this case the seed can not be moved from the secure/headless environment and that creates some issues for using your web based tool. Is there any difference in the process of derivekey and lncli create which could cause a discrepancy?

I did go through the steps again of verifying the seed phrase in both chantools and LND to rule out something strange occurring the first time. Again, I got a different result in chantools than I got from LND (expected value).

[guggero]

What version of chantools are you using? I did change something related to key derivation in 2f5f081, which was wrong in any version <=v0.10.0. So if you download v0.10.1, do you still see the discrepancy?

[cryptosharks131]

This was using the most recent version v0.10.1 of chantools.

[guggero]

Damn, then I don't know what could cause the difference. Going to do some testing to dig deeper.

[guggero]

Okay, I found a seed that produces a different node key with chantools than lnd:

about hockey ahead embark sound burger horse secret violin equip father erupt episode stay valley vote record true high voice hip drum sausage neglect

It looks like 2f5f081 is the culprit for the difference. But I cannot yet explain why after this change the result is different, since lnd seems to always use the old DeriveNonStandard way. Something is definitely very strange here.

[guggero]

Okay, I finally figured it out... Because the underlying btcwallet in lnd serializes the key at different levels, that by itself fixes a padding issue of the key derivation library.
So I just needed to replicate that behavior exactly in chantools. See af563bb.

Closed by af563bb. Going to release a fixed version v0.10.2 in a few moments.

[cryptosharks131]

Works, thanks for the quick fix!