Introduce NIP-44 encryption standard #715
Conversation
|
+Version 0 is not defined, however implementations depending on this NIP MAY choose to support the payload described in NIP 04 in the same places a NIP 44 payload would otherwise be expected. This is intended to allow a smooth transition while clients and signing software adopt the new standard.
nip4 is broken cryptography that will expose your private keys. It should never be used.
do you have a demo of this?
|
|
because if this is true then there should be an advisory and every client should remove it? |
This is an important point. Signing applications don't support the new version, so compatibility with NIP 04 could ease adoption (or may it more complicated, I'm not sure). @paulmillr can you confirm how bad it would be to enshrine NIP 04 as a deprecated version 0? |
|
Looping in a comment from #706 (review) we have a few different proposals for the payload:
I'm going to leave what we currently have unless anyone has strong opinions so we can put this to rest. |
|
base64 is useful, because the result data is binary, and representing it in hex instead of base64 would increase the size by 1.5x in my benchmarks. Saving a traffic is cool. |
|
Just for completeness, the most performant option is a custom TLV, which we also already implement in Nostr. CSV is fast, but not as fast/simple as TLV because of sequential bytearray copy instructions. We can even do the inner gossip event as a TLV as well. I would still go for JSON. |
|
Before further discussion, I would like to remind that nostr DMs, if nostr gets a traction, would be used by all kinds of people, including dissidents in dangerous places. Would you be willing to risk someone getting killed just because you want to keep backwards compatibility with bad encryption?
Footnotes
|
|
Thank you, that's exactly the kind of answer I was looking for. I'm so glad you're here to help us devs get things right. I will remove the recommendation for NIP 04 and will not include backwards compatibility in my implementation. Tangentially, does this mean that anyone who has sent a substantial number of NIP 04 messages is more likely to have their key brute-forced? |
Yes. That's from my understanding, but I think any auditor of the protocol would come to similar conclusions. |
|
Well that sucks haha |
|
Why is custom TLV fast? |
CSV Parser has to walk the byte sequence until it finds a comma. A TLV has length before the byte array. You can just do an arraycopy(0,length) and move the pointer to the next type value. |
|
@fiatjaf what do you think? It's just encryption here, that can be used in any part of nostr. No DMs. |
Wasn't obvious to me, I was today years old when I learned the term "malleable".
Added that to the NIP. Making edits is no problem, if you see problems in the PR feel free to suggest changes in github and I'll merge them. |
|
And this affects all those test vectors. EDIT: hang on... I don't know how getSharedSecret() maps to libsecp256k1_ecdh(). And doing a double SHA256 in my code still isn't getting me data that matches the test vectors. But running the typescript does match the test vectors. EDIT2: OK if the JavaScript was like this (without the subarray(1,33) on the shared secret output), then the shared key matches my rust code's output (which does not explicitly run a SHA256). export const getSharedSecret = (privkey: string, pubkey: string): Uint8Array =>
sha256(secp256k1.getSharedSecret(privkey, '02' + pubkey))The shared key for test vector 0 is b1c9938f01121e159887ac2c8d393a22e4476ff8212de13fe1939de2a236f0a7 |
|
I share my testing code here: https://github.com/mikedilger/nip44tests |
|
@mikedilger With regards to padding, I support your decision number 2, it's simple and no-bs. As for ecdh:
There is no single ECDH standard, unfortunately. For example, this rust library returns 64-byte unhashed point. What's needed: investigating more libraries to see how they behave and if they can be simply adjusted to one way or another. See Thomases comment on stack overflow:
We just need to understand all the use-cases of the NIP and decide if the result message would always be signed with schnorr. If they won't, perhaps it's worth switching to poly1305. |
|
Ok @paulmillr I have gotten my code to match and I get it now. The rust lib |
I think that's a safe assumption for now, we can introduce a new version if we need it |
|
Stupid question, but isn't the event id/hash also serving the same function the poly1305 thing would? |
|
Implemented this on https://github.com/nbd-wtf/go-nostr/blob/cd86ee25147ac3ec26326bbac52f90ebe2fb3710/nip44/nip44.go with the test vectors. The test vectors don't check the type of the base64 encoding though. |
hash can be re-calculated if the underlying data is changed. poly1305 is like hmac. they both receive so, Again, signatures resolve all the issues here. It's not possible to forge a new message that's valid for the same signature. Unless, of course, you're using shitty malleable signature scheme. Bonus point: Poly1305 is malleable and its outputs can be forged. Bonus point 2: tweetnacl implementation of ed25519 is malleable, so was secp256k1 ecdsa some years ago. @fiatjaf I want to pursue padding addition, so this will need another day or so. Above, the good point was made with regards to it: |
|
I think this PR needs:
|
|
Check on 2 and 3, just waiting for test vectors and padding. On padding, we cover that in #686, but a lower-level version would probably be better. |
| Currently defined encryption algorithms: | ||
|
|
||
| - `0x00` - Reserved | ||
| - `0x01` - XChaCha20 with same key `sha256(parity + ecdh)` per conversation |
There was a problem hiding this comment.
Actually the parity byte is part of the output of ecdh for most libraries, as is the sha256 part.
I would put something like ecdh(private_key, public_key) where the ecdh() function computes the shared point, and then returns the sha256 hash of the x-only component prefixed by the parity byte ('02').
There was a problem hiding this comment.
except that the libraries implement ECDSA and not schnorr. There is no parity byte in schnorr.
There was a problem hiding this comment.
I only understand a little. I thought the parity byte represented the Y coordinate and there were two options since the curve loops back around on itself. And I know that with Schnorr signatures it is always even (2) for some reason. And I thought DH just did successive multiplications to come up to the same point g^a * g^b = g^b * g^a and I don't understand where either signature algorithm (DSA or Schnorr) comes in when determining the DH shared point (and does ECDSA even have a shared point?).
All I want is for the NIP to do the default thing that libraries do, rather than a custom thing that requires lower-level operations.
|
I don't even know how to query what RFC4648 is. Is that what the default |
|
Besides everything, we need to analyze if the scheme is CCA-secure. I think the API we provide to users should be bulletproof and not allow to shooting oneself into foot. Perhaps build encryption into event creation process PaddingFor padding, i've did a deep dive a few days ago and padme seems to be good: https://en.wikipedia.org/wiki/Padding_(cryptography)#Traffic_analysis_and_protection_via_padding JS version is shitty, need to rewrite it https://github.com/sh-dv/padme/blob/main/index.js ECDH
To reiterate:
Both schemes do not consider Schnorr signatures which work over X coordinates without parity bytes like ECDSA. On the other hand, there seem to be a separate development for X-only-ECDH-over-secp256k1. We need to decide if |
|
Are you guys able to make suggestions on the PR? This is beyond me. |
Section 4 of https://www.rfc-editor.org/rfc/rfc4648 specifies the alphabet and pad character. Some variants use different symbols for character 62 and 63 to be "url safe" for example. Some variants don't use padding characters (they are implied by the length). But that RFC is the original canonical base64 and if library doesn't give you options to pass into their base64() function, it is very probably the right one. |
The lack of randomness in the choice of key, and then using XOR, leads me to think it is not. If you take an unknown message, XOR it with your chosen plaintext, do you get back the secret stream? That would be a serious flaw. EDIT: Scratch that. I momentarily forgot that XChaCha has this huge random nonce. Carry on. |
|
I like PADMÉ it is elegant. But it presumes that if you want to send a 3-character message, using 6 characters would be a huge waste of space, and that there are probably lots of 3-character messages so no padding is needed. But my "yes"/"no" example still stands. In our case, given the overhead of every event, I think every message should be padded out to a minimum of 32 bytes. So I'm proposing MAX(padme(len), 32) |
|
32 bytes min seems fine. Any objections? Maybe 24? |
|
NIP44 is not robust against chosen-ciphertext attack. See code that proves it.
We need to change everything here. I don't think poly1305 would help, since it can be forged. |
|
I don't get it. So what if Mallory signs encrypted content that she doesn't understand. Bob will get the message, verify the valid signature, and it will decrypt into garbage because it was encrypted with Alice's key, not Mallory's. If alternatively Bob gets it thinking it is Alice, the signature will not be valid. |
Not certain, it may decrypt into a valid text sometimes.
It's definition of CCA. It could leak the key in extreme cases. |
|
Closed in favor of #746 |
A distilled version of #574 focused on the cryptography standard, and omitting DM-specific stuff. @paulmillr please feel free to incorporate this diff into the original PR and I can close this one, or close the original and we can continue from here, it's up to you.