NIP44 encryption standard, revision 3 #746
Conversation
|
If this is different than v1, can we do a v2 instead? We made versions because of these types of evolutions. |
|
The old, non-secure revision was never merged into NIP repo, nor was it finalized, nor was it reviewed for security by outside auditors. Some people implemented JSON-stringified revision, instead of concatenated one. What about them? I don't mind switching to |
|
I don't think it needs a version bump. We are pre-versioning still. Maybe in our experimental code we should be using version 0. |
|
I expect versions 3, 4, 5 to follow quickly after 2. So, I don't see any reason to keep replacing v1 all the time. Yes, we already did with the JSON stuff, but the WHOLE POINT of making a versioned standard is to afford these changes without breaking the past. I don't understand why proposing a versioned payload if we are not going to use the versions. The fact that hasn't been merged is irrelevant. |
|
If there are events floating around using v1 out there this should be v2, but yes, let's just not describe v1 in the spec. |
|
There are over 5K GiftWraps with v1 already, growing steadily every day. So, please let's use what we built and start v2. |
Of course, but the fact that the scheme has not been audited or reviewed is relevant. Do you think anyone should deploy any e2ee dm scheme without that?
That would mean any nostr client would need to bring a huge amount of different crypto primitives if they would be different. |
Yep. They will no matter what happens. There are always messages in the past that must be recovered. So, all clients will likely implement all versions no matter what we do here. That's the curse of signing things that will last forever. They can always choose to implement just the most recent ones at the cost of making users angry for not recovering older ones. |
|
In any case, with or without v2 designation, don't implement this until nostr hires someone to do an external security audit. Preferably, even two different companies. |
This is going to take time. Even more reasons to keep v1 and v2 separate. We can keep testing v1 and when v2 is ready, we all switch. Maybe name it a V2-DRAFT? |
Fine. I did not know that. I have 3 or 4 versions of code to deal with encrypted private keys, and that NIP isn't accepted either. |
How do we do this? |
|
Ping an auditor or two, discuss the terms and scope. I've did this in the past. If you can secure the funding, I can talk to someone again. |
|
@paulmillr maybe you could apply to OpenSats to get a grant to cover this? |
|
I do think we should keep this as v1 and get it merged. A full security audit is going to take some time — v1 is much better than NIP 04, which will allow us to keep innovating on nostr while these details get figured out. Shall I re-open #715 and we can look at merging it? |
|
What's the rush? It's not like people don't have a place to chat right now. Proper encryption is complicated. If you want to speed-up things, let's hire an auditor and maybe a second protocol designer to help developing the scheme further. People are going to get killed if this is not right and nostr becomes popular. Do you want people to get killed because you want your client to "get DMs first"? |
|
You can't prevent people from building on NIP 04 by not merging this proposal. |
It's not about rush. Perfection is the enemy of the good. If the first version is good enough (and better than NIP-04), we should move forward until this one gets ready. Most clients are still doing NIP-04, which I think by now we all agree is not secure and must be replaced. |
|
There is software that could get released and iterated asap. There is software that should not. BIP340, which nostr exclusively uses for cryptography, was in development for 1.5+ years. Smart contracts and protocols that are responsible for tens of millions of dollars have thorough security practices and months of testing. We've started the discussion in february. The whole thing could have been iterated, released and audited in maybe 4 weeks. If you really want the proper standard out, everyone should get their act together and:
Easily doable by the end of september! This is not about perfection: it is user security, their private chats, and I see a lack of motivation for building secure DM system. I just see the motivation for "getting something out". Properly audited scheme does not need to be "iterated on". It's likely to last for 5-15 years. |
I don't think this is fair. In February no one was working full time on this. A few weeks ago I opened those various PRs (now that I have funding) because I was committing myself to pushing new DMs through. Unfortunately, I have zero experience in this area, so the best I can do is ask for action items and nag people to do them. That being the case, would you be willing to take the lead on this? Let me know if there's anything I can help with. And back to the topic at hand — merging the current draft. I honestly don't see a downside. When a new, harder form of encryption is available, all it will take is a large client (for example Amethyst, Snort, or Coracle) beginning to sign messages with the new version to force others to support it. Unless NIP 44 v1 is not better than NIP 04 at all? |
As you've correctly mentioned, we can't force anyone to add or not add something to their clients. Some year-2025 clients would implement v1, some would do v2, there will need to be interoperability between them. That's a huge mess! Bloat is bad. Bloat in immutable systems is terrible.
There was no independent cryptanalysis of the previous revision, so we can't know for sure if it's okay at all. NIP 04 is terrible and should be immediately market as "not secure".
Yes, but not the funding part. I don't want to touch the money just to move them to auditors. I can connect pieces, but can someone else please do opensats (etc). |
I just don't see this happening without an alternative. You're asking all clients to just drop support for DMs. Not saying we shouldn't, that's just a very big ask. You're right about bloat, that is a real downside, but it makes it possible to drop support for NIP 04. But I guess if we don't know 44 is any good, then it's not an improvement. Anyhow, I'll talk to the OpenSats committee. I don't want to administer the funds either, but maybe we can get someone there to do it. |
Not really, i'm just asking to mark the standard "not secure". I don't think any client would drop existing DMs just after that. |
|
We could just add risk disclaimers and a notice that a new standard is cooking. We will need to deprecate the standard at some point and why not now. |
|
Ok, I don't think that solves the problem, but we might as well. Just for completeness, is there no off-the-shelf standard we could use? It seems like a fairly simple problem that should have been solved at this point. Is nostr really in unexplored territory such that we need to invent a new cryptographic primitive? |
Yes, let's add these disclaimers and notices on v1.
Can can definitely deprecate it once we have a better solution everyone agrees with. We can't deprecate something without an alternative ready to go. |
No. It's popular to use Signal's axolotl ratchet, but it still needs to be integrated into nostr system. That itself takes time and is not too simple.
The closest we get to an open standard is MLS from RFC9420, for group chats. But it's also very complicated. |
|
Excuse me for asking, is it appropriate to release nip44 v1 on the client side now? I've implemented nip44 v1 and sealed dm, but after seeing your discussions, I'm a bit torn. |
|
It's great that it has been merged. But as we move forward with this, somebody should just rewrite this NIP to remove all of the cybersecurity jargon, the past experiment information, justification of choices, etc and make it extremely simple for normies to implement. The two paragraphs below for instance don't have a place in this NIP. This NIP is just a basic encryption scheme. It shouldn't define relay behaviors and or client behaviors. Those two should be written in the NIPs that use this encryption.
|
|
@vitorpamplona on its own, the scheme may seem shitty to an outsider, a security-conscious person, who's looking to implement the NIP in their product. There have been questions, asked by auditors. These questions need to be answered. If you don't like it in the beginning, maybe moving it to the end is ok. If we leave these out, people would think the scheme is bad, and they need to create and implement their own. A person would look at this and think "why the f did they use hmac instead of perfectly normal standard chapoly?". I have ensured:
and believe "normies" can implement the scheme easily. |
|
Also it's obvious that people would use NIP-44 to implement homebrew encryption schemes for their files, servers, and all kinds of non-nostr cases, because it's "audited". That's why it's important to mention the scheme can't be used without a signature scheme (NIP-01). |
|
I agree the spec could be cleaned up and organized better as a follow-up, but having read the audit report a lot of these extra caveats are important to include. |
Sorry, I didn't intend to be disrespectful to the work done. Yes, those questions must be answered. I would just argue that the NIP itself should NOT be the place to have that level of discussion. We have it now because we are all going through it for the first time, but new devs just want to know how to code to get on with their lives. They are never going to ask "why hmac instead chapoly?". They just want to follow what everyone is doing, trust this is the best we can get, and integrate as fast as possible.
No, sorry. The "normies" we want to target can't even read the text as is, much less understand what it means. Frankly, I don't even understand the meaning of half of the decisions here. We are not dumb, we just have 1000s of other things to do in our day to be bothered by unnecessary complexity. :) We got buttons to move 2px left. That's what we do. And that's the people you want reading and coding your stuff :)
These are great and should stay. We just to simplify the text to get them there. |
Isn't this the point of libraries? New devs shouldn't have to look at this NIP at all. Imo, this NIP is only for people creating NIP44 libraries in all kind of languages for other people to use. App developers should just use the libraries and look at the documentation of these libraries. I think we definitely don't want app devs to implement crypto stuff. That should be done by people with experience in that area.
I hear you. But you don't need to understand what's going on here. You should just use a library and look at the documentation of that library. Libraries should be easy to use. This spec doesn't have to be easy to understand (dumbed down) for everyone. At least that's my opinion. |
They will understand the meaning of part relevant to them, I think it’s simple enough. part which lists cryptography justification is not for them. You’re saying it should not be in NIP, but where should it be then? In github discussions? I don’t like reliance on github: think “banned” countries. In nostr? Nostr isn't permanent storage AFAIR, and provides no guarantees as for the data availability. |
I don't know, but It could be a habla.news post. NIP04 doesn't have any discussion in it. It all happened in GH issues and other forums. I think there is space for an amazing Blog post that explains most of the decisions made here in detail, in ways that people who are deep into the weeds can try to poke holes in the logic.
Sure, but one of the goals of Nostr is decentralization. Even NIP44 libraries should be very easy to code and verify (with @paulmillr's vectors). NIP-44 doesn't develop any new crypto prmitive. It just merges existing concepts. That merge work should be easy to do. The bigger the amount of implementers, the stronger the understanding of the crypto protections and weaknesses made here. I definitely do not want just 3-5 libs that everyone uses and no one questions anymore. |
|
Habla.news is centralized and alive today, and could be dead tomorrow. Nostr specs will likely be copied over and over. It's a git repo and a bunch of markdown files, easily hosted anywhere. As an example of the discussion behavior, see BIP340.
Of course. And we would save 40 hours for each crypto implementer out there with their research effort. |
What do you mean? Habla is just another Nostr client. It saves your post in your Nostr relays. You can read the post on Amethyst and many clients right now. |
|
Understood. But there have been efforts on relay clean-ups, haven't there? Like, deleting old events, of something. Where are guarantees the post will "live" on nostr? |
|
Nothing "lives" on Nostr, just like nothing lives in the "cloud". Things live in your personal devices and are temporarily stored in relays/servers until those are taken out. If you have your blog post in your device, then it's just a matter of rebroadcasting the event when the relay you were using goes bust. That being said, I don't think many relays are deleting old posts at all. I can still find all my posts since 2 years ago. Some relays exist as long-term archives (like a.nos.lol) but I assume that if you are not paying to help them secure the storage, that will probably not stay there for more than a decade. |
NIP-44 doesn't replace NIP-04. It defines an encryption scheme, but outside of any contextualization. NIP-04 has a contextual usage: a kind-4 event and information as to what the tags should be. So if we stop using NIP-04, we currently have nothing to replace it with. In particular there is no new kind number. Kind-44 is channel mute user. We can't use kind-4 without existing clients failing. |
Huh? People in this thread allegedly used kind: 44 for DMs. |
|
My 2 sats. @paulmillr - excellent work! It's fine as it is, the code and the NIP. I am implementing it now in my project. There will be no NIP4 messages in my app, only NIP44! - Coming before the new year. The language in the NIP should stay there as it is. The community should not attempt to "dumb things down". Give readers the benefit of the doubt and don't try to hand hold them. |
|
NIP-17 using giftwraps of rumors will replace NIP-04: #686 |
Yes, let's not do that. Though I understand the confusion. NIP-04 is both a DM spec and an encryption spec other NIPs use. NIP-44 is just the encryption spec and not a DM spec. |
|
Please add these clarifications into the spec, because people will - very likely - use |
|
New PR: #939 |
|
@paulmillr what's the status of the implementation at https://github.com/paulmillr/nip44/tree/post-audit? neither branch is really synced with main. Which branch should we be looking at? I saw you made a commit yesterday so I assume it's still a WIP. |
|
@staab it's all in |
|
FYI I opened a PR to update NIP07 with nip44 calls: #940 |
|
@paulmillr is there an npm package yet? |
|
Oh, wait, sorry, ignore me, I'm adding it to nostr-tools |
|
@jb55 you aren't happy with this merged, are you? |
|
huh? why would I not be happy? I am mostly indifferent. |
|
You’ve mentioned “We now have more breakage between clients though. yay?” - seemed like a complaint. 🤷♂️ As for your question “how does it reduce metadata leakage”, the answer is simple: padding helps a lot, esp with small messages. before it was “yes” => 3b ciphertext, which is really bad. Can’t reply on nostr, because my client doesn’t see your post there |
|
it's not really a complaint, more just "meh". DM privacy is more important to fix imo, just replacing nip04 with this isn't that interesting and just causes unncessary implementation incompatibility and pain. Will be happy to see this when combined with a real privacy solution, whether that is giftwrap or some scheme that sends notes in a different way. |
"isn't that interesting" - "unnecessary implementation incompatibility and pain" That also sounds like complaints to me, tbh. Complaining that NIP-44 wasn't necessary, to be specific. I don't think NIP-44 was unnecessary as explained by @paulmillr. Before, the ciphertext was able to leak all information about the plaintext given the correct circumstance (which a adversary serious enough could engineer). And nostr wants to protect the security and privacy of individuals who are targeted the most and thus need it the most, right? Now, with proper cryptographic padding, at least this is no longer the case. I don't think that's "unnecessary" or "not interesting". I think or hope that NIP-44 is the necessary foundation for a lot of other serious cryptography in nostr.
I am also looking forward to this. |
https://github.com/paulmillr/nip44
Goals
Why NIP-04 was bad
Footnotes
Kozaki, Kutsuma, Matsuo; Remarks on Cheon’s Algorithms ↩
https://derekwill.com/2021/01/01/aes-cbc-mode-chosen-plaintext-attack/ ↩
"AES-256 can not serve as a replacement for an ideal cipher", Biryukov-Khovratovich; Related-key Cryptanalysis of the Full AES-192 and AES-256 ↩