NIP-04 considered harmful

[adiabat]

Hi - just nostr seems promising, and end to end encrypted communication is a very important part of it, but the NIP-04 spec as written should not be implemented.

There is another issue here: #72 pointing out the non-uniform AES key. That issue has been closed but probably shouldn't be -- the spec still has the unhashed / truncated result of the DH as the AES key.

Another, I think more severe problem is that protocol as described uses aes-256-cbc with no message authentication. This means that messages can be undetectably altered in transit. Anyone relaying the message can change the message, and the receiver can't tell that it was changed. I would remove NIP-04 before people start trying to use it.

There are other encryption protocols that are used with the secp256k1 keys used in nostr that could be adapted for direct messages, such as BIP324 (bitcoin/bips#1378) or lightning's bolt 8 (https://github.com/lightning/bolts/blob/master/08-transport.md). There are also ratcheting protocols which have forward secrecy like the one used by signal, but those have other trade-offs.

The BIP324 and bolt 8 protocols aren't for discrete messages; they are for encrypted & authenticated communication channels. That means it doesn't directly replace NIP-04; the bolt 4 (https://github.com/lightning/bolts/blob/master/04-onion-routing.md) onion messages would be a closer fit. But encrypted transport between nodes is also important.

The code from BIP324 and LN is available in different languages and open licenses so I think that's the best bet for getting some code that's been used and tested.

[jonasschnelli]

I have to agree with @adiabat here. NIP04 is potentially harmful.

However, @fiatjaf's comment on the non-uniform AES key issue seems to be a reasonable approach. There is just the risk that people start to use NIP04 with sensitive information without knowing the downsides.

By the way NIP-04 was never supposed to be the thing in the first place, it was made as a prototype to show it was possible and expecting someone to supersede it with a better protocol (I personally think we must still find some other more p2p more private way to send direct messages between Nostr contacts that doesn't involve broadcasting to relays).

IMO NIP04 should at least state its infancy and thus risks of metadata exposure (who sent to who, when and how many bytes, etc.), missing message authentication (alteration by relays) and potential vulnerable crypto (no key derivation after DH). I would even go a step further and put a paragraph into NIP04 that clients implementing is, MUST warn users of the its infancy and risks.

NIP04 is probably not production ready which can lead to harm,... especially with faster adoption of nostr in general.

Of course fixing the crypto part (use AES-GCM, run HKDF after DH, etc.) is probably the trivial part (new NIP). But what needs more thinking is how much users have to trust a nostr relay and if so, could clients be force to send certain EVENTS to only relays controlled by a trusted operator (receiver himself), etc.

A more p2p-ish approach where clients connect to each other directly sound interesting at first, but seems not to be practical because not having to be "online" 24/7 is a major feature of the nostr protocol.
Relays could proxy connections to avoid having to build true incoming connections to clients (with the bandwidth downside for relays).

At the end, I guess its very important to give users a clear sense of what level of security the get with DMs on nostr.
... we still use emails without PGP for a lot of important communication.

[gkbrk]

@adiabat wrote

This means that messages can be undetectably altered in transit. Anyone relaying the message can change the message, and the receiver can't tell that it was changed.

@jonasschnelli wrote

NIP04 should at least state its infancy and [...] missing message authentication (alteration by relays)

I want to double-check something about those warnings. Nostr events are all signed. Is it really possible to alter encrypted message content without making signature checks fail?

I know NIP-04 messages are signed by the sender's pubkey, and not the shared secret between the sender and receiver, but I don't think this would allow any relays or third parties to alter message content while keeping the signature valid.

[ismyhc]

I guess I could be missing something, but the content is AES encrypted with shared secret of sender/receiver. So there are only 2 keys that can read the content. Sender, receiver. No?

The Event is then signed by the sender. As long as client is validating signature of received Event, I don’t see how the content can be altered?

What am I missing?

[ismyhc]

@adiabat wrote

This means that messages can be undetectably altered in transit. Anyone relaying the message can change the message, and the receiver can't tell that it was changed.

@jonasschnelli wrote

NIP04 should at least state its infancy and [...] missing message authentication (alteration by relays)

I want to double-check something about those warnings. Nostr events are all signed. Is it really possible to alter encrypted message content without making signature checks fail?

I know NIP-04 messages are signed by the sender's pubkey, and not the shared secret between the sender and receiver, but I don't think this would allow any relays or third parties to alter message content while keeping the signature valid.

the content IS encrypted with shared secret.

[gkbrk]

Did I say anywhere that it's not encrypted by the shared secret?

[ismyhc]

Did I say anywhere that it's not encrypted by the shared secret?

Yeah in the bottom part of your comment. 😅

[eskema]

he's talking about the signature, not the content

[ismyhc]

he's talking about the signature, not the content

Ahhh... I misunderstood. Sorry @gkbrk

[ismyhc]

Anyways, I think maybe we are on the same page, but as I see it, the Event can be verified on the client that it's not been altered and as far as I see the encrypted content can only be decrypted by the sender/receiver as its encrypted with the shared secret from those keys.

Just trying to understand what I am missing.

[gkbrk]

Anyways, I think maybe we are on the same page, but as I see it, the Event can be verified on the client that it's not been altered and as far as I see the encrypted content can only be decrypted by the sender/receiver as its encrypted with the shared secret from those keys.

Just trying to understand what I am missing.

Yes, I agree with those. I don't see any way to alter the message content anywhere without being able to sign arbitrary Nostr events without having a private key. If the way we do schnorr signatures is broken, we have bigger problems anyway.

[jb55]

even if nip-04 isn't broken it still sucks, either party leaking their private key and having their entire convo history public is concerning. we should be brainstorming ratchet-like specs so that if a root key is leaked you're not screwed. would be happy to sponsor anyone who wants to take on that task.

[ismyhc]

even if nip-04 isn't broken it still sucks, either party leaking their private key and having their entire convo history public is concerning. we should be brainstorming ratchet-like specs so that if a root key is leaked you're not screwed. would be happy to sponsor anyone who wants to take on that task.

Yup, I don't disagree here, but the original premise of the issue here seems to be misguided unless Im missing something.

Im totally fine with using another approach, but just wanted to point out that the current NIP can be used if implemented correctly and as long as each key holder doesn't leak their key 😅

[adiabat]

Events which encapsulate the NIP-04 messages being signed helps, and prevents the straightforward attacks, but ... why tempt fate? aes128-gcm is available pretty much wherever aes-256-cbc is, it's faster, and prevents these vulnerabilities if, for example, someone relays an encrypted message for someone else.

This may be out of scope, but for sending encrypted messages, it also feels sub-optimal that the protocol generates publicly verifiable signatures on top of all the encrypted messages it sends. Why does Alice need to prove to everyone that she's sent encrypted messages to Bob?

While hiding metadata is hard, and the perfect is the enemy of the good etc etc, there are straightforward, already implemented ways to avoid having a publicly verifiable record of all encrypted messages (including source and destination keys). If the system relies on the top level clear signing as the only way to maintain authenticity, and encryption and authentication aren't linked, it will be hard to improve metadata privacy.

[fiatjaf]

It is probably better to make a parallel protocol that reuses Nostr's key aspects but optimized for direct communication. It could be basically the same relay infrastructure, but the event format would be an encrypted binary blob that only the receiver would be able to read, relays would be programmed to not leak anything except to the intended receiver, senders would not be identifiable at all and receivers would only be identifiable by a temporary decoy id.

[jonasschnelli]

The Event is then signed by the sender. As long as client is validating signature of received Event, I don’t see how the content can be altered?

I guess this is a fair observation. While the NIP04 encryption itself is missing a MAC, the envelope signature done by the same key might provide a similar result (but again, this is cooking your own crypto scheme and should be reviewed carefully).

[arik-so]

This is a bit more of a high-level question, but is message authentication necessarily a desideratum? A lot of e2ee messengers don't sign the messages for the explicit purpose of plausible deniability. It might be worth considering the possibility that a potentially alterable message might actually be preferable to the alternative, should the decryption key fall into the wrong hands.

[ajtowns]

I think there's two components here -- one is if a stranger wants to start a direct message with you how do you notice that without being swamped by spam, and the other is maintaining a private conversation with someone you know over a public broadcast medium (which is not really any different to broadcasting over radio or similar anyway)...

For unsolicited messages from strangers, you could do a bunch of things: rate limit by requiring proof of work or an advance payment, have a foaf network and require an introduction first and just not allow messages from anyone who is too many degrees of separation away.

I think once you've done that, you can just focus on the second part -- I have a pubkey A, you have a pubkey B, we both know those things, and both of us want to communicate via a (semi-trusted?) relay R without anyone else knowing what we're saying, how much / how often we're saying things, possibly that we're communicating at all, or being able to confirm after the fact when A or B's private keys have been obtained, that any particular thing was communicated?

That is, I think you want (at least) two event kinds -- an introduction event that you can listen for to see if strangers want to talk to you, and the actual direct messages that can focus more on privacy, and could have single-use pubkeys and lossy filters to make it hard for even the relay you're using to do traffic analysis.

NIP 4 makes it easy to see who's talking to each other, how often/much they're talking, and lets anyone who compromises the keys later see past history. Might be worth updating the NIP to make it clear that it's a prototype and is known to have those flaws?

[lucash-dev]

I think once you've done that, you can just focus on the second part -- I have a pubkey A, you have a pubkey B, we both know those things, and both of us want to communicate via a (semi-trusted?) relay R without anyone else knowing what we're saying, how much / how often we're saying things, possibly that we're communicating at all, or being able to confirm after the fact when A or B's private keys have been obtained, that any particular thing was communicated?

Some two years ago I was working on a simple protocol (not completely sure it's original, you guys probably might tell) that looks useful as the second part -- it was based on a public database that stores values indexed by public keys. I had no idea anyone was going to implement such a database (Nostr relays seem to be an implementation of such databases).

Basic idea is:

1. Alice and Bob have a shared secret S1 (by whatever shared secret generation scheme).
2. Alice and Bob both derive the same public/private key pair P1 from S1.
3. Alice generates a second secret S2 (or both Alice and Bob might have generated multiple secrets from the start).
4. Alice posts publicly a message M1 (Nostr note in this case) encrypted symmetrically (using S1 as key), and use P1 as the public key of the "user". The message includes info for Bob to generate S2, and a payload.
5. Bob queries relay for messages that matches the public key P1.
6. Bob decrypts M1, and obtains the payload and S2.
7. Bob restarts the process from (1), using the new secret S2 as the shared secret.

As long as one of Alice or Bob interacts with the relay in a perfectly anonymous way (eg using Tor and with no authentication), the two ends of the communication can't be matched. With fixed rate decoy messages and queries, it becomes impossible to know when/how much each end is communicating.

Messages in each sequence can't be linked to one another unless the secrets are leaked.

If any of the secrets is leaked, you still can't say who wrote the messages, much less prove it. The secrets aren't linked to identities.

It's impossible for the relay to censor any specific exchange without stopping all communications of that type within the relay.

There are many other possible ways to implement it, but the fact that Nostr lets you query relays by public key makes a shared sequence of throw-away key pairs the obvious choice.

Please note this would still be secure and censorship-resistant (in the sense of not being possible to censor specific exchanges) even with just one relay available. The relay has also plausible deniability as they have no idea what's being sent (basically like a router in an ISP). For further reduction of legal liability, messages could be dropped after a fixed TTL.

Could also use "remailers" within this protocol to further enhance privacy. The remailer could either use this new scheme or just plain old NIP4, receiving an event encrypted, then posting it. Chaining multiple remailers means network analysis can't work as long as at least one remailer doesn't collude with the relay. This can remove the dependency on Tor (but needs new infra-structure).

I think that's actually a better use-case for Nostr relays than public messages (which isn't censorship resistant in any strong sense, likely in any practical sense at all).

[xz-cn]

Events which encapsulate the NIP-04 messages being signed helps, and prevents the straightforward attacks, but ... why tempt fate? aes128-gcm is available pretty much wherever aes-256-cbc is, it's faster, and prevents these vulnerabilities if, for example, someone relays an encrypted message for someone else.

Using a library such as aes-gcm is indeed a better choice than the current aes-cbd scheme as the MAC part has been taken care of.

If the key is based on Curve25519, you can use the mature TweetNaCl library's authenticated encryption for end-to-end encrypted communication.

[paulmillr]

noble-secp author here. just want to provide some information:

1. Topicstarter is right: nip04 is harmful
2. aes-cbc also needs padding, and bad padding can be pretty bad.
3. aes-gcm-128 is not a future-proof option. Use at least aes-gcm-256 or even better: xchacha20-poly1305.
4. curve25519 (@xz-cn) is not cool because: 1) it needs a separate elliptic curve, which can be pretty big if written from scratch 2)

Ideally: secp256k1 (perhaps x-only ECDH) + HKDF + XChaCha20-Poly1305; will still need an audit and comparison to protocols like the one Signal is using. It's important to not implement just anything from scratch without consensus first.

At least, if you deploy something like this, make it versionable. If protocol gets broken, you'll at least be able to upgrade it.

[mikedilger]

I think that crypto is about right. I pointed out the lack of an HKDF a while back.

As for protocol versioning, if the crypto is ever broken we can just create a new event kind with a new form of DM. In fact, that is what we have to do if we were to make changes, because NIP-04 is implemented in many places we can't go changing it now. So a new DM spec would be a new event kind.

BUT, I'm not in favor of DMs over nostr. I'm in favor of advertising an endpoint in your metadata, and using out-of-band messaging.

[xz-cn]

I think that crypto is about right. I pointed out the lack of an HKDF a while back.

As for protocol versioning, if the crypto is ever broken we can just create a new event kind with a new form of DM. In fact, that is what we have to do if we were to make changes, because NIP-04 is implemented in many places we can't go changing it now. So a new DM spec would be a new event kind.

BUT, I'm not in favor of DMs over nostr. I'm in favor of advertising an endpoint in your metadata, and using out-of-band messaging.

In that case, you will need external support for the DM function... I guess that is not the idea of the Nostr protocol.

What we can do is indeed to add a new type of event. Which not only makes the messaging part safer, but also includes the protection of the DM metadata. In that case, other people won't know to whom you have DMed and when. I understand there is a repo working on this already? https://github.com/SebastiaanWouters/emon

[mikedilger]

I think @jb55 #107 (comment)
and @fiatjaf #107 (comment)
made very good points, and @paulmillr #107 (comment) on the crypto. I offer my concurrence with them and as I won't be leading this effort, I have nothing else to offer so I'll bow out of this issue. I look forward to one day a new way to do DMs more securely.

[earonesty]

1. with the signature, the need for a mac isn't really present - message is already tamper proof, gcm is fine tho.
2. versioning is key
3. the dm-spec, as is, leaks no more public keys than are already leaked in the existing protocol, fixing that is a separate issue
4. doing dm's "more securely" is challenging. leaking the main key, even with discarded channel keys, results in a full loss of metadata privacy. and most clients would keep channel keys for a long enough time that the additional security isn't notable. only decent way is to establish a more "direct" connection with a DM, then at least you have pure deniability. which, of course, you can use nip-4 style shared secret to establish.
5. i think "considering it harmful" isn't really considering what its intended use case is: "persistent censorship resistant e2e encrypted messages". if you don't like the persistence (which can always leads to metadata leakage), don't use nostr
6. any "dm" system should probably be layered on nip-4 shared secret as the initial channel creator anyway. maybe just change nip-4 wording to say "don't use this for dm's... use it at a lower layer in a secure dm system"

[braydonf]

@ajtowns I was thinking something similar in regards to two events kinds, a request and then messages.

@lucash-dev I had a similar thought, looking for messages at an agreed upon shared secret "location".

Event Kind A:

A request for Direct Message. To prevent spam there is either a NIP-57 private payment, proof-of-work, or is a known existing pubkey that is followed. For example a Diffie-Hellman key exchange can be made for all pubkeys that are followed to be "locations" to watch for incoming messages.

Event Kind B:

After confirming the Event Kind A, messages are sent using Event Kind B. The identities of the messages are not known and encrypted. How each party knows where to find the messages is based on information exchanged in Event Kind A. The messages do not need to stay on relays until after they have been received, or in the case that both are online they can be sent directly peer-to-peer. Read receipts can be optional for confirming the delivery.

[lucash-dev]

@ajtowns I was thinking something similar in regards to two events kinds, a request and then messages.

@lucash-dev I had a similar thought, looking for messages at an agreed upon shared secret "location".

Event Kind A:

A request for Direct Message. To prevent spam there is either a NIP-57 private payment, proof-of-work, or is a known existing pubkey that is followed. For example a Diffie-Hellman key exchange can be made for all pubkeys that are followed to be "locations" to watch for incoming messages.

Event Kind B:

After confirming the Event Kind A, messages are sent using Event Kind B. The identities of the messages are not known and encrypted. How each party knows where to find the messages is based on information exchanged in Event Kind A. The messages do not need to stay on relays until after they have been received, or in the case that both are online they can be sent directly peer-to-peer. Read receipts can be optional for confirming the delivery.

Precisely. I think much of the details could be just copied from existing protocols, eg based on Double Ratchet like Signal. The only thing that needs to be "invented" is the generation of the "location", which might be as trivial as using DH, hashing the result, and using that as a private key for the "author" of the new events, then with each new secret generated just using its hash as a private key.

However, I'm also intrigued by the possibility of using "private zaps" (receipts of LN payments with encrypted data) for encrypted direct messages. By the pace of development it might be ready well before a good replacement for NIP-4 is figured out.

Maybe both might be combined, with a "private zap" used as Kind A to jump start the protocol.

[braydonf]

Yeah, Private Zaps for Kind A to jump start sounds useful.

The location could be a DH location, seems simple. Could be useful to implement to see how it might work in practice.

Those locations could also be shared among many recepients that might not be involved. AES-GSM could be used to determine which messages can be decrypted and intended for the recepient (this would have some additional CPU and bandwidth cost for stored messages at-least). Peer-to-peer would then optimize bandwidth and CPU usage. Specific relays should likely also be part of the defined location from Kind A.

Want to look more into Double Ratchet and how Signal is working with groups.

[lucash-dev]

Simplest incrementally better thing would be to just take pretty much exactly NIP-4, but derive properly
three keys from the DH (using prefixed hashes):

• encryption key
• "fake" author key
• "fake" recipient key

then send it as a nip-4 event. it would look like just a regular nip4 thing and relays don't need to be aware of anything different.

(might use each of the "fake" keys for one of the users, just to make it easier for clients as it is closer to nip4, though
it leaks a bit more info).

clients need only know what public keys to use for DH (maybe just everybody you follow) and request them from relays.

disadvantages:

• still leaks when a specific "location" (pair of "fake" author/recipient keys) is used
• users have to always use same location for talking to same people
• no forward secrecy
• relays can still figure out which users are talking to each other by connections and requests for the location (not sure how much that could be avoided anyway).
• sort of a hack

much easier to implement than a proper private messaging protocol though. however, like I said, by the time anyone has time to do it, a usable alternative might already be available.

[braydonf]

still leaks when a specific "location" (pair of "fake" author/recipient keys) is used

It's association would only be known to the two parties though (mostly).

users have to always use same location for talking to same people

The "location" could be random after the initial DH exchange (e.g Event Kind A).

relays can still figure out which users are talking to each other by connections and requests for the location

This is where being able to use short location queries could be useful. Let's take for example, if the full location was 3bf0c63fcb93463407af97a5e5ee64fa883d107ef9e558472c4eb9aaaefa459d. It could be possible to query it by using only the leading bytes, for example 3b. If there were too many results, 3bf0 could be used and so forth.

[earonesty]

ok, i checked this. crypto subtle has a serious bug. so yes, iv's of 16 don't work. this doesn't matter much if we use the 16 bytes iv in the salt. we're back to uuid-level

openssl is smart about long iv's - adding the bytes after 12 to the box (see EVP_MAX_IV_LENGTH, and set params, you can specify a length of 16 and it does the right thing)

crypto subtle is not - it TRUNCATES the iv

we can fix this by always just using the iv as an input to the salt of the hkdf, and we should probably force-truncate to 12 bytes to fix nostr-tools interop with openssl

good thing we're switching to an hkdf! honestly, this is so broken rn. we should switch to the new algo asap, and not sit on thumbs.

[earonesty]

ok, new code for replacement for nip04 (bidirectional persisted encrypted content)

async nip04XEncrypt(privkey: string, pubkey: string, content: string, version: number) : Promise<string> {
    const key = secp256k1.getSharedSecret(privkey, '02' + pubkey)
    const normalizedKey = key.slice(1,33)
    const iv = randomBytes(16)
    const derivedKey = hkdf(sha256, normalizedKey, iv, undefined, 32);
    
    let plaintext = utf8Encoder.encode(content)
    let cryptoKey = await crypto.subtle.importKey(
      'raw',
      derivedKey,
      {name: 'AES-GCM'},
      false,
      ['encrypt']
    )

    let ciphertext = await crypto.subtle.encrypt(
      {name: 'AES-GCM', iv},
      cryptoKey,
      plaintext
    )

    let ctb64 = base64.encode(new Uint8Array(ciphertext))
    let ivb64 = base64.encode(new Uint8Array(iv.buffer))
   
    return ctb64 + "??" + ivb64 + "??" + version.toString()
  }
 
async nip04XDecrypt(privkey: string, pubkey: string, data: string): Promise<string> {
    let [ctb64, ivb64, version] = data.split('??')
    if (version != "1")
      throw Error("unknown version")

    let iv = base64.decode(ivb64)
    let ciphertext = base64.decode(ctb64)
    const key = secp256k1.getSharedSecret(privkey, '02' + pubkey)
    const normalizedKey = key.slice(1,33)

    const derivedKey = hkdf(sha256, normalizedKey, iv, undefined, 32);

    let cryptoKey = await crypto.subtle.importKey(
      'raw',
      derivedKey,
      {name: 'AES-CBC'},
      false,
      ['decrypt']
    )

    let plaintext = await crypto.subtle.decrypt(
      {name: 'AES-CBC', iv},
      cryptoKey,
      ciphertext
    )

    let text = utf8Decoder.decode(plaintext)

    return text
  }  ```
  
  this is the base .  it fixes all the problems and in the way we discussed.  the "1" is the version.  the code above is "directed" and it can use this as a primitive (remove version from tag, re-use this code)
  
  base layer/primitive arch is important for security too

[earonesty]

now look how nice the directed-blinded message is:

 async nipXXEncrypt(pubkey: string, inner: UnsignedEvent, version: number): Promise<NostrEvent> {
    const event = await this.signEvent(inner)
    const content = JSON.stringify(event)
    const epriv = generatePrivateKey()
    const epub = getPublicKey(epriv)
    const encrypted = await this.nip04XEncrypt(epriv, pubkey, content, version)
    const unsigned = {
      kind: 99,
      content: encrypted,
      pubkey: this.pubKey,
      tags: [["p", pubkey], ["v", JSON.stringify(version)]]
    }
    const signed = await this.signEventWith(unsigned, epriv, epub)
    return signed
  }

[paulmillr]

it fixes all the problems and in the way we discussed

No. It doesn't bring AES-GCM.

[earonesty]

it never re-uses the key, so we have no problem. derived key is new each time. and then the 12byte iv is fine

[paulmillr]

The whole point was to ditch CBC mode because it's shitty.

[earonesty]

was that the "whole point?" i thought the point was to make a safe low-level primitive that was easy to implement and worked in settings like browser and mobile. (we're using gcm not cbc, technically). do you have a recommendation that is better? we know cbc is shitty, we're not using it. gcm with a salted hkdf is fine. check the code above. i think we're clean now

[paulmillr]

That was the point. CBC is a can of worms.

The recommendation is, as i've mentioned several times, AES-256-GCM, or ChaCha20 / XChaCha20 Poly1305, but for now no need to complicate and we can just keep GCM.

Your code above is NOT using gcm - it's using CBC. And it's not verifying GCM mac.

[earonesty]

agreed! it was a typo. i knew we were ditching cbc. cool!. thanks for that (edited the post), crypto subtle will fail if the mac fails. so i think we're good there. plus we're already checking the sig. which means auth is not really important. how is the code now? i replaced ?iv= with "??" so we can have 3 (or more) fields in payload. 3rd field is version. prob we should flip it (version first)?

i think this should be 1 NIP with 2 kinds, "shared encrypted content", and "directed encrypted content", with 1 "internal kind" supported ("direct mesaage") that can be used

ie: kind 99 - > kind->501 -> dm
ie: kind 98 - > old-style shared dm

the advantage of 98 is that content is persisted for both parties
the advantage of 99 is that the sender doesn't reveal their public key

there are good reasons you may want both, i see no reason to discard one over the other. we should recommend clients using nip04 now should switch, immediately to the new kind 98. this requires little change to the underlying code and structure, and supports the same features and is strictly an improvement.

we can use 99 as the base-layer for encrypted group-chat and "derived-epub-key direct messages". the hard part with derived epub is that there's no "author tag" to scan for. so you're left looking at all messages sent to someone and checking which is yours. which can be prohibitive.

[paulmillr]

Is there any other way to pass iv? Maybe as a tag or something? Isn't that how they are used? I mean, encoding ciphertext to base64 and then appending ?iv=base64(iv) seems shaky.

[paulmillr]

from docs it seems like content can be an object.

So we should make it an object:

{
  "iv": base64(iv),
  "ciphertext": base64(ciphertext),
  "mac": base64(mac)
}

[earonesty]

In crypto subtle, the ciphertext is a combination of <actualCipherText><MAC>.
The MAC is appended as the last 16 bytes of the ciphertext and is always validated on decryption (missing mac==fail)

Having a key-value split is probably safer. we can't pass objects around as strings. so we'll have to encode them.

We can JSON.string({
{
"i": base64(iv),
"c": base64(ciphertext_with_mac),
"v": version
}
})

instead of iv??cipher_mac??ver

instead, if you think that's nicer. again, mac is already in there. no need to deviate from that web/standard way of gcm'ing. i guess we could split it out and put it back again. but that feels like a recipe for making mistakes.

[paulmillr]

<actualCipherText><MAC> is not too bad. It's not the standard, but it's a standard. That's how webcrypto works -- others don't work this way.

OK, here's another crazy idea.

How about we ensure this new corrected scheme is group-chat-friendly? I don't think more features are needed, we can just ensure this works properly in group chat contexts. Signal group chats, AFAIK, are just a collection of 1-to-1 messages between every chat member.

[earonesty]

group-chat can use the nipXXEncrypt directed code (above) to send a privkey out as an invite. so the new standard is already friendly as a base-layer for group chat where all the invitations are blinded and the group itself is blinded.

indeed, that's what i'm using it for. fixing dm's is just part 1, ok, i think we're all on the same page finally. lets write up the base-layer (shared private message) with the versioned encryption scheme and make an NIP out of it. it should be a drop-in replacement for nip04

then we can make an NIP for the "generic encrypted message" layer, and then mention, in there, that group chat and other encrypted systems should, ideally, use this instead.

[paulmillr]

then we can make an NIP for the "generic encrypted message" layer, and then mention, in there, that group chat and other encrypted systems should, ideally, use this instead.

I don't think it's possible with nostr, a public tool.

[earonesty]

wut? of course it is. you can use the blinded encrypted direct send as an invitation, and then all the members have the privkey of the group, and can chat. you can revoke by rolling a new group and sending a message indicating revocation. this is how NIP38 works

[paulmillr]

No, not really. Secure messaging is a complicated problem. Nostr will always leak metadata. There is no way to prevent metadata leakage.

[earonesty]

"leaking metadata" is by design. nostr is "encrypted storage of group information". not "encrypted messaging". forward secrecy is meaningless when you want to be able to load up a new client and see your old messages. "leaking metadata" is meaningless in that context. also, nostr leaks very little if we use a scheme like above. lets list what's leaked:

• invitations to group chats could have a particular message size, so an observer can know that maybe you were invited to one (they can't be sure... they can't see the message)
• messages sent can leak sizes, so an observer can know that maybe you are participating in one (again, deniable)

only way to know those things for sure is to be invited to the group

[paulmillr]

IPs and sender machine information are always leaked.

[earonesty]

that's a separate problem, imo. sender can use tor or similar. that data is leaked in other systems too. no guarantee that signal or telgram servers discard ip's. perfect is the enemy of the good here. we need to make solid building blocks and fix the base layer instead of continually throwing our hands up and saying it's hard. we've done good work here. lets make incremental progress, draw up an NIP and move on to the next problem. NJ will always beat MIT in the end.

[paulmillr]

Can we move towards a discussion to wrap up our concrete proposals? We're getting close, but there are a few points i'd like to clarify. It is not comfortable to discuss it in the issue with 107 comments.

#570

Can the current issue be closed-down?

[earonesty]

yes, lets!

[earonesty]

proposal from discussion. https://github.com/nostr-protocol/nips/pull/572/files

• minimal code change
• still browser compatible
• constantly rotating aes key
• versioned, so we can do this again (add xchacha20, for example or add optional whitespace padding to obscure sizes)
• uses gcm instead of cbc
• removed "scary warnings", replaced with more precise language

it's not like we "rolled our own crypto", we just forgot to hash and salt the key.

[paulmillr]

I have opened pull request for (allegedly) NIP-44 Encrypted Direct Message (Versioned): #574

It uses versioning, XChaCha20 and hashed shared secret. Algorithm choice rationale is described in the pull request. New Versioning feature explicitly requires non-compatible versions to throw user-visible errors.

AES-GCM is NOT used.

Please take a look at it! Any comments are welcome.

[paulmillr]

Opened a discussion about SimpleX integration in #658

@fiatjaf could we close this issue?

[staab]

If you guys don't mind, I'd like to hijack this issue to continue the conversation about NIP-44 and downstream use cases (wrapping, DMs, chat, groups).

I've put together a survey of all the proposals and implementations of new-style encrypted messaging here. The goal is to decouple different use cases and create a solid foundation for cryptography, wrapping, and key-rotation that can be used in different configurations as we work on nailing down the UX of various features. The gist is open to comments and corrections.

@paulmillr @earonesty @v0l @vitorpamplona

References (for auto-backlinking):

• NIP-44: Encrypted Direct Message (Versioned), replaces NIP-4 #574
• NIP-59 Gift Wrap #468
• NIP-17 (old 24) Sealed Gift-Wrapped Messages for Private DMs and Small Group Chats #686
• NIP-112: Encrypted Group Events #580
• NIP-103: Onion Routed Direct Messages #499
• https://github.com/d-krause/nostr-nips/blob/nip76-draft-2/76.md
• NIP-31 Incognito Direct Messages #410
• NIP-38 - Encrypted Group Chat using a single shared secret #59
• NIP-24: Private Messages #56
• private dm events #17
• https://github.com/water783/nips/blob/nip101/101.md
• NIP-29: Simple Groups #566
• NIP-72: Moderated communities #602
• The shortest private messages NIP (Simplex) #564

[ferantivero]

It is probably better to make a parallel protocol that reuses Nostr's key aspects but optimized for direct communication. It could be basically the same relay infrastructure, but the event format would be an encrypted binary blob that only the receiver would be able to read, relays would be programmed to not leak anything except to the intended receiver, senders would not be identifiable at all and receivers would only be identifiable by a temporary decoy id.

@fiatjaf if I am not mistaken you recently exposed once again that this should be a different protocol rather than just an innovative NIP (i.e. NIP-44 v<next>), so any desire to share: Why?

[jb55]

On Tue, Jan 16, 2024 at 01:41:08PM -0800, Fernando Antivero wrote: > It is probably better to make a parallel protocol that reuses Nostr's > key aspects but optimized for direct communication. It could be > basically the same relay infrastructure, but the event format would > be an encrypted binary blob that only the receiver would be able to > read, relays would be programmed to not leak anything except to the > intended receiver, senders would not be identifiable at all and > receivers would only be identifiable by a temporary decoy id. @fiatjaf if I am not mistaken you recently exposed once again that this should be a different protocol rather than just a innovative NIP (i.e. NIP-44 v\<next\>), so any desire to share: Why?

fwiw I've been rallying crying for this as well. We just need a relay extension that adds specific features for private dms that don't leak metadata. We don't have the right primitives to express private DMs on vanilla-nostr unless you really hack it with giftwraps. nip-44 is still useful for the encryption stuff of course.