NIP-59 Gift Wrap #468
NIP-59 Gift Wrap #468
Conversation
|
The use of ephemeral keys was proposed before in different ways and was always the weak point. Relays will not be too happy to accept ephemeral keys, especially when the content is encrypted. So the relay you are talking to would have to learn it's you which brings us back to what fiatjaf and others favor: the relay learns the meta data but promises the sender and receiver of DMs to not let unauthorized clients request these DMs. The advantage of using ephemeral keys is that in theory, events could be mirrored to other relays without damage but that holds DOS and spam potential. |
Its not, it's used only wrap the actual DM event, but as it turns out this has been suggested before (forgot to check closed PR's/Issues). And for reasons which @Giszmo has suggested it makes sense that this would be bad for relays, although as it currently stands free relays already have to deal with random keys spamming them. |
|
This is essentially what I have been working on with nip 76 (https://github.com/d-krause/nostr-nips/blob/nip76-draft-2/76.md) It does not get any interest, but I am still pressing on with Draft 3 that addresses more concerns. I guess you could say that Nip 76 Keys are "ephmeral", but they are also deterministic, using one determined key for the author and another determined key for the encryption. We could let relay operators in on the author determined key, but still withhold the encryption determined key. Something along these lines that masks the pubkey needs to happen and it will happen one way or another despite what relay operators like and dislike. It is high time we stop storing or data in the clear, both in terms of whether the event is readable AND who originated the event. If we can't get relay operators that control this repo on board, I can see a Nostr fork coming where there will be operators that are on board. I don't understand their resistance anyway - they are just "NIPS" - not mandates. They can approve a NIP without agreeing to conform to it. |
Well, nip-42 is only now being rolled out. No client supported it a week ago. |
|
I said it many times and I will repeat it here, too. I don't like any approaches that require limiting event replication across relays. A new relay should be easy to spin up and saturated with past events and other relay operators should only charge for what it costs them to stream over all that data. There should not be information privy to one relay or another. That is why I am in favor of plausible deniability in the DM meta data using a nip like my #17 With approaches that limit DMs to one relay to not leak too much meta data make parties to the conversation dependent on that relay. |
|
@v0l This NIP is not quite the same as #499. In it I suggested using a random key pair to encrypt the inner event, but the key doesn't have to be random. Instead the event could be sent using one's own private key to an intermediate pubkey which is trusted to forward the message along to someone else. That should help hide sender and receiver as long as the timing isn't too obvious. To @Giszmo's point both messages can also be broadcast everwhere so there's no concern about events being limited to specific relays. Such a forwarding service could be a very simple bot that's always online, and could also be programmed to only forward messages once enough are in a queue, giving deniability to everyone using the service. |
|
Hello Everyone, This is Arun from the Current App team. There are several open pull requests aimed at solving the privacy issues surrounding Direct Messages on Nostr. Our team has conducted research on this matter over the past few weeks. Many of these requests involve the exchange of keys between two users to ensure security or create secure channels. They are complex in terms of preventing leaks and ensuring privacy measures are in place. We have also examined the use case from the perspective of end users. Almost all the proposed solutions address the scenario of "We are friends and need to communicate securely, so let's use Nostr DM." We believe this is not a valid use case for Nostr. The use case for DMs on Nostr is quite different. It begins with a message like, "Hey, I saw your post about attending this meetup. Do you have time to meet up in person?" The content of the event is already encrypted. The only issue here is the leakage of metadata, which could be used by anyone to create a social graph or for spamming profiles that receive DMs from trending profiles. This NIP-59 proposal, which wraps DMs as a gift into another event, works well for this use case by addressing the metadata leakage. After reviewing other solutions, we kept returning to this NIP because of its simplicity and effectiveness within Nostr. Here are the different levels of DM privacy that I see: Level 1 - Kind 4: Content is encrypted, but I have no concern about metadata leaks. We should begin by implementing level 2, which requires changes to the client, and then move on to implementing level 3. Level 4 should never be implemented within Nostr and direct them to Signal or SimpleX. Of course, this does not solve everything, and a couple of issues were raised during this discussion. Let me address them.
As of today, There are 120 free relays Vs 65 paid relays. Using any of the free relays with NIP-59 will stop the metadata leakage and make it anonymous.
This is also backward compatible and no user flow changes required. Clients query for both kind 4 and 1059 and unwraps kind 1059 into kind 4 automatically. When starting a new conversation they can choose a less(kind 4) or more secure (kind 1059) way to send DMs. Overtime we believe all Nostr clients will start to support kind 1059. Couple of weeks ago we released a version on Current App with DM and push notifications. @v0l we are ready to start supporting and releasing a test version to see how the users react to it. @Semisol - Your requested changes has been addressed. Can you review it one more time? One reason why I enjoy developing on Nostr is its simplicity. We should not compromise that for complex solutions that no one will use in the end. It's a simple, straightforward solution. Simple always wins... not SimpleX. |
|
Thanks for the analysis @starbackr-dev, good to have someone who's thinking about threat model. The pull requests looks good to me. One small concern is that |
|
Agreed. Kind-1004 may be a good one to pick. Kind 4 - legacy DM encryption with metadata leaks @v0l - your comments? |
|
Why not 444? |
|
Yes. 444 is even better..! |
|
In the meantime i've introduced NIP-44 that fixes broken cryptography of NIP-4 and introduces versioning: |
Because of NIP-16. I think kinds below 1000 are considered legacy, because it isn't clear how relays should handle them. |
Afaik they don't use NIP-42. Paid relays are just checking the pubkey of received events, so the side-effect is that one can only send events of allowed pubkeys. That's why NIP-59 won't work for now. This probably has to do with the fact that NIP-42 flow is unpredictable in the sense that it is hard for the client to know when and if the relay 'AUTH' message will arrive. So just a few clients enable this, which makes relay don't require it. Please support NIP-43 - Fast Authentication PR for a better alternative. |
|
this is exactly how we are doing encrypted group chat invitations in arclib. we're also using a "fixed" nip4, which is basically the same as nip44. i see no problem with this proposal and would love it to be a standard. relay authentication can be trivially handled in many ways: - a special "auth" tag that the relay strips on send for example. i would think that a simple json tag that the relay provider agrees to remove immediately is the best way to go for auth it's easy to understand, and if they fail to remove it, they've only doxxed one ephemeral key... now you can ditch them and move to someone that honors it |
Interesting thought. Some trust in the relay is always required anyways unless you use different proxies for every connection to cloak your up address... so you envision an auth-tag that has the "main" pubkey as well as a signature on it? |
|
I haven't been following the NIP 04 discussion, but I think general-purpose gift wrap is an amazing idea. Maybe not to fix DMs necessarily, but if we combine it with nsec bunker and NIP 51, we get role-based authentication for private groups for basically free. Either I'm nuts or this is a huge idea. Blogged about it here |
|
@staab Great idea. Can you point me to where I can read more on nsec bunker ? |
Here is link to an article and utube video |
the AUTH spec is already ephemeral, you don't need the tag. provider tosses away a fake AUTH event. so imo, no doxxing is needed. better than a tag, auth the connection in accordance with https://github.com/nostr-protocol/nips/blob/master/42.md we could come up with a new auth tag per event. that could help with stuff like real onion routing through private relays. tradeoffs: bigger & routable vs. faster & stateful |
59.md
Outdated
|
|
||
| This event kind can be used to wrap DM's or other events which are considered private. | ||
|
|
||
| This is similar in concept to Onion Routing. |
There was a problem hiding this comment.
if the sender of the event would like to decrypt it at a later date, they can use their "tweaked" private key, rather than a random key when producing the event. the "tweak value" can use any outer-envelope data, depending on the needs of the user. tweaking secp256k1 private keys is a well-known mechanism, and should be considered if a client wants to be able to read these events later from a message store, for example.
There was a problem hiding this comment.
Any change in how key material is handled needs to be added to NIP-07 or else it's not possible for me to use, this is why ephemeral keys work for me because i create the secret material.
There was a problem hiding this comment.
this would not be a change to how key material is handled. these keys are still ephemeral. its just that if you really wanted to decrypt it later, however annoying that may be. you, optionally, could.
doesn't have to be part of this NIP, but it might want to be mentioned that since the creator of the event controls the ephemeral key, there's no hard requirement that it not be a derived key or a weak key, and no guarantee that the recipient can be assured that the creator cannot decrypt it later, or that the key was made poorly
if you wanted to be able to guarantee the key is hard, you can derive the ephemeral key in such a way that the recipient can verify it
for example, you can use the inner event id as the private key
then you'd have to crack the dlp or reverse a hash to create a weak key - the recipient can be assured the ephemeral key is free of artifacts or structure
There was a problem hiding this comment.
What do you think of adding a nonce key with a random string to the wrapped event JSON (not as a tag, but as an extra field)?
So for the DM example, you could send the gift wrapped DM with 2 1059 events from different random keys and 2 nonces, one to the recipient and another one to yourself (so you can keep chat history). With 2 nonces, the 2 gift wrap contents won't match, so no one can tell it is the same message.
This way you don't need to keep track of the wrapping key for decrypting chat history.
The down side is the created_ats will be close to each other and your copy won't have a p tag with the recipient so you cant fetch just the messages sent to a specific user. =(
There was a problem hiding this comment.
Here is another way to solve this problem (not the decryption part, but the querying part).
What about optionally adding tags to the outer event, but encrypted? So ["p", pubkey] would be added to the wrapper as ["p", "encrypted pubkey"]. This would allow users in the know to still execute filters, supporting larger groups (though still not scaling as well as native nostr). A new k tag could be used to denote the kind of the inner event as well.
A 1k member group, each publishing 1k events over the course of a year would result in 1MM events, which would be far too many to process without filters.
There was a problem hiding this comment.
@staab nice so the sender should encrypt the receiver pubkey using the shared secret
But from NIP-04, is secp256k1.getSharedSecret(SENDER_privkey, '02' + RECEIVER_pubkey).slice(1, 33) the same value as secp256k1.getSharedSecret(RECEIVER_privkey, '02' + SENDER_pubkey).slice(1, 33)? So the receiver can use it to also encrypt his own pubkey and query with { #p: [encrypted_receiver_pubkey] }?
There was a problem hiding this comment.
No idea, encrypted tags would rely on signing with a shared secret. This PR uses ephemeral keys, but it would be easy to convert it to using a shared secret (since one is necessary for decryption anyway). Encrypted tags could then be encrypted symmetrically.
59.md
Outdated
| "pubkey": "63fe6318dc58583cfe16810f86dd09e18bfd76aabc24a0081ce2856f330504ed", | ||
| "content": "73ZaB2WzPlIwwC55YnhsFA==?iv=1R2WlyrAk9VQ1NdhNuWmsw==", | ||
| "kind": 4, | ||
| "created_at": 1682286690, |
There was a problem hiding this comment.
created_at can be made optional to save space, verifier assumes created_at is the same, and verifies the sig as if it is present (not as if it is excluded!).
There was a problem hiding this comment.
I don't think the additional complexity is worth the space saving.
|
We can create another name. The only point is that any event can be unsigned and put it into a warp to hide it's nostr properties from the public. |
|
its not necessary to leak any keys
you calculate the shared secret between sender and receiver
then you use that as the private key and post an encrypted message
you can calculate this for all your follows
and send a subscription for the authors
now you have two way blinded dms
…On Sun, Jul 23, 2023, 10:38 AM Vitor Pamplona ***@***.***> wrote:
I think I am getting a good grasp on this + #574
<#574> encryption scheme. But
I would bring them together in a different way. Let's take private DMs as a
use case.
This protocol obfuscates the DM metadata from relays and the general
public WHILE being recoverable in any client with the receiver or the
sender's private key.
1. A new Kind 9 for Private DMs.
This kind is always unsigned and will never be seen in the wild without a
signed wrap.
Complete Direct Message Payload:
{
"content": "This is a DM. Notice that there is no sender or receiver here, no signatures, and no ids",
"kind": 9,
"created_at": 1686840217,
"tags": [],}
2. A new kind 10: Encrypted Unsigned Event
Kind 10s sign *any* unsigned event for distribution in the protocol. The
unsigned event is placed with the XChaCha encryption scheme (NIP-44) inside
the .content. Clients should treat inner fields as if they were fields of
the kind 10 event (merge them).
Encrypted Unsigned Event:
{
"id": "<The usual hash>",
"pubkey": "<Real Author's PubKey>",
"content": "<NIP-44-Encrypted Kind 9>",
"kind": 10,
"created_at": 1686840217,
"tags": [],
"sig": "<Real Author's PubKey Signature>"}
This event can be broadcasted, but, for DMs, this event should *not* be
broadcasted automatically.
Notice that there is no receiver exposed here even though the encrypted
procedure encrypts only to the receiver's key. If this event leaks, it will
only link to the sender. One will have to have the private key of the
receiver to prove that this was a message for the receiver.
3. A Gift Wrap to hide the sender from the public.
Public Wrap (Kind 1059):
{
"id": "<The usual hash>",
"pubkey": "<Random, one-time-use PubKey>",
"content": "<NIP-44-Encrypted Kind 10>",
"kind": 1059,
"created_at": 1686840217,
"tags": [
[ "p", "<Receiver>" ]
],
"sig": "<Random, one-time-use PubKey Signature>"}
The receiver is now exposed, but the sender is hidden.
4. A Gift Wrap to recover the sender's DM history when changing clients.
Public Wrap (Kind 1059):
{
"id": "<The usual hash>",
"pubkey": "<Random, one-time-use PubKey>",
"content": "<NIP-44-Encrypted Kind 10>",
"kind": 1059,
"created_at": 1686840217,
"tags": [
[ "p", "<Sender>" ]
],
"sig": "<Random, one-time-use PubKey Signature>"}
By broadcasting this event, the private key of the sender can unwarp sent
and received messages and reassemble the DM history in any client.
The two wraps can be made separate with separate times to avoid time
collision. And if we allow GiftWraps to contain more than one event (which
I strongly recommend), then the 4th step can be made once at night with a
sum of all messages sent during the day.
The implementation is fairly straightforward:
1. Implement NIP 44 encryption, serialized in a Json style like @v0l
<https://github.com/v0l> did in Snort
2. Implement kind 10s with overriding functions to pull from the inner
event fields if present.
3. Render kind10 overridden by kind9s as DMs.
4. Implement Kind 1059, with overriding functions for the inner event.
5. Pull all gift wraps with p = logged in user from relays and process
the inner event accordingly.
Interesting benefits.
- Sender and receiver are not linked in public.
- Kind, tags, and created_at are hidden from the public.
- Wraps create a bigger anonymity set because they can be anything,
not only DMs, not only messages to me, but also from me.
- The generic unsigned event allows us to take any Nostr event
private. That's why we don't need an encrypted DM event anymore.
—
Reply to this email directly, view it on GitHub
<#468 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAMMUOHVD63B5RU4XIVH6TXRUZOVANCNFSM6AAAAAAXIZN43Y>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
blind dms are implemented here
https://github.com/ArcadeLabsInc/arclib
also encrypted group chat that uses gift wrap
…On Thu, Jul 27, 2023, 11:30 PM Erik Aronesty ***@***.***> wrote:
its not necessary to leak any keys
you calculate the shared secret between sender and receiver
then you use that as the private key and post an encrypted message
you can calculate this for all your follows
and send a subscription for the authors
now you have two way blinded dms
On Sun, Jul 23, 2023, 10:38 AM Vitor Pamplona ***@***.***>
wrote:
> I think I am getting a good grasp on this + #574
> <#574> encryption scheme. But
> I would bring them together in a different way. Let's take private DMs as a
> use case.
>
> This protocol obfuscates the DM metadata from relays and the general
> public WHILE being recoverable in any client with the receiver or the
> sender's private key.
> 1. A new Kind 9 for Private DMs.
>
> This kind is always unsigned and will never be seen in the wild without a
> signed wrap.
>
> Complete Direct Message Payload:
>
> {
> "content": "This is a DM. Notice that there is no sender or receiver here, no signatures, and no ids",
> "kind": 9,
> "created_at": 1686840217,
> "tags": [],}
>
> 2. A new kind 10: Encrypted Unsigned Event
>
> Kind 10s sign *any* unsigned event for distribution in the protocol. The
> unsigned event is placed with the XChaCha encryption scheme (NIP-44) inside
> the .content. Clients should treat inner fields as if they were fields
> of the kind 10 event (merge them).
>
> Encrypted Unsigned Event:
>
> {
> "id": "<The usual hash>",
> "pubkey": "<Real Author's PubKey>",
> "content": "<NIP-44-Encrypted Kind 9>",
> "kind": 10,
> "created_at": 1686840217,
> "tags": [],
> "sig": "<Real Author's PubKey Signature>"}
>
> This event can be broadcasted, but, for DMs, this event should *not* be
> broadcasted automatically.
>
> Notice that there is no receiver exposed here even though the encrypted
> procedure encrypts only to the receiver's key. If this event leaks, it will
> only link to the sender. One will have to have the private key of the
> receiver to prove that this was a message for the receiver.
> 3. A Gift Wrap to hide the sender from the public.
>
> Public Wrap (Kind 1059):
>
> {
> "id": "<The usual hash>",
> "pubkey": "<Random, one-time-use PubKey>",
> "content": "<NIP-44-Encrypted Kind 10>",
> "kind": 1059,
> "created_at": 1686840217,
> "tags": [
> [ "p", "<Receiver>" ]
> ],
> "sig": "<Random, one-time-use PubKey Signature>"}
>
> The receiver is now exposed, but the sender is hidden.
> 4. A Gift Wrap to recover the sender's DM history when changing clients.
>
> Public Wrap (Kind 1059):
>
> {
> "id": "<The usual hash>",
> "pubkey": "<Random, one-time-use PubKey>",
> "content": "<NIP-44-Encrypted Kind 10>",
> "kind": 1059,
> "created_at": 1686840217,
> "tags": [
> [ "p", "<Sender>" ]
> ],
> "sig": "<Random, one-time-use PubKey Signature>"}
>
> By broadcasting this event, the private key of the sender can unwarp sent
> and received messages and reassemble the DM history in any client.
>
> The two wraps can be made separate with separate times to avoid time
> collision. And if we allow GiftWraps to contain more than one event (which
> I strongly recommend), then the 4th step can be made once at night with a
> sum of all messages sent during the day.
>
> The implementation is fairly straightforward:
>
> 1. Implement NIP 44 encryption, serialized in a Json style like @v0l
> <https://github.com/v0l> did in Snort
> 2. Implement kind 10s with overriding functions to pull from the
> inner event fields if present.
> 3. Render kind10 overridden by kind9s as DMs.
> 4. Implement Kind 1059, with overriding functions for the inner event.
> 5. Pull all gift wraps with p = logged in user from relays and
> process the inner event accordingly.
>
> Interesting benefits.
>
> - Sender and receiver are not linked in public.
> - Kind, tags, and created_at are hidden from the public.
> - Wraps create a bigger anonymity set because they can be anything,
> not only DMs, not only messages to me, but also from me.
> - The generic unsigned event allows us to take any Nostr event
> private. That's why we don't need an encrypted DM event anymore.
>
> —
> Reply to this email directly, view it on GitHub
> <#468 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAAMMUOHVD63B5RU4XIVH6TXRUZOVANCNFSM6AAAAAAXIZN43Y>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
|
I agree. Would be cool if window.nostr had a method for getting That's actually what I used to do in the old Iris which used gundb, and it worked nicely. |
|
yeah, we do this with the arcade app, and it allows blinded dm's .. we don't use gift wrap for that, because that leads to outbound subscription issues for state reloads. |
Hum... what do you mean by that? I seem to be running fine. |
you cant load the history of outbound messages when you log out and log back in... because with gift-wrap you have no idea who sent it. but if you use the hash(shared-secret + pub) as a private key... then you can easily subscribe to that author on the relay, and see all the fully blinded dm's. also you don't leak metadata if you use hash(shared-secret + pub) .... because you don't need any tags. you just always sub to the author. |
ohh I see. Yes, I agree. That's why I am sending a GiftWrap to myself as well.
How do you communicate the shared-secret to the other side?
There should be at least one tag of the subscription id, no? |
it's the standard shared secret (my priv * your pub)
no need to have tags on blinded dm's. no receipient, no sender. everyone just subs to the join well-known shared-secret key purely private dm |
I am confused on this. Can you provide a Nostr filter example? Do they filter by author or by ptag? |
In this case, you won't receive messages from strangers, right? Because the app woundn't be able to subscribe to each other active pubkey out there. |
|
correct, our system does not allow messages from strangers. also we allow private follows too (encrypted in a replaceable event) here's the sub we use: |
Yeah, that's a no go here. Oh I see. So, you do have a "tag", but your tag is the pubkey. It's just that you are using that pubkey as a central point. |
|
no I think that code is just to deal with various legacy stuff basically you can tag or not tag depending on how much metadata you want to leak vs whether or not you want to allow strangers to message people. we chose more private but I imagine a good write-up of the protocol would allow for both |
If two users are communicating with the same Private and Public Key, how do you know which one wrote each message without tags? |
|
ohh I see what you are doing. You have a custom payload where you place the pubkey of the writer together with the message in the So, in your scheme, you have to trust the counterparty, because the counterparty can add your pubkey in their message to pretend their message came from you. Tricky bet. |
I just re-read over this spec again and this is the only glaring thing that appears to be missing. Why no mention of deterministic gift wrap keys? Are clients just supposed to fire off random gift wraps with random keys and have no way of fetching them again? Maybe I'm missing something. |
|
You make an extra one for yourself firing from yet another random key. It's actually really good. You can choose "Disappearing Messages" if you don't give yourself a gift for your own DMs, for instance. It happens to make a lot of sense. In summary, you only need one filter p=your key to ALL gift wraps. Including those from and to yourself. |
|
nm I see it in there now, not sure how I missed it |
|
I see why I was confused, I was looking at the modified one by @vitorpamplona which doesn't mention deterministic ephemeral keys. who's the real gift wrap spec ? https://github.com/nostr-protocol/nips/blob/1c9c9719e068b4dbd405133decb4bf1ec8d367fe/59.md |
|
We have gift wraps now in 2-3 PRs, but I think they are mostly the same, just applied in different ways to add protections. The one I did is basically a double wrap of an unsigned event to make sure the wrapped event cannot leak and expose metadata. Then @staab created a version for large groups with Group management events to try to reduce the amount of GiftWrapped messages in those large group implementations (1,000+ people). |
|
Right, pull request #706 is my version. It distills double wrapping from Vitor's version down to its pure form (and has a distilled form of nip 44 as well). NIP 87 in the same PR is largely un-reviewed. If there's interest in getting 44 and 59 in, I can separate those into separate pull requests, or we can update the originals with my edits. I think the DM-specific stuff in the current versions should be superseded by Vitor's PR. Ultimately, I think we have four separate concepts here: versioned encryption, double-wrapping, small groups/dms, and large groups. Edit: I also did a survey of DM proposals which you can read here. It doesn't include #706 because that was written afterward. |
|
Replaced by #716 |
A stupid simple solution to DM metadata leakage.
Is it terrible? Yes!
Does it work? Also Yes!
https://github.com/v0l/nips/blob/59/59.md