[NIP-47] Add versioning and migrate to NIP-44.

[jklein24]

NIP-04 is deprecated and no longer recommended for use. Authors of NIP-44 have confirmed that it can act as a drop-in replacement for NIP-04 for NWC's use case.

Versioning for NIP-47 allows the protocol to evolve and even make breaking changes while maintaining backwards-compatibility where needed. Versions are represented as <major>.<minor> (e.g. 1.3). Major version bumps imply breaking changes that are incompatible with the previous major version. Minor version bumps, however, only include non-breaking feature additions or improvements which can maintain full compatibility with the previous minor version.

[vitorpamplona]

Ack

FYI, NIP-46 (#1248) is just dynamically figuring out if the encryption is nip-04 or nip-44.

fun isNIP04(ciphertext: String): Boolean {
    val l = ciphertext.length
    if (l < 28) return false
    return ciphertext[l - 28] == '?' && 
           ciphertext[l - 27] == 'i' && 
           ciphertext[l - 26] == 'v' && 
           ciphertext[l - 25] == '='
}

The migration over there has three steps:

1. Everybody decrypts both nip 04 and nip 44
2. Everybody starts encrypting with NIP44 in their own timeline.
3. Everybody drops support for encrypting and decrypting with nip04.

[jklein24]

Ack

FYI, NIP-46 (#1248) is just dynamically figuring out if the encryption is nip-04 or nip-44.

fun isNIP04(ciphertext: String): Boolean {
    val l = ciphertext.length
    if (l < 28) return false
    return ciphertext[l - 28] == '?' && 
           ciphertext[l - 27] == 'i' && 
           ciphertext[l - 26] == 'v' && 
           ciphertext[l - 25] == '='
}

The migration over there has three steps:

1. Everybody decrypts both nip 04 and nip 44
2. Everybody starts encrypting with NIP44 in their own timeline.
3. Everybody drops support for encrypting and decrypting with nip04.

Thanks, this also makes sense as the alternative path for NIP44 migration, and is certainly simpler. My main reason for not going this route right now is so that connections can use NIP44 right away if both sides indicate support, without needing to wait for a migration or try encrypting with NIP44 first, getting an error, and then re-encrypting with NIP-04.

I also think in general a versioning scheme will be useful for NWC for future breaking changes if they're needed, but I recognize that it adds complexity.

[rolznz]

Linking the notifications PR which ideally is merged (it has multiple implementations using it already) and has implications for versioning.

Because there is no communication from the client application to the wallet service to listen to notifications (the client application is simply subscribing to the relay), I think the simplest way would be to update the notifications PR to use a different notification event kind for version 1.0, and start including the tag in the notification event. That way, version v0.0 apps will not break by trying to decode v1.0 notifications.

I do not know how useful the version tag will be on notifications though, or if another breaking change in the future would require another change to the notification event kind, since the wallet service might still want to support the previous version.

[ekzyis]

Looks good!

The v tag leaks metadata about connections but it also allows us to track adoption rate of new versions and thus when we can start dropping old versions.

Is this implemented in any wallet service yet with which I could test against as a client?

[ekzyis]

What about patch versions? I think clients would also be interested in knowing if the service has the latest patches.

[jklein24]

My take was the same as @bumi mentioned with regards to patch versions. If they don't really change compatibility, and versions don't bump often, I think patch versions will add more complexity then benefit.

[bumi]

lgtm, thanks!
I do not expect many versions (and rather bigger changes like the switch to nip-44) because too many versions (of a spec) lead to incompatibilities which must be avoided. (thus I do not care about patches here)

Alby plans to implement this with NIP44

[jklein24]

Looks good!

The v tag leaks metadata about connections but it also allows us to track adoption rate of new versions and thus when we can start dropping old versions.

This is a really good callout, thanks for pointing it out. The adoption rate tracking is nice though. I think this is a reasonable tradeoff.

Is this implemented in any wallet service yet with which I could test against as a client?

Currently only in a demo with the uma-nwc-server docker image, but it sounds like it'll be in alby hub in the not-so-distant future too :-)

[jklein24]

Linking the notifications PR which ideally is merged (it has multiple implementations using it already) and has implications for versioning.

Because there is no communication from the client application to the wallet service to listen to notifications (the client application is simply subscribing to the relay), I think the simplest way would be to update the notifications PR to use a different notification event kind for version 1.0, and start including the tag in the notification event. That way, version v0.0 apps will not break by trying to decode v1.0 notifications.

I do not know how useful the version tag will be on notifications though, or if another breaking change in the future would require another change to the notification event kind, since the wallet service might still want to support the previous version.

Thanks for flagging this! I defer to you on what's best for notifications and how versioning could be relevant for them. What you've suggested with a new notification event kind sounds somewhat reasonable to me for forward-compatibility, but I could see how it'd be a bit of an annoying migration flow :-/

[ekzyis]

This is a really good callout, thanks for pointing it out. The adoption rate tracking is nice though. I think this is a reasonable tradeoff.

I realized that it actually doesn't leak anything about connections since the requests and responses use ephemeral events (or are at least supposed to) so we can't track adoption rate of clients. We can only track adoption of wallet services by filtering for kind:13194. I think knowing which wallet service supports which version is unrelated to privacy unlike being able to distinguish individual connections from each other.

Was just something I noticed and wanted to mention, it wasn't (and still isn't) a real concern of mine.

I'll make sure that SN supports NIP-44 for NWC soon to see if I have any other feedback.

[frnandu]

Notifications got merged , #1164
so here it needs to adapt to new spec, conflicts resolved, and there is the issue of which kind of encryption to use on kind =23196 notifications, which might be subscribed by different client with same nostr+wallet:// connection that support different versions (nip04 or nip44) to be discussed...

[jklein24]

Notifications got merged , #1164 so here it needs to adapt to new spec, conflicts resolved, and there is the issue of which kind of encryption to use on kind =23196 notifications, which might be subscribed by different client with same nostr+wallet:// connection that support different versions (nip04 or nip44) to be discussed...

Apologies for the slow response here, thanks for poking on it @frnandu. @rolznz I'm curious whether you have thoughts on the right path here wrt notifications. I can update to use a new notification event kind for v1 like you suggested and we can just use nip44 to encrypt those as well as add the version tag. 23196 events would still need to be nip04-encrypted though in that case. In addition, I guess if a wallet service supports both v0 and v1+, it might need to send both notification even kinds for a given event until it knows which version the client supports (via the client's most recent nwc request). As an example:

1. Client establishes a connection.
2. Wallet receives a payment and needs to send a notification, so it sends both 23196 (nip04) and 23197 (nip44) events to the client.
3. Client makes some request (get_info) which indicates a v tag of 1.x. The wallet service can optionally cache this version as its notification version as well, so that it only needs to send one event to that client next time. Note that there's some risk here in that clients need to make some request after upgrading their version in order to receive the right notification type. However, I think in practice, it's pretty likely that this would happen anyway.

The extra per-client-version caching does add some complexity on the wallet side, which is unfortunate. However, since wallets already need to cache info about clients (permissions, budgets, etc), this seems pretty manageable. It would also be good to have some way for clients to reliably dedupe 23196 and 23197 events if they have to listen for both at first. I guess for the existing notifications, the payment hash is sufficient. If we don't add any more notification types before v1, maybe that's good enough?

Does that sound right to you?

[rolznz]

@jklein24 that sounds good to me, except the below, I am not sure is necessary:

The wallet service can optionally cache this version as its notification version as well, so that it only needs to send one event to that client next time. Note that there's some risk here in that clients need to make some request after upgrading their version in order to receive the right notification type. However, I think in practice, it's pretty likely that this would happen anyway.

I hope we can convince client apps to update within 6-12 months, and then stop sending NIP-04 notifications from wallet services after that. As far as I know, Alby Hub is the only wallet service that supports sending NIP-47 notifications right now (although there are multiple clients that consume them). As long as we update Alby Hub to support both, client apps are free to switch to NIP-44.

[ekzyis]

Regarding SN: we will rely on NDK to handle NIP-04 vs NIP-44 (stackernews/stacker.news#1590) but it doesn't use NIP-44 for NWC yet.

There is a ticket though (nostr-dev-kit/ndk#166) and two related PRs (nostr-dev-kit/ndk#233, nostr-dev-kit/ndk#279) but not sure if they will include NIP-44 for NWC. I'll ask.

[frnandu]

3. Client makes some request (get_info) which indicates a v tag of 1.x.
   The wallet service can optionally cache this version as its notification version as well, so that it only needs to send one event to that client next time.
   Note that there's some risk here in that clients need to make some request after upgrading their version in order to receive the right notification type.
   However, I think in practice, it's pretty likely that this would happen anyway.

Mostly yes, but a NWC connection can be used by multiple clients, even if that is not the recommended way, I sure do it often.
And each client might support different versions, so assuming that a connection is tied to a strict client version support might not always be the case.
Worst UX scenario is for example a case where notifications work for some legacy client, but then you use that same connection in a newer supporting-nip44 client and the wallet service decides that from now on notifications for that connection will only send in new 23197 kind, and the legacy client stops receiving notifications in 23196.

[jklein24]

3. Client makes some request (get_info) which indicates a v tag of 1.x.
   The wallet service can optionally cache this version as its notification version as well, so that it only needs to send one event to that client next time.
   Note that there's some risk here in that clients need to make some request after upgrading their version in order to receive the right notification type.
   However, I think in practice, it's pretty likely that this would happen anyway.

Mostly yes, but a NWC connection can be used by multiple clients, even if that is not the recommended way, I sure do it often. And each client might support different versions, so assuming that a connection is tied to a strict client version support might not always be the case. Worst UX scenario is for example a case where notifications work for some legacy client, but then you use that same connection in a newer supporting-nip44 client and the wallet service decides that from now on notifications for that connection will only send in new 23197 kind, and the legacy client stops receiving notifications in 23196.

That's another really good point, @frnandu. I guess we're stuck with what @rolznz is suggesting for now then - we just always send both events and clients will need a way to dedupe (payment hash for now). @rolznz do you have a sense of how broad client usage is for notifications at the moment? I'm trying to get a rough gauge on how painful/long the migration will be so that hopefully we don't need to be firing both notification events for long. The client-side deduping is kinda gross :-/.

[rolznz]

@jklein24 I would not do client-side deduping, just use the get_info call first to figure out if the wallet service supports NIP-44 or not, and then only subscribe to the correct kind, like you suggested in 3. Or do you see issues there?

I'm not sure how long the migration would take, but as soon as we release a version of Alby Hub that supports it I would contact all known developers and ask them to update. The apps I know about are listed in #1164 (comment)

We are actively working on supporting NIP-44.

[jklein24]

@jklein24 I would not do client-side deduping, just use the get_info call first to figure out if the wallet service supports NIP-44 or not, and then only subscribe to the correct kind, like you suggested in 3. Or do you see issues there?

I'm not sure how long the migration would take, but as soon as we release a version of Alby Hub that supports it I would contact all known developers and ask them to update. The apps I know about are listed in #1164 (comment)

We are actively working on supporting NIP-44.

Sorry for the delay here, but I've rebased and added proper context for notifications @frnandu @rolznz. Let me know if this looks right to you.

[rolznz]

Would it also make sense to propose v as a standardized tag (further down in this file)?

[jklein24]

I actually considered this, but decided to avoid the larger discussion on a general version tag or implications in nostr events more broadly than in nwc. That being said, maybe it's worth listing here to be explicit. @vitorpamplona any thoughts on adding the v tag here as a standardized tag?

[vitorpamplona]

Better to stay out of it since NIP-47 seems to be the only place using it. Let's see if other nips decide to adopt first.