Why DID Document streaming (resolveStream)?

[wyc]

Hi, I've been working on the DID URL Dereferencing section as per #364 and was also reviewing the DID Resolution section for consistency. I noticed that there were two functions, resolve and resolveStream. To me, streaming is closer to an implementation detail, and I'm not sure that it's within our purview to require both modes. At this time I have to consider if DID URL Dereferencing needs streaming or not within the spec for my work. It seems to have been added here: 2da9671

Unfortunately I couldn't find any discussion, so I'm opening an issue.

1. Are there notes from the discussion preceding this decision?
2. Why did we want to require a stream function at all?
3. Can someone explain how it was decided that an implementer MUST implement both resolve and resolveStream?

Here are some reasons I came up with that could make sense as to why things are they way they are:

• Why have streaming at all in the spec? DID Documents could get huge, and this could be future-proofing
• Let's just assume we want streaming. Why are they two different functions? We want conforming implementations to always be able to produce a DID document conforming to the data models listed, so therefore we cannot "bunch together" streaming and non-streaming responses, because this would allow terrible things (tm) like a DID method author returning some absurd binary packed message and telling a resolver to "go figure, I'm conforming to the spec because you can get a DID document out of this hot mess if you cast the right spells and incantations but I won't actually return one conformant to one of the data models described (JSON-LD/CBOR-LD/etc.) because I don't have to."

I feel that if we changed the definition of did-document into one that also accepted byte streams, then we could eliminate the latter abstract function along with did-document-stream. Here's some possible language to make that happen:

A: Incorporate Streaming

did-document
If the resolution is successful, and if the resolve function was called, this MUST be a DID document conforming to the abstract data model or a byte stream which can be parsed by the caller of the resolve function into a DID document conforming to the abstract data model. Implementers MAY use a combination of did-resolution-metadata fields and "Read" operation of the DID method to indicate values which are byte streams. If the resolution is unsuccessful, this value MUST be empty.

This is scary for the reason above, that an implementation never actually needs to return a conforming data blob.

B: Eliminate Streaming

did-document
If the resolution is successful, and if the resolve function was called, this MUST be a DID document conforming to the abstract data model. If the resolution is unsuccessful, this value MUST be empty.

I like this, in addition to suggest that an implementer MAY implement streaming functions in addition to this one.

Any answers to questions 1, 2, & 3 + constructive opining on my perspective above appreciated. Thanks.

cc (from git-blame) @peacekeeper @msporny @jricher

[peacekeeper]

@wyc there have been numerous issues, PRs, and special topic calls in the WG that have led to this design, e.g. see:

• https://www.w3.org/2019/did-wg/Meetings/Minutes/2020-05-21-did-topic
• https://www.w3.org/2019/did-wg/Meetings/Minutes/2020-05-28-did-topic
• https://www.w3.org/2019/did-wg/Meetings/Minutes/2020-06-11-did-topic
• https://www.w3.org/2019/did-wg/Meetings/Minutes/2020-06-18-did-topic
• https://www.w3.org/2019/did-wg/Meetings/Minutes/2020-06-25-did-topic

The PR that eventually introduced the two functions after a long discussion process is #331. It has links to various previous issues and PRs were all the discussions can be found.

The short explanation is that in some situations, you can resolve a DID to a DID document in its abstract form, without ever having to serialize or parse the DID document in any concrete representation (JSON-LD, JSON, CBOR, etc.). An example is resolving a DID of the "did:key" method. In other situations, you resolve a DID to a DID document in one of the concrete representations (e.g. JSON-LD). In this case the DID document is returned as a byte stream and can be parsed by a client.

[wyc]

Thank you for diligently adding the context @peacekeeper, I'm so glad we take notes. Will review the above.

[wyc]

I largely agree with @jricher's points here, in that resolve and resolveStream should exist as two separate sections:

Justin Richer: I only just now got a chance to read through 331
… thank you markus
… i think it has probably the right content, i’ve not had chance to read it in detail
… but i think it’s probably right
… i would not have organise the two functions next to each other like that, editorially better to define them separately and have their inputs described separately
… otherwise you end up with if it’s this you must use this otherwise don’t do this… it’s messy

Manu Sporny: agree with Justin, and that it’s editorial.

Justin Richer: the functions should probably be defined separately but that can happen editorially after

Manu Sporny: (so we can do that later)

I am willing to open an issue and do the work here, pending merge of #381.

Two follow-ups:

• Is there a specification for the "abstract function" formats we use, or is that introduced in this spec? As in (input0, input1, ..., inputN) -> (output0, output1, ..., outputN)...
• Am I correct in my thinking that in many implementations, resolve can likely use resolveStream as a dependency?

[peacekeeper]

Is there a specification for the "abstract function" formats we use, or is that introduced in this spec? As in (input0, input1, ..., inputN) -> (output0, output1, ..., outputN)...

I don't think so, we made that up ourselves. We briefly considered using WebIDL, but that is for concrete browser APIs rather than abstract function definitions.

Am I correct in my thinking that in many implementations, resolve can likely use resolveStream as a dependency?

The way I see it, this depends on how concrete DID methods and resolver implementations are architected. I could see potential dependencies between the two functions both ways:

1. Let's say you want to call resolve(did:btcr:xxx, {}). A resolver could implement this function by first calling resolveStream(did:btcr:xxx, {"accept": "application/did+ld+json"}), which in turn could retrieve a JSON-LD did-document-stream from a remote endpoint, which is then parsed and returned as a did-document. Now you could say that resolve() uses resolveStream() as a dependency.

2. Let's say you want to call resolveStream(did:key:xxx, {"accept": "application/did+ld+json"}). A resolver could implement this by first calling resolve(did:key:xxx, {}), then serializing the did-document to JSON-LD, and then returning it as did-document-stream. Now you could say that resolveStream() uses resolve() as a dependency.

I think this kind of architectural flexibility is one of the reasons why we ended up with two different functions, one that deals with the abstract data model, and one that deals with concrete representations. Further implementation details are out of scope for DID Core.

[msporny]

This issue needs to result in a PR in the next 30 days or it will be closed/deferred to the next version of the specification.

[OR13]

its been renamed https://w3c.github.io/did-core/#did-resolution

"resolveRepresentation"

Propose closing, and addressing any challenges raised by tests, when we have tests.

[msporny]

The DID WG discussed this during the 2021-01-12 teleconference, the language has been renamed to "resolveRepresentation" (as @OR13 notes above).

The group does not think that there is further action that needs to be performed for this issue. This issue will be closed in 7 days unless there are objections.

[peacekeeper]

I agree with closing this.

[brentzundel]

No objections raised, closing issue