Added DID resolution and dereferencing contracts.

[jricher]

---

Preview | Diff

[kdenhartog]

Few general comments to get clarity, but in general I'm in favor of this PR.

One question worth considering with this. Typically, document loaders handle the resolution process in JSON-LD in which case the idea of them complying with this contract for resolve and dereference doesn't quite make sense to me. Could someone explain how that would be reconciled with this PR?

[jricher]

@kdenhartog I'm not sure what you mean by "a document loader". If you already have the DID document, you don't need to do resolution or dereferencing -- those are used to GET the document (or some portion of it). If you're talking about loading an item referred to by a DID from within a document's field, then yes you'd call resolve if it's a DID and dereference if it's a DID URL, from within the document loader.

[iherman]

Isn't it possible to allow for a plural here, ie, a resolver may list several mime types that are accepted (say, JSON or CBOR). This is pretty much like the HTTP Accept header after all.

[jricher]

I agree that for this specific field a plural value makes sense. Supporting it is another story that's less clear. I think we've got a few choices here:

1. we allow multiple accept fields. I am strongly against this because it breaks the model described here, which currently maps nicely to JSON objects and other simple data structures like a Java Map interface
2. We allow rich objects like arrays as the values. I am strongly against this because it breaks the simple model here, and I think keeping the values as plain strings is going to be important in interoperability across multiple platforms and systems where different data structures aren't as prevalent. We also don't want to get into a world of recursive objects for the metadata.
3. We allow the field itself to define an internal syntax to express multiple values. This is already allowed by the surrounding text, but it's just that this field needs to define what that means. We can do comma separation, space separation, or just take the ABNF from the HTTP Accept header. I am in favor of this approach, but would suggest we add a new issue to define the details outside of this pull request.

[kdenhartog]

@kdenhartog I'm not sure what you mean by "a document loader". If you already have the DID document, you don't need to do resolution or dereferencing -- those are used to GET the document (or some portion of it). If you're talking about loading an item referred to by a DID from within a document's field, then yes you'd call resolve if it's a DID and dereference if it's a DID URL, from within the document loader.

In regards to what I meant by document loader, it was in reference to the function that I commonly see as a parameter in JSON-LD libraries. As for what you explained, around "loading an item referred to by a did..." this makes sense. That answer aligns with what I've seen, and I wanted to clarify in the event it potentially caused misalignment in other areas of this issue.

[msporny]

The DID WG should revisit our plan to attempt to express DID Resolution as a contract in the DID Core specification. This PR is making normative statements about the resolution protocol. It's clear that there is a subset of the group that really wants to work on DID Resolution and believe that we cannot do the DID Core spec without doing so. Rather than put overly complex language in the DID Core spec in an attempt to go half-way there, let's instead figure out how to pull DID Resolution into the group WITHOUT violating our charter. We need a clean separation of concerns between DID Core and DID Resolution. It is possible to pull in DID Core and refer to DID Resolution informatively while also giving a clear indication to implementers to the direction in which we're headed (standardizing DID Resolution).

-1 to this PR as is and the "contracts" approach, let's try to execute on a strategy that has a clearer separation of concerns.

[msporny]

Suggested change

	an identifier for a resource (the <a>DID document</a> or other). Alternatively, the
	an identifier for a resource. Alternatively, the

The parenthetical is vague enough to confuse, sentence is fine w/o it.

[jricher]

This was text not changed by this pull request apart from adding the link.

[msporny]

Suggested change

	parameters that are part of the <a>DID URL</a> should be used to specify
	parameters that are part of the <a>DID URL</a> are used to specify

[jricher]

These changes are editorial fixes outside this pull request and therefore I am not inclined to include them in order to keep pull this pull request smaller and more focused. Please submit a separate pull request to change this language instead of addressing it here.

[msporny]

"client" and "caller" are being used interchangeably in this PR and it's confusing if something different is meant when one is used over the other. If there is a difference, it's so nuanced that I can't tell. Are we talking about clients, callers, or both?

[msporny]

How would we test this "MUST NOT" normative statement? I'm struggling to think about what test would be written.

[jricher]

The client is defined as the component that calls the function, above. However it is awkward to say the "client of the function". Would appreciate suggested text.

[msporny]

If your argument is that we're merely outlining functions here and not protocol - then I suggest staying away from client. Alternatives include "caller of the function", "when the function is executed" or "consumer of the output".

Out of all of those, I'd prefer "when the function is executed", but it doesn't work here. Something like the following might be better:

In other words, a consumer of DID resolution output MUST NOT need to know the underlying representation of the metadata used by the implementation of the function.

Doing so doesn't give me any more insight into how we'd write a test in a test suite for this item.

[jricher]

I'm fine with having more precise definitions to use here, as client is a fairly generic term.

I'm not sure what you mean about the test suite. These function definitions actually make it much easier to write a test suite, and in fact make it easier to test the entire rest of the specification. You simply have the test suite implement a hook for resolve which is then implemented by whatever adapters you need to call different methods to do the actual resolution. The test suite code then doesn't need to change whether it's testing an implementation of the input of a function or the implementation of an output of a function or the implementation of tying those two together. So your test suite would test this by sending in a set of standard or common inputs and expecting a set of standard or common outputs. If those work, it passes. The test suite doesn't care what happens underneath as long as the contract is fulfilled, which is the very nature of a test suite.

[jricher]

I've removed the term "client" from this section, even though it's used elsewhere in the document.

[jricher]

@msporny The entire point of a contract is to make normative statements about the edge of the resolution process without making requirements for the internals of the resolution process. That's what this pull request does.

[msporny]

@msporny The entire point of a contract is to make normative statements about the edge of the resolution process without making requirements for the internals of the resolution process. That's what this pull request does.

Yes, and after having read two attempts at doing this in the DID Core spec, I'm no longer supportive of doing so in the DID Core spec. It confuses concerns -- the layering is wrong. Let's not do something half-baked, let's put this PR in DID Resolution, get clear about where normative statements should go about resolution, and make that the next focus of the group.

I'll submit some counter-proposals to this one in an attempt to provide some additional constructive input.

[jricher]

This PR does not belong in DID resolution. Without this structure, the rest of this document makes no sense. This is not a layering violation, and without this, there is in fact no layering in the document. The edges of everything are too fuzzy, which is why we have endless arguments about what's a parameter on the DID and what goes into the document. We need to have normative language about how the pieces fit together or else nobody will agree on where the definitions of the pieces start and stop, let alone how they might fit together in any fashion.

I believe that the language of this PR strikes a balance of defining what we need in terms of connections without defining at all what goes on within those connections. This is why I pushed back against the language in #247 as it went into too much detail on how to fulfill the contracts, with language on local vs. remote bindings and the discussions of underlying protocol structures. As this PR is written, it simply says you have to take in A and B, and you have to output X, Y, and Z. It says nothing about how you get from one to the other, and all existing DID methods should be able to comply with this pretty easily.

This PR does not define the mechanics of DID resolution beyond what is needed to understand the connection between a DID and its DID Document. If we take this out, or make it non-normative, then we need to remove all of section 9 (Methods) as well, especially 9.2 (Method Operations), as that goes into even greater depth than this at defining the required functions for DID Method implementations and creates just as much of a "protocol", if not more so. To be clear, I believe removing section 9 would be an egregious mistake, as I believe that not having the clear normative definitions that I am proposing here would be an egregious mistake.

Finally, If you have specific concerns about the requirements upon method implementations posed by these function definitions, then I would like to know what those are as I believe that such details can be worked through.

I look forward to reading and reviewing your counter-proposals and continuing this discussion.

[dlongley]

At a high level, there are two important pieces of "DID resolution" that must be defined in specs:

1. The thing a DID resolves to is a DID Document.
2. When two different pieces of software resolve a DID, they must get the same DID Document, and to do that they follows steps X, Y, and Z.

I believe we're chartered to do the former and not the latter. At this point, I think this PR only does the first (maybe we still need a tweak here or there, but it's on target). I also think that people will be able to write software that would pass tests for the new normative language here that could fail tests for the second piece. And that's a good thing; it means we've created the right separation.

We need to, as a group, stop saying we're not chartered to work on DID Resolution ... because I think that's very confusing. We are chartered to work on a piece of it, just not all of it.

[kdenhartog]

First off, thank you to everyone on the call that helped discuss this further. I think we're working towards alignment in a positive manner. After we got off the call I started thinking about some new angles of how we could approach this riffing on the idea of "a did MUST identify a did document that contains an id name-value pair with the value being that same did".

If we went down this road, it at least makes it clear that we've got a "deterministic mapping" between the identifier and the data model that it identifies. The edge cases I think everyone here wants to avoid are ones like urn:did-like:123 => { "@id": "did:example:123" } and did:example:123 => {}. As far as I can tell, there's nothing invalid about these deterministic, but clearly out in left field, mappings. I'm sure we also REALLY don't want to see things like this

did:example:123#key1 =>

{ 
   "@id": "did:example:123#endpoint1", 
   "type": "CredentialRepositoryService",
   "serviceEndpoint": "https://repository.example.com/service/8377464"
}

because then we've gotten to a point where interoperability is going to be nearly impossible between different implementations without behind the scenes coordination and we're allowing so much ambiguity that it leaves me scratching my head and wondering what do dids actually identify?

With that in mind, I think reasonably speaking what we're all after here is dids, did-urls, and did documents that are a means to an end, not the end in and of itself. To do that, I just can't seem to understand how it's valuable to us to leave so much room for ambiguity that did methods become their own silos. I think statements like:

The DID resolution function resolves a DID into a DID document by using the "Read" operation of the applicable DID method. The details of how this process is accomplished is outside the scope of this specification, but all implementations MUST implement a function in the form:

are incredibly useful for reducing the ambiguity in this spec and removing the edge cases like the ones above. So, let's make that the main objective here while keeping the spec simple and allow us to deliver the value that we all recognize dids and did documents, are useful for.

[msporny]

There is a line in our current charter that was a topic of debate in our DID WG special call today. Namely, this one:

Establish a deterministic mapping between DID method identifiers and the resolution process used to resolve that DID method.

Interpretations of what that sentence means have been varied and have spilled into the DID Resolution discussion. If the DID WG takes a consistent (and official) position on the interpretation of that statement, it'll help us figure out how to bring this DID Resolution discussion to a close.

Hindsight being 20/20, that statement probably should have said something more along these lines:

Define a concrete, generalized, and extensible DID resolution process for DID Method implementers that transforms a DID into a DID Document.

Specifically, this means that:

• It is in scope for the DID WG to define a concrete process that takes a DID as input and provides a DID Document as output.

• It is in scope to make that process take in options and provide back a document along with different classes of metadata (e.g., subject, document, and resolution metadata).

• It is in scope to add resolution tests to the test suite that exercise the generalized DID resolution process in the specification on concrete DID Method implementations.

• It is out of scope to normatively define DID Method specific details of implementing DID resolution.

• It is out of scope to normatively define DID Resolution protocols and representation formats.

• It is out of scope to test concrete DID Resolution protocols and data formats beyond the necessary process to demonstrate interoperability between the test suite and an implementation.

If we can get consensus on the above, it'll provide clarity around exactly how far we can take this DID Resolution discussion and exactly what is in scope and out of scope.

My request is that the Chairs put this (or some variation) to the WG during the next call where getting consensus on this would be an appropriate use of everyone's time.

/cc @brentzundel @burnburn @iherman

[talltree]

Manu, I think you did a great job of interpreting what was meant by that line in the charter. So +1 to your proposal. =Drummond

…

On Thu, Apr 16, 2020 at 11:18 AM Manu Sporny ***@***.***> wrote: There is a line in our current charter that was a topic of debate in our DID WG special call today. Namely, this one: Establish a deterministic mapping between DID method identifiers and the resolution process used to resolve that DID method. Interpretations of what that sentence means have been varied and have spilled into the DID Resolution discussion. If the DID WG takes a consistent (and official) position on the interpretation of that statement, it'll help us figure out how to bring this DID Resolution discussion to a close. Hindsight being 20/20, that statement probably should have said something more along these lines: Define a concrete, generalized, and extensible DID resolution process for DID Method implementers that transforms a DID into a DID Document. Specifically, this means that: - It is in scope for the DID WG to define a concrete process that takes a DID as input and provides a DID Document as output. - It is in scope to make that process take in options and provide back a document along with different classes of metadata (e.g., subject, document, and resolution metadata). - It is in scope to add resolution tests to the test suite that exercise the generalized DID resolution process in the specification on concrete DID Method implementations. - It is out of scope to normatively define DID Method specific details of implementing DID resolution. - It is out of scope to normatively define DID Resolution protocols and representation formats. - It is out of scope to test concrete DID Resolution protocols and data formats beyond the necessary process to demonstrate interoperability between the test suite and an implementation. If we can get consensus on the above, it'll provide clarity around exactly how far we can take this DID Resolution discussion and exactly what is in scope and out of scope. My request is that the Chairs put this (or some variation) to the WG during the next call where getting consensus on this would be an appropriate use of everyone's time. /cc @brentzundel <https://github.com/brentzundel> @burnburn <https://github.com/burnburn> @iherman <https://github.com/iherman> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#253 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZOITPW76J5ZXKXGMSG3HTRM5DXLANCNFSM4MEHR2JQ> .

[kdenhartog]

At a high level, there are two important pieces of "DID resolution" that must be defined in specs:

1. The thing a DID resolves to is a DID Document.

2. When two different pieces of software resolve a DID, they must get the _same_ DID Document, and to do that they follows steps X, Y, and Z.

I believe we're chartered to do the former and not the latter. At this point, I think this PR only does the first (maybe we still need a tweak here or there, but it's on target). I also think that people will be able to write software that would pass tests for the new normative language here that could fail tests for the second piece. And that's a good thing; it means we've created the right separation.

We need to, as a group, stop saying we're not chartered to work on DID Resolution ... because I think that's very confusing. We are chartered to work on a piece of it, just not all of it.

While I'd like to see 2 get answered too so we can avoid examples like I provided in my previous comment, I could live with this approach and think it's still a huge step in the right direction.

[peacekeeper]

One thought about the metadata section.. I agree with this language:

All bindings MUST provide a way for the caller of the function to input or receive the map of key-value string pairs as described in this specification. The exact representation of these metadata structures as part of a protocol or serialization is out of scope for this specification. For example, an HTTP-based binding can use HTTP headers of an HTTP request to convey input metadata and HTTP headers of an HTTP response to convey document and resolver metadata. Alternatively, a fully JSON-based binding can use a JSON object to represent the metadata structures.

However, I don't like the DID-CONTENT-TYPE example very much. It goes too much into implementation, and I don't think it's a particularly realistic example (more likely a concrete HTTP-based binding will just use the standard HTTP Content-Type header). Can we remove that whole example?

Also, I think an important point in my original PR #247 that may have gotten a bit lost here is that metadata may be linked to the representation of the DID document itself. The metadata may be part of the DID document, or there may be a higher-level data structure that contains both the DID document and its metadata (this is what the DID Resolution spec currently calls a DID Resolution Result). I can create a suggestion to add this idea back into the text, should only be a single sentence.

[OR13]

This PR feels like its trying to do too much at once, and its inviting objection due to its size.

I would love to see just the interface defined first...no algorithm, not a lot of normative spec text, just a signature... similar to https://fetch.spec.whatwg.org/#body-mixin

def resolve = (did: string, meta: object) => 
Promise<{didDocument: buffer, didDocumentMeta: object, resolverMeta: object}>

Let's get this first layer agreed too, before we talk about how to do parsing, or validation.

We can work our way up to a higher level interface that has error codes, etc... but lets just define the simple interface first.

IMO, this PR is closest to making it, and we should close the others and focus on getting this PR to the point where it can be merged, in smaller chunks, incrementally.

[msporny]

Let's get this first layer agreed too, before we talk about how to do parsing, or validation.

@jricher, this request came about after the DID WG Editor/Chair discussion last week. The result of the conversation last week was mixed, where some felt like we didn't get anywhere, and others felt like there was more clarity.

The end result is that none of the Editors felt like they could channel what you wanted and would rather see you take the current spec and define just the most basic interfaces that you can. Note that this would mean that there would be no mention of algorithms, no normative statements, what is processed where, etc. Meaning, large chunks of your original PR would have to be taken out.

The hope is that once we can get a very minimal thing on the inputs and outputs agreed to, we can decide how far to take things.

Do you feel like you would be able to put such a PR together by the special topic call this week?

[jricher]

@OR13 I started with it all together because all of the parts depend on each other in ways that are easy to miss when you only see the pieces. Those interlinks are important, but nonetheless I see the value in minimizing a change set. Manu already tried to split it into parts, and a small piece of it of that has already made it into the specification. I agree that there shouldn't be an algorithm, which is why this pull request doesn't have one listed in here. #263 is nearly the same text for the resolution section, but #286 is a separate pull request alltogether -- it added an algorithm, removed normative constructs, changed data requirements, and otherwise deeply altered the content and of this PR. It's a very different beast.

As for the fetch spec, that's the pattern I tried to use here, but without tying it to a specific programming language or platform (which is what fetch can do). I'll note that the entire fetch spec is normative and algorithmic, though, so I'm not sure how you can describe it otherwise. The function statement doesn't mean anything without the additional text surrounding it that describes what it does and how it works.

My pull request here did not try to define all actions and error conditions, but it did try to define a common set of data structures and ways to tie them together. This way, when the group decides what errors and actions need to be taken, we have a place to put them and a way to talk about them. This is why there isn't a strict processing algorithm but there is a strict definition of what the data types are, and what goes where.

@msporny I will gladly work with the editors to incorporate the changes that I and many others in the working group believe this spec needs to be workable. I do not think it makes sense to work on things in isolation, though. I only think it makes sense if all the pieces are available, even if they're in separate pull requests. I am very worried about a non-normative piece of guidance being accepted for this and all of the issues marked as resolved, as I have stated previously.

I look forward to continued work and discussion.

[dmitrizagidulin]

@OR13

I would love to see just the interface defined first...no algorithm, not a lot of normative spec text, just a signature... similar to https://fetch.spec.whatwg.org/#body-mixin

+1, very much agree to the WHATWG fetch() approach to the resolution layers.

[msporny]

PR #331 has been merged. @jricher, can we close this PR now?

[peacekeeper]

I agree that after merging #331, this can probably be closed. If there is anything in this PR that we still want to get into the spec (e.g. language around bindings and transformations), then this can still be added on top of the current spec in a separate PR. @jricher do you agree?

Also, I created a new issue to track DID URL dereferencing: #364. This PR here has some content about DID URL dereferencing that can potentially be re-used in a new PR to address this issue.

[msporny]

Closing this in 48 hours if we don't hear back from the PR author.

[msporny]

No response from original PR author, closing.