-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement the Identity Agent #322
Conversation
5816a38
to
0b5eb5b
Compare
3839cfe
to
074e6cd
Compare
43dcd57
to
be49946
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great,
Thank you for all the extra effort you put in to finding the best possible names, write great documentation and vastly improve the readability of the tests.
As we discussed we may want to change the architecture somewhat in the feature in order to give users more control over performance, but that should be considered after we've merged this (unstable feature) and got some experience working with it (when implementing some DIDComm protocols).
Finally, I left a few comments you may wish to address before merging this.
Description of change
The identity agent is a peer-to-peer communication framework for building SSI agents on IOTA Identity. It is intended to host implementations of the DIDComm protocols with future updates. Together with these protocols, this will, for example, allow for did-authenticated communication between two identities to exchange verifiable credentials or presentations.
This description serves as a technical introduction to the agent. For a high-level and less technical introduction, see the blog post on the agent (formerly known as identity actor).
The most important dependency of the agent is libp2p. What is libp2p?
We use libp2p because it can easily secure transports using the noise protocol, is agnostic of transports so agents could conceivably communicate over TCP, websockets or Bluetooth, and because of how flexible it is, we can make it suit the agent nicely.
Building an agent
To build a minimal working agent, we generate a new
IdentityKeypair
from which theAgentId
of the agent is derived. TheAgentId
is an alias for alibp2p::PeerId
, which allows for cryptographically verifiable identification of a peer. This decouples the identity concept from the underlying network address, which is important if the agent roams across networks. If we want the agent to have the sameAgentId
across program executions, we need to store this keypair. Next we create the address for the agent to listen on. AMultiaddr
is the address format in libp2p to encode addresses of various transports. Finally, we build the agent with a default transport, that supports DNS resolution and can use TCP or websockets.Processing incoming requests
To make the agent do something useful, we need handlers. A handler is some state with associated behavior that processes incoming requests. It will be invoked if the agent is able to deserialize the incoming request to the type the handler expects. The
Handler
is a trait that looks like this:&self
so it can modify its state, through appropriate mechanisms, such as locks. A handler will thus typically implement a shallow copy mechanism (e.g. usingArc
) to share state.HandlerRequest
trait and needs to return the defined response type.Here is an example of a handler being attached on an
AgentBuilder
. We implementRemoteAccounts
, an exemplary type that managesAccount
s remotely.An agent that receives a request will check whether a handler is attached that can handle the request's
Endpoint
and if so, will invoke it. In our case, the agent will call thehandle
function of the handler when aRemoteAccountsGet
request is received. If we wanted, we could attach more handlers to the same agent, and even implementHandler
forRemoteAccounts
multiple times, in order to handle different request types.Sending requests
To invoke a handler on a remote agent, we send a type that implements
HandlerRequest
, such asRemoteAccountsGet
.After building the agent and adding the address of the remote agent, we can send a request. The agent takes care of serializing the request, and attempts to deserialize the response into
<RemoteAccountsGet as HandlerRequest>::Response
.Agent modes
We've just seen an example of a synchronous request, one where we invoke a handler on a remote agent and wait for it to finish execution and return a result. Next to the
Agent
type we also have aDidCommAgent
type. The latter additionally supports an asynchronous mode, where we send a request without waiting for the result of the handler invocation. Instead, we can explicitly await a request:This request mode is implemented to support the implementation of DIDComm protocols, which is why a separate
DidCommAgent
is defined that extends theAgent
s functionality and handles the specifics of DIDComm. Note that the baseAgent
doesn't support the asynchronous mode, but theDidCommAgent
supports the sychronous mode.Here, the protocol expects us to first send a
PresentationOffer
request to the remote agent. This method call returns successfully if the request can be deserialized properly and if an appropriate handler exists on the remote agent, but the call might return before the handler on the remote has finished. According to the protocol we implement, we should expect the remote to send us aPresentationRequest
so we explicitly callawait_didcomm_request
to await the incoming request on the sameThreadId
that we sent our previous request on. This allows for imperative protocol implementations within a single handler. This is nice to have, because the alternative would be that each request invokes a separate handler in an agent, which would force protocol implementors to hold the state in some shared state, rather than implicitly in the function (such as thethread_id
here). This setup is intended for DIDComm protocols, as it directly implements DIDComm concepts such as threads.Hooks
While the asynchronous mode of operation allows for implementation of DIDComm protocols, it does not yet allow users to hook in their own logic. This might be required to ask a user for consent before proceeding with the exchange of a credential.
Hooks are no longer part of this PR, and will be implemented later. The hook approaches thus far were unsatisfactory, but we keep the original text for reference.
Previous text
To that end, the `DidCommActor` has the concept of hooks, which are similar to handlers. There are various approaches to hooks:The actor currently implements the implicit hooks approach, mostly as an exploratory option, not necessarily because it was deemed the best one. Suppose we want to insert custom logic before
await_didcomm_request
returns thePresentationRequest
in the above eample. One registers a hook like so:The signature of a hook function is very similar to a handler, except for its return type. It needs to return the same type that it received as input or an instruction for the actor to terminate the connection. The endpoint of a hook includes the
/hook
postfix, and indicates to the actor that this hook should be called either before anHandlerRequest
with adidcomm/presentation_request
endpoint is sent, or before it is returned fromawait_didcomm_request
, as is the case in this example.Limitations
The limitation of this hook implementation is thus that the user can only insert custom logic in between network requests, but not at arbitrary points in the protocol. With the current implementation (and some small modifications), it would also be possible to allow protocol implementors to define arbitrary hook points and call them.
In theory hooks can be attached to the same state as a handler, so they can access and modify the handlers state. However, in practice we will likely have a function that adds certain protocol handlers to an
ActorBuilder
and a user will add their hooks at other points. At this point, however, the ephemeralHandlerBuilder
will have gone out of scope and no more typing guarantees could be given that a hook uses the same state object as a handler, e.g. there could be an incompatibility. Because of this lack of typing guarantees, it is currently not possible at all to attach a hook to some previously added state, as it seems very error-prone.An alternative would be to use
std::any::TypeId
for identification of a state object, so if a handler and a hook take a certainOBJ
type as state object, then the actor can determine theTypeId
of the object and guarantee internally, that it will be called with the same object. The downside is, that users cannot add two different state objects that have the sameTypeId
.Implementation Details
This section goes into some details of agent internals.
The overall architecture can be seen as four layers. A libp2p layer, a commander layer to interact with the libp2p layer, the raw agent layer (which uses the commander) and the
DidCommAgent
on top. This architecture is strongly inspired by stronghold-p2p.libp2p::RequestResponse
protocol, which enforces on a type level that each request has a response. This naturally maps to the sync mode of the identity agent where each request has some response, as well as to the async mode where each request will be acknowledged.EventLoop
that concurrently polls the libp2pSwarm
to handle its events as well as commands that are sent to it from theNetCommander
.NetCommander
communicates with the event loop via channels and is thus the interface for theEventLoop
.EventLoop
in the background and interacts with it using theNetCommander
.EventLoop
spawns a new task and injects a clone of the agent into it (seeEventLoop::run
and its argument).Examples
This PR does not add examples to the
examples
directory. This is mostly due to the instability of the agent. Still, there are two "examples" for each mode of operation as part of thetests
module, the remote account as a synchronous example, and the IOTA DIDComm presentation protocol as an asynchronous example (this doesn't implement the actual protocol, it just asserts that requests can be sent back and forth as expected). The DIDComm example in particular is very simple and minimal and mostly exists as a proof of concept for the async mode, but it also serves as an example for how a DIDComm protocol could potentially be implemented.DIDComm example setup
The async mode didcomm examples are worth explaining a little more. The implementation difficulty for these protocols comes mostly because of how flexible they are. In the presentation protocol for example, both the holder and verifier can initiate the exchange. On the agent level this means either calling the protocol explicitly to initiate it, or attaching a handler to let the agent handle the protocol in the background when a remote agent initiates. Thus, there is one function that implements the actual protocol for each of the roles (i.e. holder and verifier in the
presentation
protocol). As an example, this is what the signature of the holder role would look like:The holder can call this function to initiate the protocol imperatively by passing
None
asrequest
. On the other hand, if the verifier initiates, the holder defines a handler that will inject the receivedrequest
:and attaches it:
DidCommState
holds the state for one or more DIDComm protocols. When aPresentationRequest
is received, it calls the protocol function (presentation_holder_handler
) to run through the protocol. This allows us to nicely reuse thepresentation_holder_handler
function as the core protocol implementation and only requires defining a thin handler method. The verifier can follow the same pattern for their side of the protocol.DIDComm agent internals
DidCommAgent
returns an acknowledgment if 1) a handler for the endpoint or a thread exists and 2) if the request can be deserialized into the expected type for the handler or thread (e.g. a DIDComm plaintext message)AgentBuilder::timeout
.InboundFailure::Timeout
if the peer did not respond within the configured timeout. This happens on the event loop level and is handled by theRequestResponse
protocol.DidCommAgent::await_didcomm_request
can time out. This is the same timeout value as for the underlyingRequestResponse
protocol. In such a case, the event loop will receive a timeout error, but since no entry in the thread hash map is waiting for a response, it is silently dropped. Thus,await_didcomm_request
implements its own timeout, and automatically uses the same duration as the underlying protocol to ensure consistent behaviour. For this reason, theawait_didcomm_request
timeout is a per-agent configuration value, and not a parameter on the function, although that would also be possible if desired.Open Questions
RequestMessage::from_bytes
with some serialization that's faster or more compact? For now, it seems good enough to use json. Some crate like rkyv would make the agent potentially less easily portable, if it would ever receive an implementation in a foreign language, while json is highly compatible. How much that downside matters with our single source code of truth approach is debatable. But this particularRequestMessage
type is fairly simple anyway, and so the serialization performance gains might not be that significant?Links to any relevant issues
fixes #299
Type of change
Add an
x
to the boxes that are relevant to your changes.How the change has been tested
remote_account
anddidcomm
respectively.identity-agent/src/tests
which use those examples.Change checklist
Add an
x
to the boxes that are relevant to your changes.