WebSocket protocol format and definition

[acoburn]

The 0.8 Solid specification includes a WebSocket API, which defines a simple line-based interaction between client and server. For example, here is a subscription to a container followed by two notifications for child resources that are created:

> sub https://example.com/container
< pub https://example.com/container/1
< pub https://example.com/container/2

That design has the advantage of being simple to implement and simple to consume. That said, most WebSocket APIs that I have encountered use a JSON-based format, which provides a more structured format for consumption (the format could even be compact JSON-LD) along with easier points for extension (e.g. what if other keywords are to be added?).

In addition, the JavaScript API for WebSockets (WebSocket(url[, protocols])) includes an optional protocols parameter. If this interaction is formalized, would it make sense to define a solid protocol that could be used for WebSocket interactions?

[RubenVerborgh]

Indeed, we would need to engineer the protocol properly (see solid/specification#50). JSON(-LD) could be a good candidate.

[dmitrizagidulin]

Related issues:

• solid-spec/179 - [proposed spec change] Mention read caps in Updates-Via
• solid-spec/180 - Should updates about a container only subscribe to direct members, or recursively?
• solid-spec/93 - Web sockets API: when is subscription terminated?
• solid-spec/191 - [proposed spec change] subscribe to whole subtree of container
• solid-spec/123 - WebSockets and HTTP/2
• solid-spec/167 - WebSocket access tickets

[timbl]

PlanA at back in the day to take this question toward using patches in N3 so that the metadata about the patch (user, time, etc) could be put in as well as the changes (deletions, insertions). Ruben implemented N3 patch format in rdflib.js. The changes made by the UpdateManager in the client would be very easy to serialize in this this way.

This is long term goal. It connects of course with transaction logging on the server. local-first, offline first, and peer-peer networks as enhancements on the current server hub/spoke topology. We should not try to combine this enhancement with the authentication of WS bug fix.

[RubenVerborgh]

@timbl I would argue that authentication is more than a bug fix.

It's not just authentication that was forgotten, there are some other essential decisions in the protocol that are problematic. The fact that such basic things were overlooked, indicates to me that the WebSockets draft was not designed with sustainability in mind. I have strong doubts about the design quality of the WebSockets draft in general, and think it is unwise to spec it as is. I don't see it pass quality assessments.

I do acknowledge that the current draft was a quick way to address a real and important need. And I think we should just see it as that: an experiment to show that it works. Now that we know it does, we should actually design a protocol.

[timbl]

Sure lets design a nice new future protocol. But mean the lack of authentication is a serious bug with the current protocol. And current code and current apps use this feature. So we need to solve that problem urgently, on a timescale which I think will not wait for the new protocol for downstream patches.

[RubenVerborgh]

Agreed on fixing fast for apps currently in use. Just have doubts about spec'ing it, as we know it is not the sustainable protocol we want. Can it stay unspec'ed for the time being?

[d-a-v-i--]

I understand there is desire to look at other protocols and this will take time. Meanwhile I've cut a quick draft for WebSocket authentication and my aim is to make it generic enough to work on whatever protocol we choose.

[michielbdejong]

I've cut a quick draft for WebSocket authentication

@d-a-v-i-- that is requiring clients to send the auth <bearer_token> command when they connect, right?

[michielbdejong]

@acoburn did you do any work on the auth command since this? I'm about to implement it in rdflib.

[michielbdejong]

The thing that's slightly awkward about it is that we now have 2 auth headers to send, Authorization and DPop. So I was thinking of instead of creating one auth command and wrapping both into the parameters somehow, we could do:
Suppose that the client sends the http request headers like in Figure 4 then on the WebSocket you could achieve the same by sending two corresponding auth-header commands:

auth-header Authorization: DPoP eyJhbGciOiJFUzI1NiIsImtpZCI6IkJlQUxrYiJ9.eyJzdWIiOiJzb21lb25lQGV4YW1wbGUuY29tIiwiaXNzIjoiaHR0cHM6Ly9zZXJ2ZXIuZXhhbXBsZS5jb20iLCJhdWQiOiJodHRwczovL3Jlc291cmNlLmV4YW1wbGUub3JnIiwibmJmIjoxNTYyMjYyNjExLCJleHAiOjE1NjIyNjYyMTYsImNuZiI6eyJqa3QiOiIwWmNPQ09SWk5ZeS1EV3BxcTMwalp5SkdIVE4wZDJIZ2xCVjN1aWd1QTRJIn19.vsFiVqHCyIkBYu50c69bmPJsj8qYlsXfuC6nZcLl8YYRNOhqMuRXu6oSZHe2dGZY0ODNaGg1cg-kVigzYhF1MQ
auth-header DPoP: eyJ0eXAiOiJkcG9wK2p3dCIsImFsZyI6IkVTMjU2IiwiandrIjp7Imt0eSI6IkVDIiwieCI6Imw4dEZyaHgtMzR0VjNoUklDUkRZOXpDa0RscEJoRjQyVVFVZldWQVdCRnMiLCJ5IjoiOVZFNGpmX09rX282NHpiVFRsY3VOSmFqSG10NnY5VERWclUwQ2R2R1JEQSIsImNydiI6IlAtMjU2In19.eyJqdGkiOiJlMWozVl9iS2ljOC1MQUVCIiwiaHRtIjoiR0VUIiwiaHR1IjoiaHR0cHM6Ly9yZXNvdXJjZS5leGFtcGxlLm9yZy9wcm90ZWN0ZWRyZXNvdXJjZSIsImlhdCI6MTU2MjI2MjYxOH0.lNhmpAX1WwmpBvwhok4E74kWCiGBNdavjLAeevGy32H3dbF0Jbri69Nm2ukkwb-uyUI4AUg1JSskfWIyo4UCbQ

[acoburn]

I currently have this implemented as a finite state machine with the following states:
Unauthenticated, AwaitingDPoP, Authenticated and Error. All interactions start with Unauthenticated and once Error is reached, the session has to be restarted in order to proceed.

The websocket keywords are auth and dpop. For instance:

auth <bearer token>
dpop <dpop token>

Responses are either err or ack <keyword> (e.g. ack auth)

(So I'm not using the Authorization: DPoP or DPoP: prefixes)

This does work, but I don't like it.

One idea (currently unimplemented) is for the WebSocket server to act as an OIDC relying party, so before a JavaScript client establishes a connection, there is a login flow that generates a cookie-based session. That cookie is then automatically sent as part of the new WebSocket(url) browser interaction, and it is from that cookie that authN information is determined.

That sort of interaction vastly simplifies the WebSocket protocol. It also makes it possible to use DPoP as it was intended to be used -- for ordinary HTTP methods. DPoP doesn't make any sense in the context of WebSockets, since there's no method (htm) as part of the request, and the url (htu) is arguably not relevant.

[elf-pavlik]

Recent Google Chrome Developers video - Streaming requests with fetch - HTTP 203 provides some interesting updates. It even discusses having two fetches for bidirectional communication if needed (at 16:09). I think we should seriously evaluate streaming HTTP responses before committing to WebSockets.

[michielbdejong]

I'll align to use auth <bearer token> / dpop <dpop token> instead of auth-header.

[RubenVerborgh]

Also see https://github.com/solid/notifications-panel/issues/19

[csarven]

The data format is JSON-LD. The data model is defined by https://solid.github.io/notifications/protocol#notification-data-model (with proposed update as of this writing: #124 ) and further specifics on WebSocket in https://solid.github.io/notifications/websocket-subscription-2021