Support for custom headers for handshake

[Misiu]

Hi,

please consider adding ability to add custom headers for handshake.
In RFC6455 there one interesting point:

The request MAY include any other header fields, for example,
cookies [RFC6265] and/or authentication-related header fields
such as the |Authorization| header field [RFC2616], which are
processed according to documents that define them.

I've found an example how to add custom header to handshake: https://blog.heckel.xyz/2014/10/30/http-basic-auth-for-websocket-connections-with-undertow/ but this is for Java and unfortunately isn't possible in HTML5.

When searching over the net I found many places question about this option, for example:
sta/websocket-sharp#22
https://stackoverflow.com/questions/4361173/http-headers-in-websockets-client-api/4361358#4361358
aspnet/SignalR#888

For example in Python this is possible https://stackoverflow.com/questions/15381414/sending-custom-headers-in-websocket-handshake. Other languages also support this. Last place missing is the browser.

Please consider adding this into specification. Having this even as a draft would allow us to consider browser vendors to add support for it.

If this is incorrect place for adding request about specification please forgive me and please point me to right place.

[domenic]

We'd need implementer interest in this before moving forward, similar to whatwg/html#2177.

[Misiu]

Mozilla - https://support.mozilla.org/en-US/questions/1176810
Chromium - https://bugs.chromium.org/p/chromium/issues/detail?id=768328
EDGE - https://wpdev.uservoice.com/forums/257854-microsoft-edge-developer/suggestions/31600090-websocket-support-for-custom-headers-for-handsha

@domenic I'll add feature request for other browsers soon. Hopefully some of them will show some interest.

[tyoshino]

Thank you Misiu for filing the feature request and the bugs for each vendor.

Regarding complexity, for the web platform, this would require non-trivial amount of work at both spec and impl side.

To introduce custom header feature, we need to make it issue a CORS preflight. Though this is not an HTTP API, considering the original reason why we introduced the CORS preflight, we need it for WebSocket + custom headers.

On Chromium, we have separate stacks for WebSocket and XHR/fetch() (for which we have CORS logic). Some non-trivial refactoring and gluing is needed.

[ricea]

I am one of the authors and maintainers of Chrome's WebSocket implementation. I oppose this proposal.

Reasons:

• The WebSocket handshake security model hinges on exposing no more capabilities for request forgery than are already possible using an img tag. It is not possible to add arbitrary headers to the request using an img tag.
• Apart from the limited sub-protocol negotiation, the WebSocket handshake is not intended as a means to exchange metadata about the connection. Non-browser implementations may use it this way, but I would not recommend it. It is better to think of it as a low-level operation like a TCP/IP SYN packet.
• It would be possible to add this to the WebSocket protocol by using a CORS preflight. However, the utility provided simply doesn't justify the additional implementation, standardisation and maintenance cost.

[Misiu]

@tyoshino @ricea thank You guys for Your reply.
My main idea was to add ability to add Authentication header to WebSockets.

If You look at SignalR repo You will notice that many people are looking for a way to pass authentication via WebSocket - aspnet/SignalR#888 (comment)

Right now the only option to do this is to add token as part of query string.
To simplify this only Authorization header is required.

[ricea]

@Misiu In the specific case of WebSockets I think passing an authentication token in the URL is okay. The reason is that, unlike HTTP URLs, wss: URLs are never exposed to the user. They can't bookmark them or copy-and-paste them. This minimises the risk of accidental sharing. In addition, their appearance in other web APIs is minimal [1]. For example, they won't appear in history. This reduces the risk of leakage via JS APIs.

The best thing from a security perspective would probably be to perform authentication after the WebSocket is connected, but I realise it is undesirable from a resource-usage perspective to permit unauthorised connections to be established in the first place.

Given the portability difficulties with using cookies or http auth, a short-lived authentication token in the URL is probably your best option at the moment.

A specific note with respect to the Authorization header is that normally you'd need to be able to handle a 401 response to use it. But in the WebSocket API error responses are never exposed to the page for security reasons. So that wouldn't work very well.

[1] I actually can't think of any other APIs that expose ws: or wss: URLs at all. Certainly they don't appear in resource timing.

[digitalpacman]

I cannot use WebSocketSharp because it's missing the ability to customize authorization headers. I am trying to integrate with a WebSocket api that takes in OAuth2 bearer tokens.

[digitalpacman]

@ricea That's not the main reason why you don't put authentication values in URLs. It's because of server logs. Auth tokens should be treated with the same security concern as username and password combos. You don't put username and password combinations in urls, so neither should auth tokens. If you have a vulnerability and a user downloads your http logs, they then have access to virtually all of your accounts.

[digitalpacman]

@ricea sadly implementers don't get to make the decision of what is required to use an api. I am currently trying to use an api that REQUIRES oauth2 bearer tokens in the authorization header for server-to-server communication. They offer cookies for browser based security, but I do not have that luxury because I have no sessionids for the user. So all you're doing by saying "we should not do this" is saying "you have to look elsewhere to implement with this api". Which would simply just drive people away from using this implementation. You aren't "saving" anyone but not allowing this, just alienating users.

[maleta]

I encountered this issue looking for solution on my problem - how to remove sid parameter from URL that is set and used by websockets. Now I see @ricea suggestion of adding short living token to URL with explanation that "wss: URLs are never exposed to the user."

OWASP Zap report is suggesting sid param removal from URL, also this question on stackoverflow is refering that CloudFlare is also suggesting sid param removal from URL: https://stackoverflow.com/questions/42759556/how-to-remove-socket-io-sid-parameter-from-url

So my question is, @ricea, do you think that this reported issue is completely irrelevant because of fact that ws/wss urls are never exposed to users?

[ricea]

@maleta

So my question is, @ricea, do you think that this reported issue is completely irrelevant because of fact that ws/wss urls are never exposed to users?

Security measures need to be considered in terms of what threats they are intended to protect against and to what extent they are effective in mitigating those threats.

In the case of the Cloudflare vulnerability, placing the sid parameter in a header rather than in the URL would have provided no protection whatsoever.

Quoting https://bugs.chromium.org/p/project-zero/issues/detail?id=1139:

... we observed encryption keys, cookies, passwords, chunks of POST data and even HTTPS requests for other major cloudflare-hosted sites from other users.

This is the trouble with checklist security. If you removed the sid parameter from the URL, you could check that item off the list and feel that you'd improved your security. But in practice there's no improvement at all.

@digitalpacman If someone has access to your server logs, they can do much worse things than just steal credentials. There's usually all kinds of personally-identifiable information in there. Access needs to be tightly restricted just as with any other kind of user data.

[digitalpacman]

@ricea there is only personally-identifiable information in there if you aren't doing good practices. At most there should be an IP address, but you can opt to not record that either. Not putting SECURITY CREDENTIALS in the url is an extremely popular best practice. Infact, governing bodies for security will instantly fail you if you are passing them in the URL.

Please look at the OAuth2 spec for example and explanation of these vulnerabilities. There is a reason it REQUIRES them in the POST body.

[annevk]

@ricea is this perhaps something we can reconsider if https://wicg.github.io/origin-policy/ becomes a thing? That should nullify most of the CORS preflight cost.

The reason I'm somewhat sympathetic to the requests is because I learned that middleboxes basically ruin full duplex HTTP so WebSocket will likely stick around.

[ricea]

@annevk I think origin policy will probably be implemented in the browser in the same place as CORS, so it would still mean re-wiring the handshake to go down the same code path.

The WebSocket handshake started out as purely a mechanism to negotiate a WebSocket connection. Over time we've added HTTP features like cookies and authentication, but we've paid a high price in implementation complexity.

The "right" way to do WebSocket authentication is to do it at the application layer after the handshake completes. No-one asks how to put an oauth token in a TCP/IP SYN packet. But the reality is that we've ended up in a fuzzy middle-ground that is hard to explain.

[annevk]

@ricea given that we do add browser-supplied cookies and authentication, it's quite a reasonable request that we also enable headers, since much WebSocket server-side infrastructure probably depends on such information being in the handshake.

And at least as far as the specification is concerned the handshake shares many aspects with Fetch, to enable mixed content blocking, HSTS, CSP, etc. So from that perspective this is not that much of a stretch. (I wonder if Chrome's approach is shared by other browsers.)

[davidfowl]

@ricea

The WebSocket handshake security model hinges on exposing no more capabilities for request forgery than are already possible using an img tag. It is not possible to add arbitrary headers to the request using an img tag.

Can you elaborate here. I don't understand why the security model hinges on not allowing headers to be set in the client API? I'm not sure I understand this.

[annevk]

@davidfowl part of the same-origin policy is that we don't send "attacker-controlled" headers to cross-origin URLs. That's why if you want to do that you need to use CORS, which uses a CORS preflight to make an explicit check that the cross-origin URL is okay with the headers about to be transmitted.

The objection here, at least from the Chrome team, is that supporting a CORS preflight for WebSocket is too costly.

[davidfowl]

@annevk Thanks for that clarity!

So attacker controlled headers are seen as more dangerous than attacker controlled query string. Would it help if we restricted the types of headers that could be sent?

[annevk]

Yes, you can basically reach any arbitrary URL (whatever the query string or path), but you can only control headers to a very limited extent (and methods too, only HEAD/GET/POST).

The request headers we allow "attackers" to control are listed at https://fetch.spec.whatwg.org/#cors-safelisted-request-header, but note that we plan to lock that down some more in whatwg/fetch#736. For the kinds of use cases that people seem to have I don't think allowing these would be sufficient (or it would lead to hacks where you put authorization data in Accept-Language or some such).

[davidfowl]

@annevk The current design forces people to send what would typically go in the the Authorization header in the query string. Is that any less secure than allowing say the authorization header to bet set on the upgrade request? (even cross origin)

[annevk]

The concern is not with the security of the application making the request, it's with the security of the remote server. The remote server will have to be robust against arbitrary URLs, but it does not have to be robust against arbitrary headers, since it'll assume (due to the long history of the web and early establishment of the same-origin policy) that those cannot come from browsers.

[davidfowl]

I see, understand the push back now. So there needs to be a pre-flight request integrated into the flow and that's expensive in chrome because of the current implementation split between websockets and xhr/fetch. Is that a fair summary?

[annevk]

Excellent summary.

[davidfowl]

Should we get some of the other browser implementors to chime in? I can probably get somebody from the edge team to chime in here about the difficulty there. I'm not sure how to go about getting the other browser vendors interested enough to look at this issue (safari, firefox, anything else?)

[annevk]

Maybe @mcmanus for Firefox and @youennf for Safari? Feedback from Edge would be good.

[mcmanus]

I'm kinda surprised people aren't smuggling this through subprotocol - seems obvious enough.

architecturally speaking, a CORS preflight wouldn't be a big burden for firefox to implement.

otoh, designs that result in a lot of preflights suck so I'm not convinced we want to enable this rather than pushing it into the post websocket-handshake data.

[benaadams]

rather than pushing it into the post websocket-handshake data.

Then its a per web app custom protocol implementation; open to a slow loris type, slow auth issue from an unknown user; depending on how the application developer implements it; vs a well understood header handling implementation by the webserver?

[ricea]

I haven't found any WebSocket servers that support CORS preflights.

You can test whether your WebSocket server can handle CORS preflights with a command like:

curl -i '<url>' \
  -X 'OPTIONS' \
  -H 'accept: */*' \
  -H 'access-control-request-headers: authorization, upgrade' \
  -H 'access-control-request-method: GET' \
  -H 'origin: <origin>'

Replace <url> with the URL of a WebSocket endpoint on your server, with ws changed to http or wss changed to https. Replace <origin> with an origin your server permits access from.

If WebSocket preflights would work on the server, it will respond with a 200 or 204 status, and include at least the following response headers:

Access-Control-Allow-Origin: <origin>
Access-Control-Allow-Methods: GET, OPTIONS
Access-Control-Allow-Headers: authorization, upgrade
Access-Control-Allow-Credentials: true

If you think it would be useful, you can report your results back here in the following format:

pywebsocket3: fails (status code 501)
cloudflare: fails (no -allow- headers)

[annevk]

Why would any WebSocket server support that today though?

[ricea]

Why would any WebSocket server support that today though?

It's common for HTTP servers to have WebSocket functionality, so it's not outrageous to think they might already support preflights.

If there are environments where it would actually work, that would be an argument in favour of supporting custom headers. Conversely, if none of the people who want custom headers are actually going to be able to use it, it's not worth spending time on.

[annevk]

That presumes that if browsers add support for WebSocket + CORS, WebSocket servers will not want to update. I don't see how that is a reasonable assumption.

[benaadams]

That presumes that if browsers add support for WebSocket + CORS, WebSocket servers will not want to update. I don't see how that is a reasonable assumption.

Preflight would be on the HTTP(s) UPGRADE request; the first request of a WebSocket is regular HTTP (and so same origin rules could still apply, or different origin for CORS)

[JokerQyou]

I don't see why it's necessary to find WebSocket servers that currently support CORS preflights. It's quite easy to write a websocket server that adds CORS preflight logic:

g := gin.Default()
g.Use(func(c *gin.Context) {
	c.Writer.Header().Set("Access-Control-Allow-Origin", "*")
	c.Writer.Header().Set("Access-Control-Allow-Credentials", "true")
	c.Writer.Header().Set("Access-Control-Allow-Headers", "Authorization, Upgrade")
	c.Writer.Header().Set("Access-Control-Allow-Methods", "OPTIONS, GET")

	if c.Request.Method == "OPTIONS" {
		c.AbortWithStatus(204)
	} else {
		c.Next()
	}
})
g.GET("/ws", func(c *gin.Context){
	conn, _ := websocket.Upgrader{
		ReadBufferSize: 1024,
		WriteBufferSize: 1024,
	}.Upgrade(c.Writer, c.Request, nil)
	conn.SetPingHandler(
		// Handle ping
	)
	// Other business logic here
})

[ricea]

A weird thing I noticed is that you could have a preflight over HTTP/1.1 and then have the real WebSocket connection go over HTTP/2. HTTP/2 uses CONNECT instead of GET for the upgrade method. It seemed like a good idea at the time.

I think the solution is for servers to send

Access-Control-Allow-Methods: CONNECT, GET, OPTIONS

even over HTTP/1.1 where it makes no sense, just in case the real connection happens to be on a different version of HTTP.

Should user agents enforce this? Or should they assume that the server probably intended to allow CONNECT? The latter options seems risky, but we don't want a situation where CORS checks fail 0.1% of the time.

[ricea]

I don't see why it's necessary to find WebSocket servers that currently support CORS preflights.

Not everyone has the ability to modify their WebSocket server implementation. Even if they request an update from the vendor, there may be a very long time before it is usable in production.

[annevk]

That's an interesting case. What would Access-Control-Request-Method contain? I think if we wanted to be absolutely certain we would require an additional header as WebSocket opt-in, e.g., Access-Control-Allow-WebSocket. And only if that is set would we proceed with the cross-origin WebSocket handshake. That would allow us to not care about CONNECT (we would consider it safelisted much like GET is, but only for the purposes of the WebSocket handshake).

[ricea]

Access-Control-Allow-WebSocket sounds good to me. If we're going to require a custom preflight anyway, it might as well be explicit. Maybe pair it with Access-Control-Request-WebSocket in the request side?

[annevk]

Yeah that seems reasonable. And maybe we wouldn't output Access-Control-Request-Method then as it can be confusing given the difference across HTTP versions.

[sergioisidoro]

A quick note to people that encounter this thread that token smuggling in the Sec-WebSocket-Protocol header seems to work on Firefox, but non in Chrome at the time of writing. Also some API tools will reject the connection with the message Server sent no subprotocol as a consequence of abusing the protocol header.

I reiterate the multiple voices here that said that adding a token in the URL goes completely against best practices (mainly because of logs, which include not only the servers in the service but ingresses and proxy services).

It feels like by being strict about the risks in the handshake ( #16 (comment) ) might be leading to thousands of people engaging in less safe practices, resulting in a net negative outcome for security of web services.

Not allowing custom headers is especially hard for me to to understand given that the RFC6455 clearly mentions additional headers, and given how many other frameworks already allow it.

[minrk]

@sergioisidoro thanks for that note! FWIW, I'm looking at this right now, and the Sec-WebSocket-Protocol approach can indeed work in Chrome. Some clients (including Chrome) require that the response specify a selected protocol that is one of the requested subprotocols, so if the server doesn't respond with a matching subprotocol, the connection is dropped (this is described in the spec, but I think not required? Most implementations don't enforce this in practice). It should work in general as long as the server responds with a subprotocol that is in the requested subprotocol list. If you weren't already using subprotocols, one way is to request two subprotocols (also following Kubernetes' example), one with the token, and the other either describing your protocol in general, or just the fact that you're using this token-subprotocol scheme.

So if request has:

Sec-WebSocket-Protocol: v1.token-nogotiated.example.org, v1.token.example.org.the-actual-token

and the response comes back with

Sec-WebSocket-Protocol: v1.token-negotiated.example.org

Chrome accepts the connection because v1.token-negotiated.example.org is one of the requested subprotocols.

[silverwind]

I would suggest to spec the alternate constructor signature from #16 (comment), e.g. similar treatment than addEventListener:

new WebSocket(url, protocols)
new WebSocket(url, {headers, protocols})

[dead-claudia]

I've got an alternate way to address this, by reusing much of the fetch API and adding a light extension to the streams API.

Readable streams would gain an extra argument to readableController.close() to send a close value, via readableController.close(value).

• This would be forwarded through readable.pipeThrough(transform) and such, and the {done: true, value: undefined} result from reader.read() would instead return this close value through its value property.
• Writable streams would receive this additional property through their close handler method, and they can themselves be .close()d with a value like readable streams.

To set up a WebSocket, you'd do response = await fetch(url, {mode: "websocket"}).

• The body you'd pass in the request is an object stream of strings and array buffers to send. Invoke senderController.close({code, message}) to close the WebSocket connection. TypeScript would likely model this as a ReadableStream<string | BufferSource>.
• response.body contains a similar object stream of received frames, if response.status === 101. On close, its readers return {done: true, value: {code, message}} from its .read() method, reflecting the above.
• response.type is set to "websocket" if it successfully connects.
• If the response is anything other than 101, it'll return a response identical to that of a normal fetch request. If it is 101, but failed to establish a WebSocket connection, it'll report an error.
• This otherwise would work just like fetch, including making requisite CORS requests, and all other options would work as usual (to the extent they apply - cache obviously will only impact CORS requests).

I looked in the spec, and much of the infrastructure appears to already exist. So this might be the path of least resistance.

[ricea]

I've got an alternate way to address this, by reusing much of the fetch API and adding a light extension to the streams API.

This is similar to the [WebSocketStream API]
(https://developer.mozilla.org/en-US/docs/Web/API/WebSocketStream).

Although it is attractive to re-use the RequestInit options for the WebSocketStream constructor, many combinations of options aren't really defined for WebSockets and it would take a lot of analysis before we could feel confident enabling them all.

[randomstuff]

I've got an alternate way to address this, by reusing much of the fetch API and adding a light extension to the streams API.

One benefit of this is that it makes it possible to receive info from failed WebSocket handshake requests.

If the WebSocket constructor is modified in order to make it possible to pass a custom HTTP headers (such as Authorization and DPoP), someone will very quickly complain that it is not possible ask to receive the WWW-Authenticate header from failed requests in order to know:

• which authentication schemes are available;
• which error happened;
• which DPoP algorithms and nonce must be used;
• which resource URI must be used.

Allowing passing Authorization is only half of the solution.

[dead-claudia]

Although it is attractive to re-use the RequestInit options for the WebSocketStream constructor, many combinations of options aren't really defined for WebSockets and it would take a lot of analysis before we could feel confident enabling them all.

@ricea That's fair. And I myself have some general ergonomic hesitations with this API, and almost stated that in the initial comment posting it. (The fetch API is just generally awkward for duplex request streams.)

I've got an alternate way to address this, by reusing much of the fetch API and adding a light extension to the streams API.

One benefit of this is that it makes it possible to receive info from failed WebSocket handshake requests.

@randomstuff That was part of my motivation for this, actually. It offers a standard API that can expose that info.