Documenting non-browser implementations of Web APIs

[adrianhopebailie]

We recently had a challenging day fighting with the fetch API in a non-browser runtime.

We have a proxy service that runs on nodejs (it’s actually using Remix so, in theory COULD run on a variety of runtimes including Deno and Cloudflare Workers so ideally the application code shouldn't need to know which runtime is being used).

All of these non-browser runtimes seem keen to adopt the Web APIs as their standard API. The most obvious example is a coalescing around fetch for HTTP requests.

In our case, the proxy makes a call to a backend that returns a response with some set-cookie headers and we need to add some of our own cookies to the response that goes back to the browser.

This wasn’t working as expected and two very smart engineers spent a whole day figuring out WTF was going on. As all good engineers do, they searched for the the API reference and as most engineers are likely to do, they landed on MDN.

In an ideal world the MDN docs would have helped them resolve the unexpected behaviour in 5 mins by providing some pointers on the inconsistent behaviour of non-browser implementations or adding some extra guidance on how to work with cookies.

But, since modifying response headers is not a common use case for browser engineers, the docs have no guidance on this.

Our code was extracting a Headers object from the fetch response and then trying to extract the set-cookie headers and add them to the other Headers object that was part of the response to the browser.

The obvious ways to do this (cloning the object, get() then append()) don’t behave as expected and debugging is challenging as the result is a valid header but not a valid cookie so you have to dump the raw responses and try to spot the issue.

The crux of the issue is that Headers.get() returns a string with all the cookies concatenated together as a comma-separated string (but cookies can contain commas and so parsing this get’s ugly if you don’t know what to expect).

e.g. If you’ve come from working with other HTTP APIs in server runtimes like node then you’d expect headers to be arrays of strings in the case where there are multiple values.

I assume there are no hints on MDN about this because in the browser, modifying response headers is not a common use case.

In the end, I dug deeper and looked at the Fetch spec and then found out this is a well-known issue:
whatwg/fetch#973

I think that what Domenic misses in his dismissal of non-browser implementations of the Web APIs is that for the majority of developers, the API docs they land on for the APIs is MDN, even if they are working with non-browser implementations.

So, assuming these non-browser implementations will continue to use the Web APIs as their standard and MDN Web Docs has become the de-facto standard developer documentation for these APIs, what should the strategy be for documenting non-browser implementation anomalies or special behaviour?

Also, should code examples include examples that are specific to non-browser runtimes?