Unable to open Sentry with Safari, possibly related to Content-Encoding: gzip

[jap]

Self-Hosted Version

23.7.0

CPU Architecture

x86_64

Docker Version

24.0.2

Docker Compose Version

2.18.1

Steps to Reproduce

Go to the sentry main page with the latest Safari browser (16.5.1).

Expected Result

I was expecting a styled and working login page.

Actual Result

It doesn't work. Browser Console shows:

This appears to be related to app.js being transfer-encoded with gzip, but Safari not picking this up and trying to parse the gzipped data as javascript. This does work with Chrome for example.

Note that we're running Sentry behind Caddy, which may be interfering with headers.

Event ID

No response

[jap]

Note that this issue only popped up after upgrading to 23.7.0, previous versions had no issues.

[azaslavsky]

Is there any way to connect directly to the Sentry instance without Caddy in the middle, to see if the headers are still incorrect?

[jap]

Yes; I've just found a local instance that is directly accessible and has the same issue. Trying it out with Safari and using its "copy as curl" and executing that (with an additional -v and --output) gives:

curl from safari

$ curl -v --output app.js.gz 'http://192.168.121.33:8999/_static/dist/sentry/entrypoints/app.js' -X 'GET' -H 'Accept: */*' -H 'Pragma: no-cache' -H 'Cookie: sc=<redacted>; sentrysid=<redacted>' -H 'Referer: http://192.168.121.33:8999/auth/login/sentry/' -H 'Cache-Control: no-cache' -H 'Host: 192.168.121.33:8999' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.5.2 Safari/605.1.15' -H 'Accept-Language: en-GB,en;q=0.9' -H 'Accept-Encoding: gzip, deflate' -H 'Connection: keep-alive'
Note: Unnecessary use of -X or --request, GET is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 192.168.121.33:8999...
* Connected to 192.168.121.33 (192.168.121.33) port 8999 (#0)
> GET /_static/dist/sentry/entrypoints/app.js HTTP/1.1
> Host: 192.168.121.33:8999
> Accept: */*
> Pragma: no-cache
> Cookie: sc=<redacted>; sentrysid=<redacted>
> Referer: http://192.168.121.33:8999/auth/login/sentry/
> Cache-Control: no-cache
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.5.2 Safari/605.1.15
> Accept-Language: en-GB,en;q=0.9
> Accept-Encoding: gzip, deflate
> Connection: keep-alive
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx
< Date: Sun, 23 Jul 2023 12:15:45 GMT
< Content-Type: application/javascript
< Content-Length: 20121
< Connection: keep-alive
< Content-Disposition: inline; filename="app.js.gz"
< Last-Modified: Mon, 17 Jul 2023 22:09:09 GMT
< Content-Encoding: gzip
< Vary: Accept-Encoding
< Access-Control-Allow-Origin: *
< Cache-Control: max-age=0, must-revalidate
< X-Frame-Options: deny
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< 
{ [641 bytes data]
100 20121  100 20121    0     0   375k      0 --:--:-- --:--:-- --:--:--  436k
* Connection #0 to host 192.168.121.33 left intact
$ file app.js.gz 
app.js.gz: gzip compressed data, max compression, from Unix, original size modulo 2^32 61614

so it looks like it is serving the right thing (a gzip-ed file, with the correct Content-Encoding and Content-Type headers) - but Safari is choking on it.

Doing the same with Chrome gives:

curl from Chrome

$ curl -v --output app.js 'http://192.168.121.33:8999/_static/dist/sentry/entrypoints/app.js'   -H 'Accept: */*'   -H 'Accept-Language: en-US,en;q=0.9,nl;q=0.8'   -H 'Cache-Control: no-cache'   -H 'Connection: keep-alive'   -H 'Cookie: sc=<redacted>; sentrysid=<redacted>'   -H 'DNT: 1'   -H 'Pragma: no-cache'   -H 'Referer: http://192.168.121.33:8999/auth/login/sentry/'   -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36'   -H 'sec-gpc: 1'   --compressed   --insecure
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 192.168.121.33:8999...
* Connected to 192.168.121.33 (192.168.121.33) port 8999 (#0)
> GET /_static/dist/sentry/entrypoints/app.js HTTP/1.1
> Host: 192.168.121.33:8999
> Accept-Encoding: deflate, gzip
> Accept: */*
> Accept-Language: en-US,en;q=0.9,nl;q=0.8
> Cache-Control: no-cache
> Connection: keep-alive
> Cookie: sc=<redacted>; sentrysid=<redacted>
> DNT: 1
> Pragma: no-cache
> Referer: http://192.168.121.33:8999/auth/login/sentry/
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
> sec-gpc: 1
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx
< Date: Sun, 23 Jul 2023 12:19:02 GMT
< Content-Type: application/javascript
< Content-Length: 20121
< Connection: keep-alive
< Content-Disposition: inline; filename="app.js.gz"
< Last-Modified: Mon, 17 Jul 2023 22:09:09 GMT
< Content-Encoding: gzip
< Vary: Accept-Encoding
< Access-Control-Allow-Origin: *
< Cache-Control: max-age=0, must-revalidate
< X-Frame-Options: deny
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< 
{ [641 bytes data]
100 20121  100 20121    0     0   500k      0 --:--:-- --:--:-- --:--:--  614k
* Connection #0 to host 192.168.121.33 left intact
$ file app.js
app.js: ASCII text, with very long lines (61535)

Note that the Chrome-generated curl command has a --compressed in it, which causes it to decompress the data before outputting it, but the payload on the wire is the same.

Trying this with Firefox yields the same results as Chrome.

(Upon further inspection, sentry.css is also mangled in a similar way. Because the app.js resource fails to load, no other compressed resources are pulled in.)

[laurikari]

Also seeing this with the latest self-hosted version. It seems to me that the files may be gzip compressed twice.

[laurikari]

Upon further inspection, nope, the response is gzip compressed only once.

Safari trips up on the Content-Disposition response header, which has apparently appeared in the most recent Sentry version.

I'm serving Sentry via nginx, so adding this made the site work with Safari again:

  proxy_hide_header Content-Disposition;

[jap]

Thanks @laurikari , can confirm a similar workaround works with Caddy
(that is, adding:

header -Content-Disposition

to the caddyfile snippet for our sentry host.)

[azaslavsky]

Our 23.7.0 upgraded Django to 3.2.20, which is causing issues elsewhere as well. Based on my reading of this similar bug, it seems like this is caused by the Django issue as well. Will confer with the team and post a fix shortly.

[azaslavsky]

We've managed to replicate this on our dogfood self-hosted instance. Currently working on getting a fix up.

[azaslavsky]

So we weren't able to replicate the fix on our local instance (Safari verson 16.5). We tried both proxy_hide_header Content-Disposition; alone and with add_header Content-Disposition 'inline'; to no avail. Is there anything else you did to your nginx setup when resolving the bug? We could also go the route of explicitly adding the Content-Encoding on safari, but that feels like a bigger and riskier change, so I want to make sure there isn't a simpler solution you've seen.

[laurikari]

Hmm, adding proxy_hide_header Content-Disposition; was the only change I did to get this to work.

If I remove it, the issue reappears.

I did have to empty caches (or do a shift-reload) after changing the nginx config, otherwise Safari uses cached values for the CSS and JS files. Latest Safari on macOS arm64 (16.5.2 18615.2.9.11.10).

[azaslavsky]

Ah, I generally use Chrome for development, I misunderstood how caches are reset in Safari 🤦 . Looks good on my machine as well, pushing the fix now.

[azaslavsky]

For future reference, the upstream Django 3 bug where they say they won't fix it :/ https://code.djangoproject.com/ticket/31082

[azaslavsky]

This fix is currently being merged. It will be included in the 23.7.1 release I am currently preparing.

[soer7022]

I have tried deleting all cache, and reload even going as far as to restart my Mac. Still no luck...

[hubertdeng123]

Looks like this hacky workaround is creating issues:
#2377

[zKoz210]

@hubertdeng123

After the release of 23.9.1, this problem began to reproduce again. Can we bring this hack back or come up with other ways to make Safari work properly?

[soer7022]

It's been broken since mid july...

[frame]

If you use Apache as frontend proxy, you can workaround this issue by adding this to your vhost config:

Header unset Content-Disposition

(requires mod_headers)

[hubertdeng123]

Unfortunately, we don't have a good solution right now. This doesn't appear to be an issue specically with our code, but an issue with the way Safari handles the headers in Django 3. This hack cannot come back because it is a bigger issue that attachments are not being downloaded properly, which is a result of the workaround. You are free to use the hacky workaround in your own instance if you'd like.

[mwarkentin]

Looking at fixing this for our single tenant deployments, and it seems like we can fix it specifically for our static files without touching anything else like so:

location /_static/ {
      proxy_hide_header Content-Disposition;
    }

[yozshujar]

Sentry 23.10.0 same issue on safari 17

[himynameisjonas]

Sentry 23.10.0 same issue on safari 17

It works for me with 23.10.0 in Safari, both version 16 and 17. Might be a browser cache issue or something