net::ERR_SPDY_PROTOCOL_ERROR for static resources

[dwelch2344]

TL;DR: browser cuts off connection of (large-ish) static files with net::ERR_SPDY_PROTOCOL_ERROR error.

Details

Not sure the best way to reproduce this without sharing the containers, but we're seeing a scenario where our static resources are being cut off mid-transmission but only in Chrome + FF. Chrome will show a 200 success and headers sent back, but the stream never finishes and we get a dreaded net::ERR_SPDY_PROTOCOL_ERROR

The container itself seems to be running fine and the resources serve up locally just fine. curling the endpoint does show the data coming through. It only seems to affect larger static resources (like our React app's minified + compiled code)

Will work on a container to reproduce, but figured worth throwing out here to see if anyone has encountered the net::ERR_SPDY_PROTOCOL_ERROR before. Is there headers / connection settings / etc that might be tweaked?

System Information

Nginx:

• gcr.io/google_containers/nginx-ingress-controller:0.8.3
• gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11

Kubernetes:

Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.4", GitCommit:"793658f2d7ca7f064d2bdf606519f9fe1229c381", GitTreeState:"clean", BuildDate:"2017-08-17T17:03:51Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.2", GitCommit:"477efc3cbe6a7effca06bd1452fa356e2201e1ee", GitTreeState:"clean", BuildDate:"2017-04-19T20:22:08Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}```

[aledbf]

@dwelch2344 spdy is not available since 0.9-beta.3 because is not available in nginx since 1.9.5 (was replaced by http/2)
If you add the flag --v=5 it will put nginx in debug mode and you can get the cause of the error

[emasean]

We have same issue here, we saw something similar that enabling brotli could be the culprit according to #1958 but we are using version 1.17.1 which should have brotli disabled.

It only happen on one out of 2 deployment with identical nginx-ingress software revision where the identical file was served from the backend.

[emasean]

Well this was because there are a lot of TCP retransmission between the Nginx Ingress and the upstream backend server and we even have see TCP out of memory warning suggesting to increase tcp_mem, so we upgraded our Kubernetes version then the performance was back to normal.