GZIP content length mismatch fix

[hiranp]

This addresses concerns raised by the following discussions:

• Gzip Middleware content-length is incorrect #803
• GZip Middleware Throws "Response content longer than Content-Length" when given a 304 empty-body response #4050
• Gzip middleware randomly raise RuntimeError but does not affect use , what are the possible causes?

We encountered this problem when developing our internal application, when we implemented both Brotli and GZIP compression. We tested the following fix using several use cases.

The solution tries to address the problem using two(2) approaches:

• Setting the respond body if NULL bytes are detected

• Not writing the message body when its empty or contains NULL byte char

• Initially raised as discussion Gzip Middleware content-length is incorrect #803

[Kludex]

Is this still an issue on fastapi/fastapi#4488 ?

[hiranp]

Also, I going to update my pull request, found another corner case. I am not handling.

[Kludex]

How can I reproduce the issue here?

[hiranp]

This issues seems to occur on a serving static files and this case from the linked issues from FastApi [4050]. However, I need to find time to translate my specific case into a smaller test case.

from typing import Awaitable, Callable

from fastapi import FastAPI
from fastapi.middleware.gzip import GZipMiddleware
from starlette.requests import Request

app = FastAPI()


@app.middleware("http")
async def noop_middleware(request: Request, call_next: Callable[[Request], Awaitable]):
    return await call_next(request)


@app.get("/", status_code=304)
def youve_seen_this_before():
    pass


app.add_middleware(GZipMiddleware)

[Kludex]

I'm able to reproduce this with:

from typing import Awaitable, Callable

from starlette.middleware.base import BaseHTTPMiddleware
from starlette.middleware.gzip import GZipMiddleware
from starlette.routing import Route
from starlette.middleware import Middleware
from starlette.applications import Starlette
from starlette.responses import Response
from starlette.requests import Request

class NoopMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next: Callable[[Request], Awaitable]):
        return await call_next(request)

def endpoint(request: Request):
    return Response(status_code=304)


app2 = Starlette(
    routes=[Route("/", endpoint=endpoint)],
    middleware=[Middleware(GZipMiddleware), Middleware(NoopMiddleware)]
)

This is more as a self note. I'll continue to check this later on.

[abersheeran]

This PR did solve the problem, but that # noqa made me confused. 🤔

[levrik]

These seem to be linting rules coming from https://wemake-python-stylegui.de/en/latest/
Probably @hiranp has this set up globally on his machine?

[Kludex]

This PR doesn't solve the issue for me...

I've tried with both examples above, and I still have the same issues.

[Kludex]

Thanks for the PR @hiranp !

After debugging this issue, it looks like it's more related to the BaseHTTPMiddleware than the GZipMiddleware.

I'll be closing this issue, and I'd really appreciate if you can take a look at #1609. 🙏