Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't omit Content-Length header for Content-Length: 0 cases #1395

Merged
merged 7 commits into from
Jan 7, 2022

Conversation

Kludex
Copy link
Member

@Kludex Kludex commented Jan 6, 2022

This PR adds the content-length header by default, if not already included. For more information, please check on #1099 and the referred RFC.

Next step would be to add a conditional to check if status_code < 200 and status_code != (204, 304)`

Closes #1099

@@ -289,7 +289,7 @@ def set_stat_headers(self, stat_result: os.stat_result) -> None:
etag_base = str(stat_result.st_mtime) + "-" + str(stat_result.st_size)
etag = hashlib.md5(etag_base.encode()).hexdigest()

self.headers.setdefault("content-length", content_length)
self.headers["content-length"] = content_length
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are sure that content-length exists at this point, considering the change of conditional in the init_headers().

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be a bit careful about that assumption. Personally I'd start by reverting this bit, making sure we've got the different test cases I've described. The StreamingResponse and FileResponse won't yet pass, but let's look at how we should resolve them once we've identified the test failures fully.

@tomchristie
Copy link
Member

tomchristie commented Jan 6, 2022

Okay, so forgetting for a moment about 1xx, 204, 304, here's a sensible set of test cases that I can think of wrt. setting Content-Length...

  • Response() - Regular empty response. Content-Length: 0 (RedirectResponse is a good real-world example of this case)
  • Response(content='...') - Regular non-empty response. Content-Length: x
  • FileResponse(...) - File response, unknown length. Transfer-Encoding: chunked.
  • FileResponse(..., stat_result=...) - File response, known length. Content-Length: x
  • StreamingResponse(...) - Streaming response, unknown length. Transfer-Encoding: chunked
  • StreamingResponse(..., headers={"Content-Length": "123"}) - Streaming response, known length. Content-Length: x

For Transfer-Encoding: chunked cases it's also potentially okay to just omit the header, since I think the ASGI server is responsible for adding that, but I've not fully verified that.

I'd suggest the next step here is expanding the tests in this PR so that we're covering each case, and can see which ones need fixing up.

@@ -309,3 +324,8 @@ def test_head_method(test_client_factory):
client = test_client_factory(app)
response = client.head("/")
assert response.text == ""


def test_empty_response():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the test case that we should probably expand to cover the 6 different cases described. I guess we want these test cases?...

  • test_empty_response
  • test_non_empty_response
  • test_file_response_unknown_size
  • test_file_response_known_size
  • test_streaming_response_unknown_size
  • test_streaming_response_known_size

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When do we have a FileResponse with unknown size?

@tomchristie tomchristie changed the title Add content-length header by default Don't omit Content-Length header for Content-Length: 0 cases. Jan 7, 2022
@tomchristie
Copy link
Member

I've retitled this PR to be more representative of the change.

@Kludex
Copy link
Member Author

Kludex commented Jan 7, 2022

Expressing the thoughts I've shared on Gitter here as well:

I need to be sure that the response class is not StreamingResponse or FileResponse before adding the Content-Length: 0, because those four cases are already being handled now (file response unknown size, file response known size, streaming response known size, and streaming response unknown size).

I don't have (or I can't see) any attribute I can check to be sure it's not a "Transfer-Encoding: chunked" on the at the init_headers() point.

I've added a conditional on the Response.init_headers() to check if the response class was one of those two before adding the content-length: 0 headers... Can we do better?

@Kludex
Copy link
Member Author

Kludex commented Jan 7, 2022

Updated the solution to check if body is present on the response. If it's not, then it doesn't add the Content-Lenght: 0 headers.

Copy link
Member

@tomchristie tomchristie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! This is a really nicely targeted fix now. 👍

Possibly improvements still for the 1xx, 204, 304 cases, but we can consider those as a separate conversation.

@Kludex
Copy link
Member Author

Kludex commented Jan 7, 2022

For future reference, some discussions about this PR can be retrieved on Gitter: https://gitter.im/encode/community?at=61d6e204526fb77b3166a445

@Kludex Kludex changed the title Don't omit Content-Length header for Content-Length: 0 cases. Don't omit Content-Length header for Content-Length: 0 cases Jan 7, 2022
@Kludex
Copy link
Member Author

Kludex commented Jan 7, 2022

I've removed the period at the end to squash the commit.

Thanks @tomchristie :)

@Kludex Kludex merged commit 4633427 into master Jan 7, 2022
@Kludex Kludex deleted the add-content-length-by-default branch January 7, 2022 11:48
assert response.headers["content-length"] == "2"


def test_file_response_known_size(tmpdir, test_client_factory):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't realise at the time, but we could just name this one test_file_response.

Copy link
Member Author

@Kludex Kludex Jan 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one above already (that also checks the content-length being present), maybe it's just redundant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing content-length header field in some responses
2 participants