Document HSTS is still honored if allow_redirects=False

[benoitc]

I tried to disable the redirection with latest httpx but it is always redirection the URL:

>>> import httpx
>>> r = httpx.head("http://www.python.org",  allow_redirects=False)
>>> r.url
URL('https://www.python.org')

status code is also 200 and not 301 as expected.

[benoitc]

so it's probably happening because the domain is using HSTS:

httpx/httpx/_client.py

Lines 213 to 222 in 94323f9

	def merge_url(self, url: URLTypes) -> URL:
	"""
	Merge a URL argument together with any 'base_url' on the client,
	to create the URL used for the outgoing request.
	"""
	url = self.base_url.join(relative_url=url)
	if url.scheme == "http" and hstspreload.in_hsts_preload(url.host):
	port = None if url.port == 80 else url.port
	url = url.copy_with(scheme="https", port=port)
	return url

I will leave it open as I expected that allow_redirects=False would prevent that. At least the current behaviour which is corrent could be documented :)

[florimondmanca]

Thanks, just reproduced quickly locally and it does look like we're just honoring HSTS, but no HTTP redirection is being made. I think your suggestion to document this behavior makes sense, so I updated the issue title to reflect this…

[florimondmanca]

Here's the TRACE to confirm that no redirect was made, we just made the request using HTTPS in the first place:

$ HTTPX_LOG_LEVEL=trace python -m asyncio
>>> import httpx
>>> async with httpx.AsyncClient() as c:
...     r = await c.head('http://www.python.org', allow_redirects=False)
...
TRACE [2020-04-07 11:55:59] httpx._config - load_ssl_context verify=True cert=None trust_env=True http2=False
TRACE [2020-04-07 11:55:59] httpx._config - load_verify_locations cafile=/Users/florimond.manca/.pyenv/versions/3.8.1/lib/python3.8/site-packages/certifi/cacert.pem
TRACE [2020-04-07 11:55:59] httpx._dispatch.connection_pool - acquire_connection origin=Origin(scheme='https' host='www.python.org' port=443)
TRACE [2020-04-07 11:55:59] httpx._dispatch.connection_pool - new_connection connection=HTTPConnection(origin=Origin(scheme='https' host='www.python.org' port=443))
TRACE [2020-04-07 11:55:59] httpx._dispatch.connection - start_connect tcp host='www.python.org' port=443 timeout=Timeout(timeout=5.0)
TRACE [2020-04-07 11:55:59] httpx._dispatch.connection - connected http_version='HTTP/1.1'
TRACE [2020-04-07 11:55:59] httpx._dispatch.http11 - send_headers method='HEAD' target='/' headers=Headers({'host': 'www.python.org', 'user-agent': 'python-httpx/0.12.0', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive'})
TRACE [2020-04-07 11:55:59] httpx._dispatch.http11 - send_data data=Data(<0 bytes>)
TRACE [2020-04-07 11:55:59] httpx._dispatch.http11 - receive_event event=NEED_DATA
TRACE [2020-04-07 11:56:00] httpx._dispatch.http11 - receive_event event=Response(status_code=200, headers=[(b'connection', b'keep-alive'), (b'content-length', b'49040'), (b'server', b'nginx'), (b'content-type', b'text/html; charset=utf-8'), (b'x-frame-options', b'DENY'), (b'via', b'1.1 vegur'), (b'via', b'1.1 varnish'), (b'accept-ranges', b'bytes'), (b'date', b'Tue, 07 Apr 2020 09:56:00 GMT'), (b'via', b'1.1 varnish'), (b'age', b'1149'), (b'x-served-by', b'cache-bwi5131-BWI, cache-cdg20742-CDG'), (b'x-cache', b'HIT, HIT'), (b'x-cache-hits', b'1, 7'), (b'x-timer', b'S1586253360.041811,VS0,VE0'), (b'vary', b'Cookie'), (b'strict-transport-security', b'max-age=63072000; includeSubDomains')], http_version=b'1.1', reason=b'OK')
DEBUG [2020-04-07 11:56:00] httpx._client - HTTP Request: HEAD https://www.python.org "HTTP/1.1 200 OK"
TRACE [2020-04-07 11:56:00] httpx._dispatch.http11 - receive_event event=EndOfMessage(headers=[])
TRACE [2020-04-07 11:56:00] httpx._dispatch.http11 - response_closed our_state=DONE their_state=DONE
TRACE [2020-04-07 11:56:00] httpx._dispatch.connection_pool - release_connection connection=HTTPConnection(origin=Origin(scheme='https' host='www.python.org' port=443))
TRACE [2020-04-07 11:56:00] httpx._dispatch.connection - close_connection
TRACE [2020-04-07 11:56:00] httpx._dispatch.http11 - send_event event=ConnectionClosed()