Digest auth not re-used after connection close.

[catsmanac]

Not sure if it is as intended, a failure or a new feature request, so starting as discussion for now.

We are using HTTPX client for communication with a server requiring digest auth on 1 endpoint. We observe connection failures during use. #1467 requested for re-use Digest and was implemented in 0.23.2.

What we observe

• When a get from the endpoint is requested, the Envoy returns a 401 with the digest information included.
• HTTPX then repeats the get request on the same connection with the digest now included.
• The server replies with TCP FIN,ACK to the message and closes the connection using RST.
• HTTPX signals a RemoteProtocolError.
• HTTPX repeats the get, new connection is build, but Digest is not included and same sequence of messages occurs. The nonce being incremented by the server.

The included capture file capture-pyenphase.pcapng contains 85 packets. The initial request for the endpoint is packet 19, performing a get on the existing connection which doesn't require the digest. That connection is then closed and a retry is in packet 25. Server replies with the 401 and digest in packet 27. Client repeats the get request in packet 29 and includes the digest.

Right after receiving the get the server closes the connection. The client reacts by opening a new connection and issuing the get request again. The Digest Auth is not included this time and the whole sequence repeats.

A similar test with curl, capture-curl.pcapng, shows same server behavior closing connection in packet 17. Curl however retries the get on a new connection in packet 21 including the Digest information received from the server.

capture-files.zip

The client uses httpx.Limits(keepalive_expiry=KEEP_ALIVE_TIMEOUT) with the timeout being 15 sec.

When utilizing limits=httpx.Limits(max_keepalive_connections=0) all is working fine and the issue does not occur.

The code employs httpx.DigestAuth and tenacity retry

httpx.DigestAuth(self.local_username, self.local_password)

@retry(
    retry=retry_if_exception_type(
        (
            httpx.NetworkError,
            httpx.TimeoutException,
            httpx.RemoteProtocolError,
            orjson.JSONDecodeError,
        )
    ),
    wait=wait_random_exponential(multiplier=2, max=5),
    stop=stop_after_delay(MAX_REQUEST_DELAY)
    | stop_after_attempt(MAX_REQUEST_ATTEMPTS),
    reraise=True,
)
async def request(

We can not change the server behavior as it is a commercial available device.

The question is if HTTPX behavior is as expected or if it should behave like CURL by re-using the Digest received when opening the new connection?