-
-
Notifications
You must be signed in to change notification settings - Fork 839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Keepalive connections aren't released when closing the ConnectionPool #720
Comments
I don't think that's actually the behavior you'd expect (though, from your code that's the behavior one should expect): since requests are concurrent here, HTTPX will use as many connections as it can to perform the requests, i.e. up to 10 (the hard limit). Once one of the 10 first requests terminates, the connection can be reused by a subsequent request. So what you'd expect is to have 10 connections almost constantly being used, but you only get one, plus a pool timeout… Your code is similar in spirit to this: while True:
url = await get_next_url()
async with client:
response = await client.get(url) The What you need to do for the connection pool to be effectively used, is to move the async with client:
while True:
url = await get_next_url()
response = await client.get(url) With this I am able to run your program without issues against a local dummy server. But that doesn't explain the pool timeout… I am able to reproduce the timeout issue with this example: import asyncio
import httpx
async def main() -> None:
url = "http://localhost:8000"
max_connections = 2
client = httpx.AsyncClient(
pool_limits=httpx.PoolLimits(soft_limit=1, hard_limit=max_connections)
)
for _ in range(max_connections + 1):
async with client:
await client.get(url)
if __name__ == "__main__":
asyncio.run(main()) (Note that if we switch out I pinpointed the origin of the issue to us not releasing the internal |
Oh, I thought instantiating the client opened the connection pool, and you'd need to use "async with" to let the pool know you wanted a connection. Please close this issue if you feel like it. Thank you! |
@florimondmanca is this truly fixed? i'm running into something that seems similar, yet i'm using version that said, i'm fairly new to
|
Hello. I am having an issue where it looks like connections aren't being closed correctly, and after i reach a number of requests equivalent to "hard_limit" of pool_limits, i get a PoolTimeout exception.
I tried upgrading to httpx==0.10.1, with no success.
Minimal example:
I checked with netstat, and only one actual connection is opened to the IP address, so pooling seems to work fine.
I really cannot understand why. I even tried using the "aclose()" syntax, without the "async with" block, but no difference at all.
The text was updated successfully, but these errors were encountered: