asyncio http call times out

[rickcoup]

Long story short

We are making async call to the a site. It times out on it. With the network sniffer, it shows the site responses within 400ms.

Expected behaviour

Get the response back.

Actual behaviour

Throwing the error:

File "/opt/app-root/src/pppLib.py", line 407, in fetch\n async with session.post(url, data=pData, headers=head, timeout=timeout) as response:\n File "/opt/app-root/lib/python3.6/site-packages/aiohttp/client.py", line 1005, in aenter\n self._resp = await self._coro\n File "/opt/app-root/lib/python3.6/site-packages/aiohttp/client.py", line 575, in _request\n break\n File "/opt/app-root/lib/python3.6/site-packages/aiohttp/helpers.py", line 585, in exit\n raise asyncio.TimeoutError from None"}
 
I've checked the client.py code, haven't get much clue on it yet.

Steps to reproduce

The code is like this:

async def fetch(session, url):
    #async with session.get(url) as response:
    async with getattr(session,"get")(url) as response:
        return await response.text()

Your environment

redhat 3.9, python 3.6.3, aiohttp 3.5.4 and asyncio 3.4.3

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are #33 (HTTPS Support), #1745 (Call https website through proxy will cause error), #320 (HTTP/2 support), #858 (replace http parser?), and #6 (HTTP pipelining support improvements).

[rickcoup]

The proxy is not an issue. We got response back in normal situation. It happens under heavy number of requests only.

[odysseusmax]

i too get this error

Traceback (most recent call last):
   File "/app/downloads.py", line 127, in down_File
     async for chunk in r.content.iter_chunked(1024*512):
   File "/app/.heroku/python/lib/python3.7/site-packages/aiohttp/streams.py", line 40, in __anext__
     rv = await self.read_func()
   File "/app/.heroku/python/lib/python3.7/site-packages/aiohttp/streams.py", line 369, in read
    await self._wait('read')
   File "/app/.heroku/python/lib/python3.7/site-packages/aiohttp/streams.py", line 297, in _wait
     await waiter
   File "/app/.heroku/python/lib/python3.7/site-packages/aiohttp/helpers.py", line 585, in __exit__
     raise asyncio.TimeoutError from None
 concurrent.futures._base.TimeoutError

[KolbyML]

so what should you limit the requests to be if its "heavy requests"

[KolbyML]

ya as this issue only affects one object in my scheduler with the proxy i use but not others so 🤔

[Dreamsorcerer]

There's not enough information here to provide any help. It could be that you've set the timeout too low or created too many tasks. The total timeout includes the time it takes to handle everything, not just the network request itself.