aiohttp client is slow when large number of parallel async requests are launched compared to other http clients using threading

[kaustubh-d]

aiohttp client is slow when large number of parallel async requests are launched compared to other http clients using threading.

Sample code to reproduce:

• aiohttp server https://github.com/kaustubh-d/aio-tests-py/blob/main/aiohttp/aiohttp-server.py
• aiohttp based client https://github.com/kaustubh-d/aio-tests-py/blob/main/aiohttp/aiohttp-client.py
  code snippet

async def async_put_api(session):
    resp = await session.put("http://localhost/hello", data="Hello world!")
...snip
connector = aiohttp.TCPConnector(limit=100, limit_per_host=100)
client_session = aiohttp.ClientSession(connector=connector)
for i in range(100):
        task = asyncio.ensure_future(async_put_api(client_session))
        task_list.append(task)
responses = await asyncio.gather(*task_list)

• requests library based client https://github.com/kaustubh-d/aio-tests-py/blob/main/requests/requests-threading.py
  code snippet

def thread_func_put_api(work_item):
    resp = work_item.session.put("http://localhost/hello", data="Hello World!!")
...snip

client_session = requests.Session()
# Start threads to upload objects.
request_threads = []
work_items = []
for i in range(100):
    work = WorkItem(client_session)
    work_items.append(work)

    t = threading.Thread(
        target=thread_func_put_api, args=(work,))
    request_threads.append(t)
    t.start()
...snip

# join and wait for all threads to complete.

• http client based https://github.com/kaustubh-d/aio-tests-py/blob/main/httpclient/httpclient-threading.py
  code snippet

def thread_func_put_api(work_item):
    conn = http.client.HTTPConnection("localhost")
    conn.request("PUT", work_item.uri, "Hello World!")
    response = conn.getresponse()
...snip
threading code same as earlier for requests module.

Test run report example- https://github.com/kaustubh-d/aio-tests-py/blob/main/reports/report-1.md

Note
The test server is aiohttp based, so server seems more capable than aiohttp client path.

1. Is there anything am missing that can help aiohttp based client perform better than other http clients using threading?
2. Could this be a improvement/bug where aiohttp client path needs performance improvement?

[Dreamsorcerer]

https://github.com/kaustubh-d/aio-tests-py/blob/main/aiohttp/aiohttp-client.py#L57
Shouldn't total_time_ms be the time taken for asyncio.gather() to complete, rather than the sum of values from individual requests?

When the client code awaits, it could end up running through every other task before returning to complete that original task.
Whereas the threaded code is likely to complete as soon as the request comes in, thus giving you a smaller time, even if the total runtime is longer.
i.e. It's possible that while it takes 65ms for a task to complete, they may all be completed then and it actually only takes 70ms to run everything, while the requests approach might be taking 4ms to complete a task but 100+ms to run everything.

[kaustubh-d]

Thank you @Dreamsorcerer It's exactly the case you mentioned, requests are launched after gather is called and each request time involved event steps from other requests. By fixing timing calculation, I see that avg response times are better.

Parallel Requests:
Params: total_count 100, max_conn 100
Total time for 100 requests = 103 ms.
Avg time per request = 1.03 ms.

aiohttp does performs better than other modules when we scale the number of requests.
https://github.com/kaustubh-d/aio-tests-py/blob/main/reports/report-2.md

Thanks for your help!!

[Dreamsorcerer]

Maybe aiohttp performs better, but it looks like you've made the same mistake on the other frameworks. You should really measure them all in the same manner.

Though, the biggest benefit of asyncio is actually avoiding the dangers of writing multi-threaded applications, which is far more complex and very easy to introduce bugs in subtle ways that may be impossible for a mere mortal to debug.

[kaustubh-d]

Yeah had realised that and fixed other tests, still aiohttp is better. Have pushed updated code and reports.
https://github.com/kaustubh-d/aio-tests-py/blob/main/reports/report-2.md

Completely agree on dangers of multithreading :). My first experience at async in python and its interesting and has very convenient interface.

Again many thanks for the guidance!