Update performance plot

[andersfylling]

Hello! I find these coroutine projects very interesting, but I'm having some issues replicating the performance. I noticed that the graph used on the readme file is around 7 years old. Is it perhaps time to update it? =)

An idea would be to update it on major python releases.

• https://github.com/MagicStack/uvloop/blob/master/performance.png

[FeldrinH]

@andersfylling For reference, since the plot in the readme hasn't been updated (yet), could you post the performance numbers that you got?

[kPsarakis]

I had the same question when I looked at the figure, especially since it was made with Python 3.5, when asyncio was not mature yet.

I tried to update the benchmarking codebase to use the latest Python version (3.13) and uvloop==0.21.0.

My setup is AMD Ryzen 7 4800H, 64 GB RAM, Ubuntu 24.04.01, Linux 6.8.0-47. The benchmark Docker container in my updated version is python:3.13.0-bookworm

I used the default benchmark parameters and got these results:

uvloop still seems to be around 1.13x better than asyncio in the 1 and 10 KiB workloads. However, in the 100 KiB it seems to be worse. A result that is not even close to the 2-4x advertised in the README.

I really think that the figure should be updated from the maintainers to a more honest value.

A thing to note is that I am not an asyncio expert. There is a possibility that my modernizations to the benchmark broke something.

@fantix I would really like to hear your thoughts about this. Let me know if I can assist the project in any way. I have been a user of uvloop for the past 4 years.

[MarkusSintonen]

Hi!

We had to get rid of uvloop in our codebase due to many issues we faced with it in heavy load situations (eg this and this). It seems uvloop doesn't improve performance in newer Python versions e.g. in 3.12 we are using. We switched from uvloop to asyncio and observed no difference (we were using uvicorn with uvloop).

We also run synthetic benchmarks which showed basically zero difference. Finally in production (which processes over 10K RPS) it neither showed any difference when switching to vanilla asyncio.

I opened a small README to update the performance claims of the repo so they are less misleading: #644

[hidaris]

Hi!

We had to get rid of uvloop in our codebase due to many issues we faced with it in heavy load situations (eg this and this). It seems uvloop doesn't improve performance in newer Python versions e.g. in 3.12 we are using. We switched from uvloop to asyncio and observed no difference (we were using uvicorn with uvloop).

We also run synthetic benchmarks which showed basically zero difference. Finally in production (which processes over 10K RPS) it neither showed any difference when switching to vanilla asyncio.

I opened a small README to update the performance claims of the repo so they are less misleading: #644

I'm curious, in your scenario, are you still using httptools, or have you also switched to h11?

[MarkusSintonen]

I'm curious, in your scenario, are you still using httptools, or have you also switched to h11?

Still using httptools

[tarasko]

A few months ago I made a few PRs for uvloop that significantly boost uvloop performance, both plain and and ssl cases. Still waiting for someone to take a look at them.

I made performance measurements in the context of my project
https://github.com/tarasko/picows

The difference between uvloop and asyncio is currently marginal, but my PRs will definitely change that.

[hidaris]

The difference between uvloop and asyncio is currently marginal, but my PRs will definitely change that.

Will your PR make asyncio just as fast?

[tarasko]

The difference between uvloop and asyncio is currently marginal, but my PRs will definitely change that.

Will your PR make asyncio just as fast?

This one can indeed be ported to asyncio
#629

Others are about Cython and using C api

[andersfylling]

@tarasko: The difference between uvloop and asyncio is currently marginal, but my PRs will definitely change that.

Do you have benchmark numbers for that, or is it a gut feeling?

[tarasko]

@tarasko: The difference between uvloop and asyncio is currently marginal, but my PRs will definitely change that.

Do you have benchmark numbers for that, or is it a gut feeling?

I do benchmark for my websockets library.
In the benchmark a client sends a message and waits for a reply before sending the next one. I measure average request per second rate for different libraries. All clients connect to the same boost.beast websockets server through a loopback interface on my average notebook. Not a perfect environment for measurements but it give a good idea about performance anyway.

boost.beast client result is also included for reference

Python 3.12.5
asyncio
ssl [9509, 11743, 16155, 15937, 24604]
plain [9748, 18735, 29214, 29907, 37827]
names: websockets 14.1 | aiohttp 3.11.7 | picows python client | picows cython client | c++ boost.beast

uvloop 0.21.0 unmodified
ssl [9693, 12844, 16911, 16890, 24637]
plain [12853, 22052, 34290, 35701, 37999]
names: websockets 14.1 | aiohttp 3.11.7 | picows python client | picows cython client | c++ boost.beast

uvloop 0.21.0 with 4 PRs merged
ssl [8600, 15219, 20272, 20518, 26117]
plain [14964, 25528, 33878, 34280, 36208]
names: websockets 14.1 | aiohttp 3.11.7 | picows python client | picows cython client | c++ boost.beast

According to this benchmark, my PRs improve aiohttp/picows ssl websocket performance by roughly 18%-20% (aiohttp 15219/12844, picows 20272/16911) .

[1st1]

Thanks for running the benchmarks. I'll dedicate some time to uvloop in about a week to look through the open PRs and potentially updating the README.

[graingert]

@tarasko would you be interested in submitting these benchmarks to the faster cpython project? Eg python/pyperformance#227