Improve the fence mechanism performances #283
Conversation
dumbbell
force-pushed
the
improve-fence-mechanism-performance
branch
from
73f219c to
a705687
Compare
[Why] `ra:member_overview/2` is a very expensive call. [How] We just need the last index and the current term from the leader and `ra:key_metrics/2` provides this piece of information too. The difference is huge: in my benchmark, the query rate goes from 15 queries per second to 100k. This is in association with a related change in Ra; see rabbitmq/ra#462.
…ition1/3` [Why] There is no need to run the same checks as a user query. We can execute the query directly.
dumbbell
force-pushed
the
improve-fence-mechanism-performance
branch
from
a705687 to
04916a5
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #283 +/- ##
==========================================
+ Coverage 89.36% 89.55% +0.18%
==========================================
Files 21 21
Lines 3104 3122 +18
==========================================
+ Hits 2774 2796 +22
+ Misses 330 326 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
This pull request is unblocked by the release of Ra 2.13.6. |
... if it is unneeded. [Why] We do that extra query to ensure that previous async commands were handled by the local Ra server before we proceed with the query with a fence. This comes with a performance penalty of course. We don't need that extra query if the previous command or query made by the calling process was synchronous. [How] We now keep a flag in the calling process dictionary to indicate if the last command was synchonous or it was a query. The flag is cleared with an async command. When we have to perform a query with a fence, we look at this flag to determine if the extra query is needed.
dumbbell
force-pushed
the
improve-fence-mechanism-performance
branch
from
04916a5 to
4c25407
Compare
|
For the record, my local testing shows that consistent queries are about 100% slower than low latency queries. In other words, the throughput doubles with low latency queries compared to consistent ones. Before the whole work in #260 and this pull request, consistent queries were slower by a single digit percent. Edit: My message is unclear. To sum up:
|
|
I think consistent queries being slower than low latency is not very concerning for the sake of RabbitMQ: we only use consistent queries very rarely |
|
Indeed. Also sync writes and sync transactions now wait for the command to be applied locally before returning. Therefore, consistent queries should be rarely useful. |
This pull request is a collection of patches that improves the performances of the fence mechanism.
It uses
ra:key_metrics/2instead ofra:member_overview/2and avoids the preliminary query that we do to serialize events if it is not needed. See individual commits for more details.This pull request is blocked because it depends on a new release of Ra that contains rabbitmq/ra#461. Also, to fully fix the performance problem introduced with the fence mechanism, we also need that that future release of Ra contains rabbitmq/ra#462.