-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker: Set RMQ consumer_timeout to undefined #6189
Docker: Set RMQ consumer_timeout to undefined #6189
Conversation
# 1000 hours in milliseconds | ||
echo "consumer_timeout=3600000000" >> "${RMQ_ETC_DIR}/rabbitmq.conf" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't it perhaps be better to add the advanced.confgi
approach instead, which allows us to simply set the timeout to undefined? I think the following should work
# 1000 hours in milliseconds | |
echo "consumer_timeout=3600000000" >> "${RMQ_ETC_DIR}/rabbitmq.conf" | |
cat > "${RMQ_ETC_DIR}/advanced.config" <<EOF | |
%% advanced.config | |
[ | |
{rabbit, [ | |
{consumer_timeout, undefined} | |
]} | |
]. | |
EOF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, cool, I didn't know this. Will give it a build locally and test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed and add test checking it is set, but better to run a calculation to test it really activated.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #6189 +/- ##
==========================================
+ Coverage 80.73% 80.73% +0.01%
==========================================
Files 548 548
Lines 40074 40074
==========================================
+ Hits 32349 32350 +1
+ Misses 7725 7724 -1 ☔ View full report in Codecov by Sentry. |
Hi @sphuber, in discourse you mentioned you can reproduce the issue correct? Can you try with |
afa7b3a
to
fdcc124
Compare
for more information, see https://pre-commit.ci
@unkcpz I baked this recipe locally so that I can run a container and try and debug why the build on Github actions was failing. I managed to build and start a container, but only as the normal user. When I try to |
Yes, I found the same that |
The failed test is on arm64 build, I can test it from my mackbook. |
I tested locally on my M1 Mac, and all passed. The failed test is the timeout of |
824f9b1
to
01f9036
Compare
Excellent, that worked, thanks. I verified that
so I am pretty sure it worked. |
As of RabbitMQ v3.8.15, a default `consumer_timeout` is set of 30 minutes. If a task is not acknowledged within this timelimit, the consumer of the task is considered dead and its tasks are rescheduled. This is problematic for AiiDA since tasks often take multiple hours even. The `consumer_timeout` can only be changed on through the server config. Here we disable it through the `advanced.config`. Cherry-pick: 33dffb0
Fixes https://aiida.discourse.group/t/workchain-except/152/36