Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ported job calculation to use coroutines for tasks #1827

Merged
merged 1 commit into from
Jul 31, 2018

Conversation

muhrin
Copy link
Contributor

@muhrin muhrin commented Jul 31, 2018

Fixes #1828

This way the event loop has a chance to schedule other operations while
a transport request is made i.e. if multiple job calculations all
request the same transport (a common scenario) then in the past they
would all run as one loop callback effectively blocking the loop (and
crucially pika heartbeats). Now they are schedule as seperate callbacks
thanks to yield, hopefully, allowing the loop to schedule other
necessary tasks in between.

This way the event loop has a chance to schedule other operations while
a transport request is made i.e. if multiple job calculations all
request the same transport (a common scenario) then in the past they
would all run as one loop callback effectively blocking the loop (and
crucially pika heartbeats).  Now they are schedule as seperate callbacks
thanks to yield, hopefully, allowing the loop to schedule other
necessary tasks in between.
@muhrin muhrin requested a review from sphuber July 31, 2018 11:00
@codecov-io
Copy link

Codecov Report

Merging #1827 into develop will increase coverage by 8.52%.
The diff coverage is 87.75%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1827      +/-   ##
===========================================
+ Coverage    58.16%   66.68%   +8.52%     
===========================================
  Files          317      317              
  Lines        32394    32409      +15     
===========================================
+ Hits         18843    21613    +2770     
+ Misses       13551    10796    -2755
Impacted Files Coverage Δ
aiida/transport/transport.py 62.2% <66.66%> (+0.36%) ⬆️
aiida/work/transports.py 90.56% <89.13%> (+45.56%) ⬆️
aiida/orm/querybuilder.py 74.16% <0%> (+0.24%) ⬆️
aiida/common/setup.py 74.63% <0%> (+0.41%) ⬆️
aiida/control/computer.py 80.9% <0%> (+0.9%) ⬆️
aiida/orm/implementation/general/node.py 78.5% <0%> (+0.95%) ⬆️
aiida/orm/implementation/general/group.py 60.82% <0%> (+1.03%) ⬆️
aiida/common/hashing.py 83.87% <0%> (+1.07%) ⬆️
aiida/orm/implementation/django/authinfo.py 82.85% <0%> (+1.42%) ⬆️
... and 42 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c552b1...2c40178. Read the comment docs.

Copy link
Contributor

@sphuber sphuber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge transport request granted

@sphuber sphuber merged commit ca2778f into aiidateam:develop Jul 31, 2018
@muhrin muhrin deleted the transport_yield branch September 3, 2018 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants