Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Worker key reference counting #3641

Closed
wants to merge 10 commits into from
Closed

Worker key reference counting #3641

wants to merge 10 commits into from

Commits on Mar 26, 2020

  1. Add worker key reference counting

    @jacobtomlinson
    jacobtomlinson committed Mar 26, 2020
    Configuration menu
    Copy the full SHA
    8edba46 View commit details
    Browse the repository at this point in the history

  2. Add test

    @jacobtomlinson
    jacobtomlinson committed Mar 26, 2020
    Configuration menu
    Copy the full SHA
    53691d2 View commit details
    Browse the repository at this point in the history

  3. Cast keys to list as it sometimes is a tuple

    @jacobtomlinson
    jacobtomlinson committed Mar 26, 2020
    Configuration menu
    Copy the full SHA
    a799dd5 View commit details
    Browse the repository at this point in the history

  4. Update test

    @jacobtomlinson
    jacobtomlinson committed Mar 26, 2020
    Configuration menu
    Copy the full SHA
    3b4b779 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2020

  1. Fix test

    @jacobtomlinson
    jacobtomlinson committed Mar 27, 2020
    Configuration menu
    Copy the full SHA
    6d6f44c View commit details
    Browse the repository at this point in the history

  2. Check both workers for value

    @jacobtomlinson
    jacobtomlinson committed Mar 27, 2020
    Configuration menu
    Copy the full SHA
    05b6069 View commit details
    Browse the repository at this point in the history

  3. Update test so that sum key is also the same both times

    @jacobtomlinson
    jacobtomlinson committed Mar 27, 2020
    Configuration menu
    Copy the full SHA
    3eaa30f View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2020

  1. Merge branch 'master' of github.com:dask/distributed into worker-refe… 

    …rence-counting
    @mrocklin
    mrocklin committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    e249c8a View commit details
    Browse the repository at this point in the history

  2. Create futures at the beginning of scatter rather than the end 

    Otherwise there is a period where the user has requested that data
exists, but we haven't incremented the reference counter yet.
This way we've stated that yes, these keys are important to us, even
while we wait for the data to transfer.  This stops the client from
sending an erroneous release message to the scheduler.
    @mrocklin
    mrocklin committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    6114d4b View commit details
    Browse the repository at this point in the history

  3. Improve logging on lost data

    @mrocklin
    mrocklin committed Apr 25, 2020
    Configuration menu
    Copy the full SHA
    e84ff97 View commit details
    Browse the repository at this point in the history