Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gradient based sampling for external memory mode on GPU #5093

Merged
merged 70 commits into from
Feb 4, 2020
Merged

Gradient based sampling for external memory mode on GPU #5093

merged 70 commits into from
Feb 4, 2020

Commits on Dec 6, 2019

  1. add skeleton gradient-based sampler

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    51b50da View commit details
    Browse the repository at this point in the history

  2. change gpu_hist to use sampling in external memory mode

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    ebd67c1 View commit details
    Browse the repository at this point in the history

  3. add failing tests

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    a2c446c View commit details
    Browse the repository at this point in the history

  4. wip: poisson sampling

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    9e91871 View commit details
    Browse the repository at this point in the history

  5. sample and scale gradient pairs

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    ed322cc View commit details
    Browse the repository at this point in the history

  6. calculate max number of sample rows

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    9377014 View commit details
    Browse the repository at this point in the history

  7. add sampler constructor

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    f26415c View commit details
    Browse the repository at this point in the history

  8. collect all pages in memory if they fit

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    f2dd928 View commit details
    Browse the repository at this point in the history

  9. optimize finalize position

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    2e5494c View commit details
    Browse the repository at this point in the history

  10. done with sampling

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    7fca606 View commit details
    Browse the repository at this point in the history

  11. add some docs

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    f661df0 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    6513f8a View commit details
    Browse the repository at this point in the history

  13. formatting

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    14361af View commit details
    Browse the repository at this point in the history

  14. explicit constructor

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    70276f1 View commit details
    Browse the repository at this point in the history

  15. no need for gmock

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    d7770b4 View commit details
    Browse the repository at this point in the history

  16. test ellpackpage copy and compact

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    6a29c38 View commit details
    Browse the repository at this point in the history

  17. use subsample to control gradient sampler

    @rongou
    rongou committed Dec 6, 2019
    Configuration menu
    Copy the full SHA
    967ff16 View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2019

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Dec 9, 2019
    Configuration menu
    Copy the full SHA
    494b179 View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2019

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    ecd2419 View commit details
    Browse the repository at this point in the history

  2. implement sequential poisson sampling

    @rongou
    rongou committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    73ba5af View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2019

  1. fix compact bug

    @rongou
    rongou committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    d2f2f69 View commit details
    Browse the repository at this point in the history

  2. fix cpp test

    @rongou
    rongou committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    827c988 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    9350ec5 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Dec 11, 2019
    Configuration menu
    Copy the full SHA
    4482181 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2019

  1. finally working

    @rongou
    rongou committed Dec 12, 2019
    Configuration menu
    Copy the full SHA
    857c9c7 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2019

  1. add uniform sampling

    @rongou
    rongou committed Dec 13, 2019
    Configuration menu
    Copy the full SHA
    7de1620 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2019

  1. better estimate of sample rows

    @rongou
    rongou committed Dec 14, 2019
    Configuration menu
    Copy the full SHA
    d3a3dbf View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2019

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    780f5e6 View commit details
    Browse the repository at this point in the history

  2. more agressive memory allocation

    @rongou
    rongou committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    f728623 View commit details
    Browse the repository at this point in the history

  3. add some documentation

    @rongou
    rongou committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    c206880 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2019

  1. use mvs

    @rongou
    rongou committed Dec 20, 2019
    Configuration menu
    Copy the full SHA
    5652b3b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Dec 20, 2019
    Configuration menu
    Copy the full SHA
    8ba43fb View commit details
    Browse the repository at this point in the history

  3. fix windows

    @rongou
    rongou committed Dec 20, 2019
    Configuration menu
    Copy the full SHA
    a5b57b1 View commit details
    Browse the repository at this point in the history

  4. address review comments

    @rongou
    rongou committed Dec 20, 2019
    Configuration menu
    Copy the full SHA
    62a9ead View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2020

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    ca64413 View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2020

  1. add sampling method param

    @rongou
    rongou committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    2409ecb View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2020

  1. gradient-based sampling in in-memory mode

    @rongou
    rongou committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    3b16e66 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    0745686 View commit details
    Browse the repository at this point in the history

  3. fix clang tidy warning

    @rongou
    rongou committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    e41496b View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2020

  1. add tests for in-core

    @rongou
    rongou committed Jan 10, 2020
    Configuration menu
    Copy the full SHA
    fd458c6 View commit details
    Browse the repository at this point in the history

  2. remove unused code

    @rongou
    rongou committed Jan 10, 2020
    Configuration menu
    Copy the full SHA
    6e3a7fa View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2020

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 11, 2020
    Configuration menu
    Copy the full SHA
    cb608a5 View commit details
    Browse the repository at this point in the history

  2. relax tests

    @rongou
    rongou committed Jan 11, 2020
    Configuration menu
    Copy the full SHA
    4daffbf View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2020

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 13, 2020
    Configuration menu
    Copy the full SHA
    5dd01e7 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2020

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 14, 2020
    Configuration menu
    Copy the full SHA
    35ac5a7 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2020

  1. add test to verify sampling

    @rongou
    rongou committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    9aaa9ba View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    83fb791 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    0eccff9 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2020

  1. inverse probability weighting estimation

    @rongou
    rongou committed Jan 18, 2020
    Configuration menu
    Copy the full SHA
    10fce05 View commit details
    Browse the repository at this point in the history

Commits on Jan 21, 2020

  1. combine weight calculation and gpair scaling

    @rongou
    rongou committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    55bbe74 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    85d2ec9 View commit details
    Browse the repository at this point in the history

  3. fix tests

    @rongou
    rongou committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    be163b7 View commit details
    Browse the repository at this point in the history

  4. review feedback

    @rongou
    rongou committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    3a0fd99 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2020

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    c9eb5c9 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2020

  1. calculate threshold

    @rongou
    rongou committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    9680a69 View commit details
    Browse the repository at this point in the history

  2. tweak test

    @rongou
    rongou committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    c93f20d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    d684148 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2020

  1. more accurate threshold

    @rongou
    rongou committed Jan 24, 2020
    Configuration menu
    Copy the full SHA
    09864ed View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 24, 2020
    Configuration menu
    Copy the full SHA
    7675a9e View commit details
    Browse the repository at this point in the history

  3. tweak test tolerance

    @rongou
    rongou committed Jan 24, 2020
    Configuration menu
    Copy the full SHA
    f8b7dbf View commit details
    Browse the repository at this point in the history

  4. wip: refactor the code to disintangle sampling methods

    @rongou
    rongou committed Jan 24, 2020
    Configuration menu
    Copy the full SHA
    3aaae89 View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2020

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 27, 2020
    Configuration menu
    Copy the full SHA
    a83f13e View commit details
    Browse the repository at this point in the history

  2. done with refactoring

    @rongou
    rongou committed Jan 27, 2020
    Configuration menu
    Copy the full SHA
    5cbca76 View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2020

  1. fix tests

    @rongou
    rongou committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    7cf9110 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    71b21c6 View commit details
    Browse the repository at this point in the history

  3. release device memory

    @rongou
    rongou committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    3a734a9 View commit details
    Browse the repository at this point in the history

  4. remove scaling in uniform sampling

    @rongou
    rongou committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    55b36f2 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 28, 2020
    Configuration menu
    Copy the full SHA
    97ae33d View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2020

  1. Merge branch 'master' into gradient-based-sampler

    @rongou
    rongou committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    df96394 View commit details
    Browse the repository at this point in the history

  2. revert rabit

    @rongou
    rongou committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    7fd7c31 View commit details
    Browse the repository at this point in the history