Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more explict sharding methods #4396

Merged
merged 6 commits into from
Apr 30, 2019
Merged

more explict sharding methods #4396

merged 6 commits into from
Apr 30, 2019

Conversation

rongou
Copy link
Contributor

@rongou rongou commented Apr 23, 2019

  • Rename the Reshard method to Shard
  • Add a new Reshard method for sharding a vector that's already sharded

@canonizer @RAMitchell @sriramch

Copy link
Contributor

@sriramch sriramch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 lgtm...

@RAMitchell
Copy link
Member

Looks good to me although this is @canonizer's baby so I will defer to him. If I understand correctly this is not required for your external memory prediction PR in its current state because you are deleting and recreating SparsePage objects? It may be nice to merge this anyway.

src/common/host_device_vector.h Outdated Show resolved Hide resolved
src/common/host_device_vector.h Outdated Show resolved Hide resolved
src/common/host_device_vector.cu Outdated Show resolved Hide resolved
src/common/host_device_vector.h Outdated Show resolved Hide resolved
src/common/host_device_vector.cc Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
@hcho3
Copy link
Collaborator

hcho3 commented Apr 27, 2019

@rongou I recently revamped the CI in #4401 and #4403 to improve the reliability of the tests. Try re-basing against the latest master if you continue to experience test failure.

src/common/host_device_vector.cc Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
tests/cpp/common/test_host_device_vector.cu Outdated Show resolved Hide resolved
@rongou
Copy link
Contributor Author

rongou commented Apr 30, 2019

@canonizer @RAMitchell I think this is ready. Please take another look. Thanks!

@RAMitchell RAMitchell merged commit eaab364 into dmlc:master Apr 30, 2019
@rongou rongou deleted the reshard branch May 3, 2019 17:18
@lock lock bot locked as resolved and limited conversation to collaborators Aug 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants