Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use heuristic to select histogram node, avoid rabit call #4951

Merged
merged 1 commit into from
Oct 17, 2019

Conversation

RAMitchell
Copy link
Member

This PR considerably improves performance for the distributed GPU algorithm. We use the sum of Hessian in the left and right node to estimate which has fewer training instances instead of using rabit to sync and calculate this exactly. In particular it improves latency.

Performance numbers from training on a DGX-1 with https://github.com/NVIDIA/gbm-bench

dataset old new speedup
airline 68.4073 59.2706 1.15
bosch 20.12437 16.1119 1.25
covtype 81.049 59.8792 1.35
epsilon 306.2088 311.6957 0.98
fraud 8.3944 6.7949 1.24
higgs 34.4294 24.0335 1.43
year 25.8719 15.3342 1.69

I also removed the old HostAllReduce function, which is redundant now that we no longer manage multiple GPUs with threads.

@RAMitchell RAMitchell merged commit 60748b2 into dmlc:master Oct 17, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jan 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant