Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quick fix for memory leak in CPU Hist. #5153

Merged
merged 2 commits into from
Dec 31, 2019

Conversation

trivialfis
Copy link
Member

@trivialfis trivialfis commented Dec 23, 2019

Closes #3579 .

The proper fix is putting quantiles in DMatrix, which we have already done for GPU Hist. See #5143 for related future roadmap. This PR is a band-aid for fixing the old issue as we want a fully working 1.0 release.

@trivialfis trivialfis requested a review from hcho3 December 23, 2019 14:24
@trivialfis
Copy link
Member Author

Looking back, this PR is only made possible by many previous refactorings.

@trivialfis
Copy link
Member Author

trivialfis commented Dec 23, 2019

@hcho3 @CodingCat Is there any particular bug for hist that's considered as critical? We @RAMitchell would like to make hist as the default algorithm. 1.0 seems to be a right opportunity.

@trivialfis trivialfis changed the title Quick fix for memory leak. [WIP] Quick fix for memory leak. Dec 23, 2019
@trivialfis trivialfis changed the title [WIP] Quick fix for memory leak. [WIP] Quick fix for memory leak in CPU Hist. Dec 24, 2019
@trivialfis trivialfis changed the title [WIP] Quick fix for memory leak in CPU Hist. Quick fix for memory leak in CPU Hist. Dec 24, 2019
@trivialfis
Copy link
Member Author

Ping @hcho3

@hcho3
Copy link
Collaborator

hcho3 commented Dec 30, 2019

@trivialfis I’m okay with making hist the default. We do need to make further improvements to hist, to make it faster and more memory efficient.

@trivialfis
Copy link
Member Author

trivialfis commented Dec 30, 2019

@hcho3 Well, improvement is endless unless there's a better algorithm succeeds hist (like what hist does to exact). But it would be the right time to make hist as the default now.

Are you Okay with this PR?

@hcho3
Copy link
Collaborator

hcho3 commented Dec 30, 2019

@trivialfis I was referring to future improvements.

@trivialfis
Copy link
Member Author

@hcho3 Got it. Will create a new PR for setting it as default.

Would you like to review this fix?

@trivialfis trivialfis merged commit 04db125 into dmlc:master Dec 31, 2019
@trivialfis trivialfis deleted the fix-memory-leak branch December 31, 2019 06:06
@lock lock bot locked as resolved and limited conversation to collaborators Apr 2, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

XGBoost - hist + learning_rate decay memory usage
2 participants