Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix early stopping in the Python package #4638

Merged
merged 4 commits into from
Jul 7, 2019

Conversation

hcho3
Copy link
Collaborator

@hcho3 hcho3 commented Jul 4, 2019

Closes #4630
Closes #4421

@hcho3
Copy link
Collaborator Author

hcho3 commented Jul 4, 2019

In the future, I may want to overhaul how the evaluation metrics are passed from C++ to Python/R/Java. Currently, the C++ function XGBoosterEvalOneIter() is returning a string consisting of metric names and values all combined. This means that Python/R/Java bindings have to parse the returned string. It may be better to have the C++ function return a structured object instead (something like a key-value store).

@hcho3
Copy link
Collaborator Author

hcho3 commented Jul 4, 2019

@sriramch @rongou

@hcho3 hcho3 merged commit 1aaf4a6 into dmlc:master Jul 7, 2019
@hcho3 hcho3 deleted the fix_early_stopping_python branch July 7, 2019 08:01
thesuperzapper pushed a commit to thesuperzapper/xgboost that referenced this pull request Jul 8, 2019
* Fix dmlc#4630, dmlc#4421: Preserve correct ordering between metrics, and always use last metric for early stopping

* Clarify semantics of early stopping in presence of multiple valid sets and metrics

* Add a test

* Fix lint
@sriramch
Copy link
Contributor

sriramch commented Jul 8, 2019

Currently, the C++ function XGBoosterEvalOneIter() is returning a string consisting of metric names and values all combined. This means that Python/R/Java bindings have to parse the returned string. It may be better to have the C++ function return a structured object instead (something like a key-value store).

@hcho3 are there plans to use protobuf or similar serialized data formats for things such as this? this may also have a nice side effect of supporting compatibility when the return data evolves, as opposed to a binary structs.

@trivialfis
Copy link
Member

@sriramch There is a RFC of using JSON for everything. But currently I'm not sure about its priority.

sperlingxx pushed a commit to alipay/ant-xgboost that referenced this pull request Jul 17, 2019
* Fix dmlc#4630, dmlc#4421: Preserve correct ordering between metrics, and always use last metric for early stopping

* Clarify semantics of early stopping in presence of multiple valid sets and metrics

* Add a test

* Fix lint
sperlingxx added a commit to alipay/ant-xgboost that referenced this pull request Jul 17, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 6, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
4 participants