[Python] Refactors scikit-learn API to allow a list of evaluation metrics

[giresg]

This PR refactors the code in LGBM's scikit-learn API to allow the user to pass a list of evaluation metrics to the parameter eval_metric in the fit method of LGBMModel and its subclasses.

The snippet bellow shows an example of the possible uses of this functionality:

import ...

def custom_recall_sklearn(y_true, y_pred):
    return 'recall', recall_score(y_true, y_pred > 0.5), True


def custom_precision_sklearn(y_true, y_pred):
    return 'precision', precision_score(y_true, y_pred > 0.5), True

parameters = {
    "objective": "binary",
    "metric": "binary_logloss",
}

model = lightgbm.LGBMClassifier(**parameters)

model.fit(
    X=train,
    y=train_label,
    eval_set=(validation, validation_label),
    eval_metric=[custom_recall_sklearn, custom_precision_sklearn, "auc"])

Where eval_metric receives a list that can combine custom and built-in evaluation functions. The output looks like this:

[1]	valid_0's auc: 0.951963	valid_0's binary_logloss: 0.620274	valid_0's recall: 0.963916	valid_0's precision: 0.932162
[2]	valid_0's auc: 0.972375	valid_0's binary_logloss: 0.558293	valid_0's recall: 0.971234	valid_0's precision: 0.93151
[3]	valid_0's auc: 0.975112	valid_0's binary_logloss: 0.507648	valid_0's recall: 0.967701	valid_0's precision: 0.935594
[4]	valid_0's auc: 0.975005	valid_0's binary_logloss: 0.463648	valid_0's recall: 0.96644	valid_0's precision: 0.938495
[5]	valid_0's auc: 0.975215	valid_0's binary_logloss: 0.426748	valid_0's recall: 0.966944	valid_0's precision: 0.939676

The user can monitor during training as many metrics as needed.

@StrikerRUS rightly mentioned in PR 3165 and in issue 2182 that it is possible to monitor multiple metrics already. This PR leverages that functionality to make it a bit more intuitive.

This PR is backwards compatible.

[guolinke]

@StrikerRUS can you help to review this PR?

[StrikerRUS]

@gramirezespinoza Thank you very much for your PR which proposes great API simplification! Please consider addressing some initial review comments below.

[giresg]

@StrikerRUS tghanks for your comments! I've included all your recommendations into the code. Also, please see my comment above about the unit-test I had to modify.

I also fixed a couple of linting errors. Nevertheless, I don't know why this action is failing:

GitHub Actions / r-package (macOS-latest, clang, R 3.6, cmake) (pull_request) 

Any idea?

[guolinke]

@gramirezespinoza you can retry to rebase to master branch, also cc @jameslamb .

[jameslamb]

oh sorry! didn't see this comment since I wasn't a reviewer.

@gramirezespinoza there's no way that failure is related to your code. If you can please merge in the changes from master, I expect CI will build successfully. Sorry for the inconvenience.

[giresg]

Rebased to latest master, thanks @guolinke and @jameslamb

[giresg]

The CI build failed. I had a look and the logs show this error:

812 URL 'http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html'
813 Name 'link2'
814 Parent URL file:///home/travis/build/microsoft/LightGBM/docs/_build/html/GPU-Performance.html, line 253, col 8
815 Real URL http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html
816 Check time 6.020 seconds
817 Result Error: ConnectionError: ('Connection aborted.', BadStatusLine('No status line received - the server has closed the connection',))

Any idea why is this failing?

[StrikerRUS]

@gramirezespinoza

Any idea why is this failing

Sorry for the inconvenience! This test fails often due to network issues. I re-run it.

[giresg]

@StrikerRUS no worries, thanks for re-running it!

[StrikerRUS]

@gramirezespinoza Thank you very much for addressing my previous comments! Looks good overall. Please consider addressing some new issues from one more review round below.

[giresg]

@StrikerRUS thanks again for the comments. I've implemented the changes requested and rebased to newest master.

[StrikerRUS]

@gramirezespinoza Many thanks for this PR! I believe it will simplify user experience significantly.

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.