Allow sklearn grid search over parameters specified as kwargs

[RAMitchell]

No description provided.

[hcho3]

"Allow sklearn grid search over parameters specified as kwargs"

Can you post a short snippet to show how grid search could be done? I'd like to see an example and see how this patch helps.

[RAMitchell]

    from sklearn.model_selection import GridSearchCV
    from sklearn import datasets

    params = {'tree_method': 'hist'}
    clf = xgb.XGBClassifier(n_estimators=1, learning_rate=1.0,**params)
    assert clf.get_params()['tree_method'] == 'hist'
    # 'max_leaves' is not a default argument of XGBClassifier
    # Check we can still do grid search over this parameter
    search_params = {'max_leaves':range(2,5)}
    grid_cv = GridSearchCV(clf, search_params, cv=5)
    iris = datasets.load_iris()
    grid_cv.fit(iris.data, iris.target)

    # Expect unique results for each parameter value
    # This confirms sklearn is able to successfully update the parameter
    means = grid_cv.cv_results_['mean_test_score']
    assert len(means) == len(set(means))

I have added this example to the python tests. Before this patch the GridSearchCV class could not modify the parameters specified as kwargs. If you do a grid search on these parameters nothing will change at each iteration. This works with the patch.