[jvm-packages]support multiple validation datasets in Spark

[CodingCat]

• converge the current training/test split with multi validation datasets
• add unit test for multiple validation set
• add support for ranking training tasks
• add unit test for ranking training
• update tutorial
• fix early stopping feature

[CodingCat]

@yanboliang @weitian @superbobry anyone of you can review this?

[CodingCat]

it's how example run looks like after the change

[CodingCat]

find one more thing to fix, early stopping

[CodingCat]

@yanboliang take a further look?

[codecov-io]

Codecov Report

Merging #3910 into master will increase coverage by 0.18%.
The diff coverage is 72.02%.

@@             Coverage Diff              @@
##             master    #3910      +/-   ##
============================================
+ Coverage     56.23%   56.41%   +0.18%     
- Complexity      205      210       +5     
============================================
  Files           185      186       +1     
  Lines         14702    14818     +116     
  Branches        498      527      +29     
============================================
+ Hits           8267     8359      +92     
- Misses         6196     6202       +6     
- Partials        239      257      +18

Impacted Files	Coverage Δ	Complexity Δ
...lc/xgboost4j/scala/spark/params/CustomParams.scala	84.21% <ø> (+20.21%)	0 <0> (ø)	⬇️
.../xgboost4j/scala/example/spark/SparkTraining.scala	0% <0%> (ø)	0 <0> (ø)	⬇️
.../src/main/java/ml/dmlc/xgboost4j/java/XGBoost.java	85.33% <100%> (+0.29%)	46 <0> (+3)	⬆️
...boost4j/scala/spark/params/NonParamVariables.scala	100% <100%> (ø)	0 <0> (?)
...c/xgboost4j/scala/spark/params/GeneralParams.scala	66.66% <100%> (ø)	0 <0> (ø)	⬇️
...xgboost4j/scala/spark/XGBoostTrainingSummary.scala	35.71% <11.11%> (-27.93%)	2 <1> (ø)
...cala/ml/dmlc/xgboost4j/scala/spark/DataUtils.scala	42.1% <60%> (+19.88%)	0 <0> (ø)	⬇️
.../dmlc/xgboost4j/scala/spark/XGBoostRegressor.scala	63.9% <66.66%> (+1.52%)	18 <0> (+1)	⬆️
...dmlc/xgboost4j/scala/spark/XGBoostClassifier.scala	65.06% <77.77%> (+0.77%)	19 <0> (+1)	⬆️
.../scala/ml/dmlc/xgboost4j/scala/spark/XGBoost.scala	77.12% <79.52%> (-0.38%)	0 <0> (ø)
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c4ff50...49d578f. Read the comment docs.

[CodingCat]

@yanboliang ping?

[CodingCat]

@yanboliang ping

[yanboliang]

Looks good to me overall, thanks.

[CodingCat]

thanks @yanboliang

[linghaogu]

I am not sure if it is the right place to comment, let me know if it is not correct.

I am a user of xgboost and I am using 0.82 for our production training. One thing I think we can improve is that the return value of this function is better to be this.type
Because as a scala user I will write code like:

val xgb = new XGBoostClassifier(xgboostParam)
  .setFeaturesCol(MSDataSchema.FEATURE_VECTOR)
  .setLabelCol(MSDataSchema.RELEVANCE_LEVEL)
  .setEvalSets(Map("eval" -> testData))

xgb will be a None in this case

[CodingCat]

Oh, it’s a typo or autocomplete by ide, feel free to file a PR addressing this!

[linghaogu]

I will thank you!