Change type for GradStats to avoid conversion for hist method

[ShvetsKS]

This PR is supposed to change internal type of GradStats to avoid conversion for hist method. Also bug related to numerical instability in NeedReplace method.

Accuracy ~ the same as was with double type, and ex. for mnist data set was even improved:

mnist	log-loss
Master	0.07304085
This PR	0.07301824

Similar changes were provided in PR:#4529 but full scope of changes was reverted in #5008.

Performance improvements (1.5x for BuildHist):

santander	full train	InitData	BuildHist	SyncHist	PredictRaw
Master	179.71	47.24	58.01	14	46.48
This PR	162.94	47.51	38.12	8.08	54.7

santander	full train	BuildHist	SyncHist	ApplySplit
Master	78.511	34.82	23.29	1.28
This PR	62.92	22.98	9.89	1.14

[trivialfis]

I don't think we can simply merge this PR. I did some experiments with Exact tree method with float32 on covertype data set (can be obtained from scikit-learn dataset module), the accuracy changed a lot.

One suggestion is to use GradientPair instead of GradStat, where you have the flexibility of defining underlying types.

You can check my implementation here: #5460 updater_exact.cc . The type is used very consistently (guaranteed no-casting by C++ type GradientT).

[trivialfis]

I will stop you from continuing the process. ;-) Please avoid copying code, use template instead if applicable. Also why not GradientPair?

[ShvetsKS]

GradientPair was used. Thanks :)

[ShvetsKS]

@trivialfis sorry, not clearly why I have error:
python-package/xgboost/core.py:1591: error (E1121, too-many-function-args, Booster.predict) Too many positional arguments for method call.

Local lint checks for changed files were successful.

[SmirnovEgorRu]

@ShvetsKS, @trivialfis,
My prev experience in Gradient Boosting dev in DAAL - single precision numbers lead to overflow for some corner cases with very large dims. In the same time it improves single-node perf for data sets with large amount of features + halves the communication cost for multi-node version.

Current XGB GPU impl has single_precision_histogram parameter, we can extend this for usage in CPU impl also. It will be consistent approach accross the library and we will eliminate the implementation-specific parameter. Keep it false by default.
What do you think?

[trivialfis]

Sounds good to me. On GPU the difference is very noticeable.

[SmirnovEgorRu]

@trivialfis, thank you.
@ShvetsKS, could you, please, rework the PR with proposed changes? Also, a update of the documentation is required, I suppose.

[RAMitchell]

Optional single precision support is a great idea.

[ShvetsKS]

@SmirnovEgorRu to be honest I expected the request of this changes :)
Work will be continued in #5624.