Calculate base_score based on input labels.

[trivialfis]

#4321 .

This PR calculates the base_score from labels for l1 regression and saves it to the output model. Will follow up on other objectives as well.

• Configure model parameter for base_score.
• Add estimation function in objective.
• Change base_score to an array. At the moment, we use only 1 element, the change is to prepare for multi-class and multi-output once we can remove some legacy in the binary model.

Multi-target and multi-class are not yet supported due to the binary model parameter.

[trivialfis]

The old binary format and the gpu_id configuration are probably too difficult to workaround for this PR.

[trivialfis]

The old binary format and the gpu_id configuration are probably too difficult to workaround for this PR.

I workaround it by limiting the base_score to a single scalar for now.

[RAMitchell]

This configuration is very fragile.

[trivialfis]

I agree. That's why I really want to remove the old model format.

[trivialfis]

I removed the use of nan as the base score flag to avoid breaking changes in downstream libraries (like treelite). Instead, a new base_score_estimated variable is introduced, but the variable is not read during model load so we don't need to keep it stable.

[RAMitchell]

Better that it was before. I wonder if it can be more robust with weighted averaging. The MSE version will need to use a weighted average also. Small example:
Worker 0 labels: 0 0 0
Worker 1 labels: 1000
True median: 0
True median mean abs error: 250
Estimated median (current method): 500
Estimated median (current method) mean abs error: 500
Estimated median (weighted average): 250
Estimated median (weighted average) abs error: 375

[trivialfis]

Thank you for the suggestion, changed it to the weighted average. I have adapted your example into a Python test.