Subsample parameter - Federated XGBoost (Histogram Based mainly)

[WilliamLindskog]

Hi there!

How does the subsample parameter work for a federated XGBoost? I mainly been using the histogram-based one.

To my understanding, subsample is based on Minimal Variance Sampling. Does that mean the each model on client-side samples based local variance reduction or global?

Best regards,
W

[YuanTingHsieh]

@ZiyueXu77 can you help answer this question? thanks in advance!

[ZiyueXu77]

Histogram-based would be identical to distributed version, so unlike tree-based where tree training only access local data, I believe it will be based on global info, @rongou can further comment.

[rongou]

Yes, we sum the gradients across all workers, so the sampling would be based on the global gradient information.

[WilliamLindskog]

Hi @rongou,

Thank you for your answer and sorry for delayed response. Does it make more sense to sample based on global gradient information and not local? I am thinking whether sampling based on local gradient information could be used for adaptation or personalization? I might be wrong here though

[rongou]

You can give it a try, but I suspect it'll just add more noise.