[CI] Use latest dask

[hcho3]

Dask tests are stuck and timing out due to dask/dask#5465: https://xgboost-ci.net/blue/organizations/jenkins/xgboost/detail/PR-4971/2/pipeline/40/#step-75-log-1677. The cause of the timeout is because one of dependencies of Dask, distributed, changed its semantics in recent versions in ways that are not compatible with Dask 2.0.0. Fix: Pin version of distributed.

[trivialfis]

I thought 2.5 is available now? Can we remove the version number and stick with the latest version?

[hcho3]

@trivialfis What is our commitment with Dask support? Is it sufficient to use the latest all the time, or should we accommodate users of older versions? (I'm assuming we'd need at least 2.0.0) In #4617, I updated XGBoost so as to be compatible with both 1.2.x and 2.0.0. Let us pick the oldest version of Dask we are willing to support.

cc @RAMitchell @mt-jones @mrocklin

[trivialfis]

No. Just keep running with the latest version. Dask is very light weight(aside from the fact that pandas is involved) and uses small releases. Also we don't use any internal API so if something breaks we can fix it in dask.

[hcho3]

@trivialfis When Dask 2.0 was released, it renamed an internal variable and broke XGBoost, so I had to submit #4617. I wonder whether something like this could happen again. We could choose the middle ground and only specify the major release of Dask, e.g. dask>=2.0.0,<3.0.0.

[trivialfis]

I wouldn't worry about compatibility with old dask. We haven't release 1.0 yet and our dask interface went through a complete rewrite during these few months. So I would very much like to start with everything clean.

…

________________________________ From: Philip Hyunsu Cho <notifications@github.com> Sent: Tuesday, October 22, 2019 4:27:05 PM To: dmlc/xgboost <xgboost@noreply.github.com> Cc: Jiaming Yuan <jm.yuan@outlook.com>; Mention <mention@noreply.github.com> Subject: Re: [dmlc/xgboost] [CI] Use correct version for 'distributed' dependency of 'dask' (#4973) @trivialfis<https://github.com/trivialfis> When Dask 2.0 was released, it renamed an internal variable and broke XGBoost, so I had to submit #4617<#4617>. I wonder whether something like this could happen again. We could choose the middle groun and only specify the major release of Dask, e.g. dask>=2.0.0,<3.0.0. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#4973?email_source=notifications&email_token=AD7YPKK2O3GECCB53RFR6PLQP22NTA5CNFSM4JDMBVP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB45WPY#issuecomment-544856895>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AD7YPKJQO62HB3QMGZXRB23QP22NTANCNFSM4JDMBVPQ>.

[hcho3]

Thanks for clarification.

[trivialfis]

Thanks!