[jvm-packages] Jenkins CI should build snapshot JARs

[hcho3]

The Jenkins CI (https://xgboost-ci.net) should build snapshot JARs for XGBoost4J and XGBoost4J-Spark.

• This simplifies our release process, since we just need to copy pre-built JARs.
• Adventurous users can download nightly builds, to use latest features and bug fixes.
• We can use Docker images with old Linux (CentOS 6) so that the JAR would use old version of GLIBC and LIBGOMP. I've seen many people try building XGBoost JAR themselves and see it fail on the cluster because the cluster has old GLIBC (and they don't have admin right to upgrade).

So in short, we should do what we are currently doing with the Python package: https://xgboost-ci.net/blue/organizations/jenkins/xgboost/detail/master/321/artifacts

@CodingCat @trams

[trams]

I am all for having nightly builds with some known retention policy.

As for how to build it I think our CentOS 6 Docker image is fine and we can just use it.

For more information one can read how Python community deals with exactly the same problem (distributing native code on linux) in this PEPs: https://www.python.org/dev/peps/pep-0513/ and https://www.python.org/dev/peps/pep-0571/

Note that manylinux2010 is build on top on CentOS 6. I would've suggested to move to use it for building python and jvm artifacts but it does not support CUDA in any way.

Let me know if you need any help with this

[hcho3]

@trams Take a look at https://github.com/dmlc/xgboost/blob/master/tests/ci_build/Dockerfile.jvm

[trams]

Yeap. I think I was not clear in my message.
I think we should use exactly the image you pointed out. It looks good

[hcho3]

@trams I'd like to get some help from @CodingCat about how to host nightly built Snapshots in a Maven repo, so that users can just add a single entry in their pom.xml.

[CodingCat]

we can publish to a maven repo with snapshot tagged in version number

the problem is where to host

• my personal github? <- I am not a fan to expose my credentials in jenkins job

• maven central? <- maybe, but we still need to setup correctly credentials with write permission in aws machines

[hcho3]

@CodingCat We can host it in a S3 bucket. Since the Jenkins workers are hosted in AWS, we can apply security policies to safely push to the S3 bucket. Is there any document I can read about setting up a Maven repository?

[hcho3]

https://tech.asimio.net/2018/06/27/Using-an-AWS-S3-Bucket-as-your-Maven-Repository.html

[hcho3]

We are creating our own Maven repository where you can find latest XGBoost4J JARs: #5533.