Support parquet or other binary format for external memory

[PeterPanonGit]

Currently XGBoost external memory only support libsvm and csv file as input file. However these types of text files are really slow to write and read when the data size is large. Is there a roadmap for xgboost to support more efficient data input format for external memory such as parquet, hdf5, etc?

[trivialfis]

I'm trying to move the external memory support to 3 party libraries by making DMatrix support iterators, an implementation for GPU is available https://github.com/dmlc/xgboost/blob/master/demo/guide-python/data_iterator.py . Others still need some more work.

[trivialfis]

Please check out the new iterator interface, with which you can load the data with any library you want.