Other file formats

[davidanthoff]

This is more a general question/remark: It would probably be nice if at some point this would also support other file format sources, right? We could think of an ExcelReaders.jl that uses the same interfaces, and then one could read from Excel files into any of the data structures supported by this package. So at some point it might make sense to pull the interface out of this package into a more general package. Or am I missing something?

[johnmyleswhite]

That might be a good idea. What I don't know is whether the interface for this package makes sense for other formats. Some questions are:

(1) Do you need to do type inference in Excel?
(2) Can you read an excel file incrementally row-by-row?

The current interface is designed to handle data that comes in a format very much like CSV files. I suspect a lot of other formats are like that, but I don't know Excel well enough to know if Excel is an example.

[davidanthoff]

Good questions... I did write a parser for the new Excel OpenXML format at some point, but its so long ago I only have vague memories. Generally these are zip files, and then in the zip files there are XML files with the content. The file format is really well documented, and I think it was originally set up in such a way that you can incrementally read in files, without ever having to have everything in memory. But I don't recall the details. I believe the older binary file formats are a competeley different beast.

[johnmyleswhite]

Ok. I think we should leave that work for another package then.