API: formalize the pandas IO API

[jreback]

#15838 (comment)

we have fairly uniform IO routines of the form

.to_format(path, df, **kwargs) (takes DataFrame)
and
pd.read_format(path, **kwargs) (returns DataFrame)

so should document various aspects of this:

• contract on input path strings
• file-like objects & is_file_like (ENH: Add file buffer validation to I/O ops #15894)
• do we do any encoding / compression (only on csv/json ATM), compression
• various guarantees on what we are sending in (e.g. no Index, string columns which are non-duplicated), no non-string objects (see feather/parquet impl).
• make these more pluggable
• perhaps allow a specification for block access / chunking.
• additional args to accept/use: mode (for writing)

[h-vetinari]

This should IMO also extend to generic write methods pd.to_xxx. For example, as of v0.23.1, there's:

• pd.to_msgpack(path, *args, **kwargs), where *args is an arbitrary number of objects to pack,
• while pd.to_pickle(obj, path, ...) switches the order around (and it would be possible to pass an arbitrary number of objects that are then wrapped in a tuple),
• and the other pd.to_xxx do not exist at all.

It may well be that it makes no sense to have a generic method for all data types (like CSV), but those that are around should have a unified interface as well.

[TomAugspurger]

@jreback do you think there's anything here that's a blocker for 1.0?

[jreback]

nothing immediately obvious

likely need to audit all of the IO methods for conformity and show a matrix of what apis we have for each

[mroeschke]

I think this is a duplicate issue as #15008 as they are covering similar points. Going to close this issue in favor of that one