ENH: Series.str.split can return a DataFrame instead of Series of lists

[billletson]

closes #8428.

Adds a flag which when True returns a DataFrame with columns being the index of the lists generated by the string splitting operation. When False, it returns a 1D numpy array, as before. Defaults to false to not break compatibility.

In the case with no splits, returns a single column DataFrame rather than squashing to a Series.

[jreback]

looks good. pls add a release note in v0.15.1.txt

I am not sure I love the option to_df, @jorisvandenbossche @TomAugspurger ?

maybe as_frame? (side issue, its possible we want to deprecate this in 0.16.0 and change it to have as_frame=True by default) - or whatever it is called

[jorisvandenbossche]

"array with one dimension" -> "series" ? (it does return a series no?)

very minor: "Boolean" doesn't need a capital B (that is more consistent)

[jorisvandenbossche]

for the name, I would certainly use frame instead of df (in line with the to_frame method), but as_frame or to_frame are both good for me

edit: maybe as_frame sounds better!

[TomAugspurger]

👍 for as_frame.

Eventually changing the default to True (to return a frame) sounds good. I think you're saying start deprecating in 0.16 right? And make the change later?

[immerrr]

As a pure "what if" option, in DataFrame.to_dict(self, outtype) method, outtype is {'dict', 'list', 'series'}, so outtype={'list', 'frame'} might be better consistency-wise.

[jorisvandenbossche]

@immerrr But this outtype kwarg is recently deprecated in favour of orient (which was more in line with from_dict) ...
#7840, http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_dict.html

I don't know if there are other examples in pandas of similar things?

[jreback]

orient sounds good

[jorisvandenbossche]

ah, that is not what I meant :-)

How does the word 'orient' relate to the fact it is a series or expanded to dataframe?

[jreback]

@jorisvandenbossche that is true. ok, outtype (sematically correct here), the other deprecation was more for consistency.

[jorisvandenbossche]

in boxplot you have the return_type kwarg

[jreback]

ohh, I like that. @billletson want to change to that kw?

@TomAugspurger yes, the idea is to change this in 0.16.0 (or maybe just deprecate and change the default later)

[immerrr]

Speaking of making it the default, it makes a lot of sense to return frames by default when n is not None, i.e. when you have an upper boundary on number of created columns. When the number of splits is unknown in advance, it seems safer to return lists instead, or it may blow up on a single unfortunate value...

[billletson]

Revised the kw, added a release note, as well as a couple more test cases.

[jreback]

looks good to me, @jorisvandenbossche ?

@billletson can you also create a new issue to have the default changed to frame? pls reference this issue (i will mark it as 0.16.0)

[jorisvandenbossche]

yep, looks good. @billletson Thanks a lot!