ENH: add ignore_index argument to DataFrame.explode / Series.explode

[erfannariman]

When we use DataFrame.explode right now, it will repeat the index for each element in the iterator. To keep it consistent with the methods like DataFrame.sort_values, DataFrame.append and pd.concat, we can add an argument ignore_index, which will reset the index.

df = pd.DataFrame({'id':range(0,30,10),
                   'values':[list('abc'), list('def'), list('ghi')]})
print(df)

   id     values
0   0  [a, b, c]
1  10  [d, e, f]
2  20  [g, h, i]

print(df.explode('values'))
   id values
0   0      a
0   0      b
0   0      c
1  10      d
1  10      e
1  10      f
2  20      g
2  20      h
2  20      i

Expected behaviour with addition of the argument:

df.explode('values', ignore_index=True)

   id values
0   0      a
1   0      b
2   0      c
3  10      d
4  10      e
5  10      f
6  20      g
7  20      h
8  20      i

[erfannariman]

If this change looks oké by one of the devs, I can submit a PR for this.

[erfannariman]

take

[TomAugspurger]

I think something like this was discussed when explode was originally implemented. @erfannariman can you go through the original pull request implementing explode and summarize the discussion on this point?

[erfannariman]

I looked at the following discussions and couldn't find anything about resetting the index:

1. ENH: (explode) Splitting a column content over multiple rows while duplicating other columns content to these rows #16538
2. New DataFrame feature: listify() and unlistify() #10511
3. ENH: Add Series method to explode a list-like column #27267
4. Explode on identical index causes duplication #28005

Not sure if I missed anything. @TomAugspurger

[TomAugspurger]

Thanks for checking.

…

On Mon, Jun 22, 2020 at 7:03 AM Erfan Nariman ***@***.***> wrote: I looked at the following discussions and couldn't find anything about resetting the index: 1. #16538 <#16538> 2. #10511 <#10511> 3. #27267 <#27267> Not sure if I missed anything. @TomAugspurger <https://github.com/TomAugspurger> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#34932 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAOIULGOMPJDVR5X7QKE3RX5CCHANCNFSM4OEOKHFA> .

[jreback]

it’s ok to add this argument (was added elsewhere after explode existed)

[WillAyd]

What's the upside of adding this as an argument instead of just calling reset_index?

[erfannariman]

Not sure if im in the position to comment on your question, but in terms of API design, isn't that in the line of other methods like DataFrame.append, DataFrame.sort_values, pd.concat? Or do you mean internally wise? @WillAyd

[WillAyd]

Ah OK makes sense since we do elsewhere

[jorisvandenbossche]

For the ignore_index, another example where this was added recently is drop_duplicates (#30405) and sort_values (#30402).
And another reason to add it is that it can be a bit more performant (avoid an additional copy as you would have with reset_index(drop=True)).

---

One aspect related to the index of the result that was briefly discussed in the original PR (#27267 (review)) is whether to add a level to the index with a "count", thus resulting in a MultiIndex (which could eg be useful if you want to do an unstack in a next step).

I personally think that could still be useful, and we could potentially think about combining that in a single keyword. However, since ignore_index is already used in other places, probably better to consider this separately.