DataFrame/Series.tz_convert with copy=False modifies original data

[hendrics]

Hi. Not sure if it is a bug, or something which needs to be clarified.

Consider the code

s = pd.Series(np.arange(0,5), index=pd.date_range('20131027', periods=5, freq='1H', tz='Europe/Berlin'))
s.tz_convert("UTC", copy=False)

s index is still the same as before. If i do the same for frames.

d = pd.DataFrame(s)
d.tz_convert("UTC", copy=False)

This time index of d has changed. From the code it is not clear if DataFrame is doing the right thing either.

So is it a bug or is it just inconsistent, or is it an intention?

Update:

In [1]: s = pd.Series(np.arange(0,5), index=pd.date_range('20131027', periods=5, freq='1H', tz='Europe/Berlin'))
      s.tz_convert("UTC", copy=False)
Out[1]: 
2013-10-26 22:00:00+00:00    0
2013-10-26 23:00:00+00:00    1
2013-10-27 00:00:00+00:00    2
2013-10-27 01:00:00+00:00    3
2013-10-27 02:00:00+00:00    4
Freq: H, dtype: int32

In [2]: s
Out[2]: 
2013-10-27 00:00:00+02:00    0
2013-10-27 01:00:00+02:00    1
2013-10-27 02:00:00+02:00    2
2013-10-27 02:00:00+01:00    3
2013-10-27 03:00:00+01:00    4
Freq: H, dtype: int32

In [3]: d = pd.DataFrame(s)
      d.tz_convert("UTC", copy=False)
Out[3]: 
2013-10-26 22:00:00+00:00 0
2013-10-26 23:00:00+00:00 1
2013-10-27 00:00:00+00:00 2
2013-10-27 01:00:00+00:00 3
2013-10-27 02:00:00+00:00 4

In [214]: d
Out[214]: 
2013-10-26 22:00:00+00:00 0
2013-10-26 23:00:00+00:00 1
2013-10-27 00:00:00+00:00 2
2013-10-27 01:00:00+00:00 3
2013-10-27 02:00:00+00:00 4

[jreback]

looks ok in 0.13.1..

In [1]: pd.__version__
Out[1]: '0.13.1'

In [2]: s = pd.Series(np.arange(0,5), index=pd.date_range('20131027', periods=5, freq='1H', tz='Europe/Berlin'))

In [3]: s.tz_convert("UTC", copy=False)
Out[3]: 
2013-10-26 22:00:00+00:00    0
2013-10-26 23:00:00+00:00    1
2013-10-27 00:00:00+00:00    2
2013-10-27 01:00:00+00:00    3
2013-10-27 02:00:00+00:00    4
Freq: H, dtype: int64

In [4]: d = pd.DataFrame(s)

In [5]: d.tz_convert("UTC", copy=False)
Out[5]: 
                           0
2013-10-26 22:00:00+00:00  0
2013-10-26 23:00:00+00:00  1
2013-10-27 00:00:00+00:00  2
2013-10-27 01:00:00+00:00  3
2013-10-27 02:00:00+00:00  4

[5 rows x 1 columns]

[alexchamberlain]

If you inspect s. it hasn't changed under 0.13.1.

>>> s = pd.Series(np.arange(0,5), index=pd.date_range('20131027', periods=5, freq='1H', tz='Europe/Berlin'))
>>> s.tz_convert(None, copy=False)
2013-10-26 22:00:00    0
2013-10-26 23:00:00    1
2013-10-27 00:00:00    2
2013-10-27 01:00:00    3
2013-10-27 02:00:00    4
Freq: H, dtype: int32
>>> s
2013-10-27 00:00:00+02:00    0
2013-10-27 01:00:00+02:00    1
2013-10-27 02:00:00+02:00    2
2013-10-27 02:00:00+01:00    3
2013-10-27 03:00:00+01:00    4
Freq: H, dtype: int32
>>> 

[jreback]

why would you expect s to change? most pandas methods return a new object

the copy flag is just tries not to actually copy the index if it doesn't need to; in this case it does so its irrelevant

[hendrics]

DataFrame does change though. If you inspect d it will have a new index. It might be that index is part of the data in the DataFrame, it's just the behaviour is inconsistent.

Updated the comment above with the output.

[jreback]

ahh..ok....will mark as a bug...thanks for the report

[jreback]

@hendrics

pls run this again on master....I am pretty sure this is fixed (and if you want to add an explicty test for this, would be gr8)

see here: b1687b8

[mroeschke]

The Series case is actually wrong now as well.

In [13]: s = pd.Series(np.arange(0,5), index=pd.date_range('20131027', periods=5, freq='1H', tz='Europe/Berlin'))

In [14]: s
Out[14]:
2013-10-27 00:00:00+02:00    0
2013-10-27 01:00:00+02:00    1
2013-10-27 02:00:00+02:00    2
2013-10-27 02:00:00+01:00    3
2013-10-27 03:00:00+01:00    4
Freq: H, dtype: int64

In [15]: s.tz_convert('UTC', copy=False)
Out[15]:
2013-10-26 22:00:00+00:00    0
2013-10-26 23:00:00+00:00    1
2013-10-27 00:00:00+00:00    2
2013-10-27 01:00:00+00:00    3
2013-10-27 02:00:00+00:00    4
Freq: H, dtype: int64

In [16]: s
Out[16]:
2013-10-26 22:00:00+00:00    0
2013-10-26 23:00:00+00:00    1
2013-10-27 00:00:00+00:00    2
2013-10-27 01:00:00+00:00    3
2013-10-27 02:00:00+00:00    4
Freq: H, dtype: int64

In [17]: pd.__version__
Out[17]: '0.24.0.dev0+1505.gcb31b2b09.dirty'