Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grouper using frequency has inconsistent behavior inside and outside a list #16746

Closed
prcastro opened this issue Jun 21, 2017 · 5 comments
Closed
Labels

Comments

@prcastro
Copy link
Contributor

Code Sample

>>> df.groupby(pd.Grouper(freq='H')).agg({'value': 'mean'})
            timestamp       delta_t
2016-06-07 00:00:00  2.729111e+11
2016-06-07 01:00:00           NaN
2016-06-07 02:00:00           NaN
2016-06-07 03:00:00           NaN
2016-06-07 04:00:00           NaN
...

>>> df.groupby([pd.Grouper(freq='H')]).agg({'value': 'mean'})
   timestamp       delta_t
2016-06-07  2.729111e+11
2016-07-07  2.509444e+11
2016-07-13  2.775778e+11
2016-07-15  2.490556e+11
2016-07-16  2.676190e+11

Problem description

This is an inconsistent behavior, and there is no mention to this on groupby or Grouper documentation.

Expected Output

>>> df.groupby([pd.Grouper(freq='H')]).agg({'value': 'mean'})
   timestamp       delta_t
2016-06-07 00:00:00  2.729111e+11
2016-06-07 01:00:00           NaN
2016-06-07 02:00:00           NaN
2016-06-07 03:00:00           NaN
2016-06-07 04:00:00           NaN

Output of pd.show_versions()

pandas: 0.20.2 pytest: 3.1.1 pip: 9.0.1 setuptools: 27.2.0 Cython: 0.25.2 numpy: 1.12.1 scipy: 0.19.0 xarray: 0.9.6 IPython: 6.1.0 sphinx: 1.5.6 patsy: 0.4.1 dateutil: 2.6.0 pytz: 2017.2 blosc: None bottleneck: 1.2.1 tables: 3.3.0 numexpr: 2.6.2 feather: None matplotlib: 2.0.2 openpyxl: 2.4.7 xlrd: 1.0.0 xlwt: 1.2.0 xlsxwriter: 0.9.6 lxml: 3.8.0 bs4: 4.6.0 html5lib: 0.999 sqlalchemy: 1.1.10 pymysql: None psycopg2: None jinja2: 2.9.6 s3fs: None pandas_gbq: None pandas_datareader: None
@prcastro prcastro changed the title Grouper using frequence has inconsistent behavior inside and outside a list Grouper using frequency has inconsistent behavior inside and outside a list Jun 21, 2017
@jreback
Copy link
Contributor

jreback commented Jun 22, 2017

You would have to show a complete example. How is it different from this. (which is on master).

In [8]: df = pd.DataFrame({'A': range(10), 'B': pd.date_range('20170101', periods=10, freq='30min')})

In [9]: df
Out[9]: 
   A                   B
0  0 2017-01-01 00:00:00
1  1 2017-01-01 00:30:00
2  2 2017-01-01 01:00:00
3  3 2017-01-01 01:30:00
4  4 2017-01-01 02:00:00
5  5 2017-01-01 02:30:00
6  6 2017-01-01 03:00:00
7  7 2017-01-01 03:30:00
8  8 2017-01-01 04:00:00
9  9 2017-01-01 04:30:00

In [10]: df.groupby(pd.Grouper(key='B', freq='H')).agg({'A': 'mean'})
Out[10]: 
                       A
B                       
2017-01-01 00:00:00  0.5
2017-01-01 01:00:00  2.5
2017-01-01 02:00:00  4.5
2017-01-01 03:00:00  6.5
2017-01-01 04:00:00  8.5

In [11]: df.groupby([pd.Grouper(key='B', freq='H')]).agg({'A': 'mean'})
Out[11]: 
                       A
B                       
2017-01-01 00:00:00  0.5
2017-01-01 01:00:00  2.5
2017-01-01 02:00:00  4.5
2017-01-01 03:00:00  6.5
2017-01-01 04:00:00  8.5

@prcastro
Copy link
Contributor Author

prcastro commented Jun 22, 2017

This data reproduces the bug, when using timestamp as the index of the dataframe (sorry to give you the data in the body of a comment, but I'm behind a coorporate proxy, and don't have access to upload platforms like pastebin):

timestamp,value
2016-06-07 00:00:39.000000,75000000000
2016-06-07 00:00:39.000000,149000000000
2016-06-07 00:00:39.000000,223000000000
2016-06-07 00:00:39.000000,297000000000
2016-06-07 00:00:39.000000,372000000000
2016-06-07 00:00:39.000000,446000000000
2016-06-07 00:00:39.000000,521000000000
2016-06-07 00:00:39.000000,596000000000
2016-06-07 00:00:39.000000,670000000000
2016-06-07 00:01:54.000000,74000000000
2016-06-07 00:01:54.000000,148000000000
2016-06-07 00:01:54.000000,222000000000
2016-06-07 00:01:54.000000,297000000000
2016-06-07 00:01:54.000000,371000000000
2016-06-07 00:01:54.000000,446000000000
2016-06-07 00:01:54.000000,521000000000
2016-06-07 00:01:54.000000,595000000000
2016-06-07 00:03:08.000000,74000000000
2016-06-07 00:03:08.000000,148000000000
2016-06-07 00:03:08.000000,223000000000
2016-06-07 00:03:08.000000,297000000000
2016-06-07 00:03:08.000000,372000000000
2016-06-07 00:03:08.000000,447000000000
2016-06-07 00:03:08.000000,521000000000
2016-06-07 00:04:22.000000,74000000000
2016-06-07 00:04:22.000000,149000000000
2016-06-07 00:04:22.000000,223000000000
2016-06-07 00:04:22.000000,298000000000
2016-06-07 00:04:22.000000,373000000000
2016-06-07 00:04:22.000000,447000000000
2016-06-07 00:05:36.000000,75000000000
2016-06-07 00:05:36.000000,149000000000
2016-06-07 00:05:36.000000,224000000000
2016-06-07 00:05:36.000000,299000000000
2016-06-07 00:05:36.000000,373000000000
2016-06-07 00:06:51.000000,74000000000
2016-06-07 00:06:51.000000,149000000000
2016-06-07 00:06:51.000000,224000000000
2016-06-07 00:06:51.000000,298000000000
2016-06-07 00:08:05.000000,75000000000
2016-06-07 00:08:05.000000,150000000000
2016-06-07 00:08:05.000000,224000000000
2016-06-07 00:09:20.000000,75000000000
2016-06-07 00:09:20.000000,149000000000
2016-06-07 00:10:35.000000,74000000000
2016-07-13 00:00:56.000000,76000000000
2016-07-13 00:00:56.000000,152000000000
2016-07-13 00:00:56.000000,228000000000
2016-07-13 00:00:56.000000,304000000000
2016-07-13 00:00:56.000000,379000000000
2016-07-13 00:00:56.000000,454000000000
2016-07-13 00:00:56.000000,530000000000
2016-07-13 00:00:56.000000,606000000000
2016-07-13 00:00:56.000000,682000000000
2016-07-13 00:02:12.000000,76000000000
2016-07-13 00:02:12.000000,152000000000
2016-07-13 00:02:12.000000,228000000000
2016-07-13 00:02:12.000000,303000000000
2016-07-13 00:02:12.000000,378000000000
2016-07-13 00:02:12.000000,454000000000
2016-07-13 00:02:12.000000,530000000000
2016-07-13 00:02:12.000000,606000000000
2016-07-13 00:03:28.000000,76000000000
2016-07-13 00:03:28.000000,152000000000
2016-07-13 00:03:28.000000,227000000000
2016-07-13 00:03:28.000000,302000000000
2016-07-13 00:03:28.000000,378000000000
2016-07-13 00:03:28.000000,454000000000
2016-07-13 00:03:28.000000,530000000000
2016-07-13 00:04:44.000000,76000000000
2016-07-13 00:04:44.000000,151000000000
2016-07-13 00:04:44.000000,226000000000
2016-07-13 00:04:44.000000,302000000000
2016-07-13 00:04:44.000000,378000000000
2016-07-13 00:04:44.000000,454000000000
2016-07-13 00:06:00.000000,75000000000
2016-07-13 00:06:00.000000,150000000000
2016-07-13 00:06:00.000000,226000000000
2016-07-13 00:06:00.000000,302000000000
2016-07-13 00:06:00.000000,378000000000
2016-07-13 00:07:15.000000,75000000000
2016-07-13 00:07:15.000000,151000000000
2016-07-13 00:07:15.000000,227000000000
2016-07-13 00:07:15.000000,303000000000
2016-07-13 00:08:30.000000,76000000000
2016-07-13 00:08:30.000000,152000000000
2016-07-13 00:08:30.000000,228000000000
2016-07-13 00:09:46.000000,76000000000
2016-07-13 00:09:46.000000,152000000000
2016-07-13 00:11:02.000000,76000000000
2016-07-15 00:02:21.000000,75000000000
2016-07-15 00:02:21.000000,150000000000
2016-07-15 00:02:21.000000,225000000000
2016-07-15 00:02:21.000000,300000000000
2016-07-15 00:02:21.000000,374000000000
2016-07-15 00:02:21.000000,449000000000
2016-07-15 00:02:21.000000,523000000000
2016-07-15 00:02:21.000000,598000000000
2016-07-15 00:03:36.000000,75000000000
2016-07-15 00:03:36.000000,150000000000
2016-07-15 00:03:36.000000,225000000000
2016-07-15 00:03:36.000000,299000000000
2016-07-15 00:03:36.000000,374000000000
2016-07-15 00:03:36.000000,448000000000
2016-07-15 00:03:36.000000,523000000000
2016-07-15 00:04:51.000000,75000000000
2016-07-15 00:04:51.000000,150000000000
2016-07-15 00:04:51.000000,224000000000
2016-07-15 00:04:51.000000,299000000000
2016-07-15 00:04:51.000000,373000000000
2016-07-15 00:04:51.000000,448000000000
2016-07-15 00:06:06.000000,75000000000
2016-07-15 00:06:06.000000,149000000000
2016-07-15 00:06:06.000000,224000000000
2016-07-15 00:06:06.000000,298000000000
2016-07-15 00:06:06.000000,373000000000
2016-07-15 00:07:21.000000,74000000000
2016-07-15 00:07:21.000000,149000000000
2016-07-15 00:07:21.000000,223000000000
2016-07-15 00:07:21.000000,298000000000
2016-07-15 00:08:35.000000,75000000000
2016-07-15 00:08:35.000000,149000000000
2016-07-15 00:08:35.000000,224000000000
2016-07-15 00:09:50.000000,74000000000
2016-07-15 00:09:50.000000,149000000000
2016-07-15 00:11:04.000000,75000000000
2016-10-07 00:01:07.000000,76000000000
2016-10-07 00:01:07.000000,152000000000
2016-10-07 00:01:07.000000,227000000000
2016-10-07 00:01:07.000000,303000000000
2016-10-07 00:01:07.000000,378000000000
2016-10-07 00:01:07.000000,453000000000
2016-10-07 00:01:07.000000,528000000000
2016-10-07 00:01:07.000000,603000000000
2016-10-07 00:01:07.000000,677000000000
2016-10-07 00:01:07.000000,751000000000
2016-10-07 00:01:07.000000,825000000000
2016-10-07 00:02:23.000000,76000000000
2016-10-07 00:02:23.000000,151000000000
2016-10-07 00:02:23.000000,227000000000
2016-10-07 00:02:23.000000,302000000000
2016-10-07 00:02:23.000000,377000000000
2016-10-07 00:02:23.000000,452000000000
2016-10-07 00:02:23.000000,527000000000
2016-10-07 00:02:23.000000,601000000000
2016-10-07 00:02:23.000000,675000000000
2016-10-07 00:02:23.000000,749000000000
2016-10-07 00:03:39.000000,75000000000
2016-10-07 00:03:39.000000,151000000000
2016-10-07 00:03:39.000000,226000000000
2016-10-07 00:03:39.000000,301000000000
2016-10-07 00:03:39.000000,376000000000
2016-10-07 00:03:39.000000,451000000000
2016-10-07 00:03:39.000000,525000000000
2016-10-07 00:03:39.000000,599000000000
2016-10-07 00:03:39.000000,673000000000
2016-10-07 00:04:54.000000,76000000000
2016-10-07 00:04:54.000000,151000000000
2016-10-07 00:04:54.000000,226000000000
2016-10-07 00:04:54.000000,301000000000
2016-10-07 00:04:54.000000,376000000000
2016-10-07 00:04:54.000000,450000000000
2016-10-07 00:04:54.000000,524000000000
2016-10-07 00:04:54.000000,598000000000
2016-10-07 00:06:10.000000,75000000000
2016-10-07 00:06:10.000000,150000000000
2016-10-07 00:06:10.000000,225000000000
2016-10-07 00:06:10.000000,300000000000
2016-10-07 00:06:10.000000,374000000000
2016-10-07 00:06:10.000000,448000000000
2016-10-07 00:06:10.000000,522000000000
2016-10-07 00:07:25.000000,75000000000
2016-10-07 00:07:25.000000,150000000000
2016-10-07 00:07:25.000000,225000000000
2016-10-07 00:07:25.000000,299000000000
2016-10-07 00:07:25.000000,373000000000
2016-10-07 00:07:25.000000,447000000000
2016-10-07 00:08:40.000000,75000000000
2016-10-07 00:08:40.000000,150000000000
2016-10-07 00:08:40.000000,224000000000
2016-10-07 00:08:40.000000,298000000000
2016-10-07 00:08:40.000000,372000000000
2016-10-07 00:09:55.000000,75000000000
2016-10-07 00:09:55.000000,149000000000
2016-10-07 00:09:55.000000,223000000000
2016-10-07 00:09:55.000000,297000000000
2016-10-07 00:11:10.000000,74000000000
2016-10-07 00:11:10.000000,148000000000
2016-10-07 00:11:10.000000,222000000000
2016-10-07 00:12:24.000000,74000000000
2016-10-07 00:12:24.000000,148000000000
2016-10-07 00:13:38.000000,74000000000
2016-07-07 00:01:43.000000,76000000000
2016-07-07 00:01:43.000000,151000000000
2016-07-07 00:01:43.000000,226000000000
2016-07-07 00:01:43.000000,302000000000
2016-07-07 00:01:43.000000,378000000000
2016-07-07 00:01:43.000000,453000000000
2016-07-07 00:01:43.000000,527000000000
2016-07-07 00:01:43.000000,602000000000
2016-07-07 00:02:59.000000,75000000000
2016-07-07 00:02:59.000000,150000000000
2016-07-07 00:02:59.000000,226000000000
2016-07-07 00:02:59.000000,302000000000
2016-07-07 00:02:59.000000,377000000000
2016-07-07 00:02:59.000000,451000000000
2016-07-07 00:02:59.000000,526000000000
2016-07-07 00:04:14.000000,75000000000
2016-07-07 00:04:14.000000,151000000000
2016-07-07 00:04:14.000000,227000000000
2016-07-07 00:04:14.000000,302000000000
2016-07-07 00:04:14.000000,376000000000
2016-07-07 00:04:14.000000,451000000000
2016-07-07 00:05:29.000000,76000000000
2016-07-07 00:05:29.000000,152000000000
2016-07-07 00:05:29.000000,227000000000
2016-07-07 00:05:29.000000,301000000000
2016-07-07 00:05:29.000000,376000000000
2016-07-07 00:06:45.000000,76000000000
2016-07-07 00:06:45.000000,151000000000
2016-07-07 00:06:45.000000,225000000000
2016-07-07 00:06:45.000000,300000000000
2016-07-07 00:08:01.000000,75000000000
2016-07-07 00:08:01.000000,149000000000
2016-07-07 00:08:01.000000,224000000000
2016-07-07 00:09:16.000000,74000000000
2016-07-07 00:09:16.000000,149000000000
2016-07-07 00:10:30.000000,75000000000
2016-07-23 00:03:12.400000,222600000000
2016-07-23 00:03:12.400000,297600000000
2016-07-23 00:03:12.400000,371600000000
2016-07-23 00:03:12.400000,482100000000
2016-07-23 00:06:55.000000,75000000000
2016-07-23 00:06:55.000000,149000000000
2016-07-23 00:06:55.000000,259500000000
2016-07-23 00:08:10.000000,74000000000
2016-07-23 00:08:10.000000,184500000000
2016-07-23 00:09:24.000000,110500000000
2016-07-26 00:01:55.000000,113000000000
2016-07-26 00:01:55.000000,226000000000
2016-07-26 00:01:55.000000,301000000000
2016-07-26 00:01:55.000000,414000000000
2016-07-26 00:01:55.000000,564500000000
2016-07-26 00:03:48.000000,113000000000
2016-07-26 00:03:48.000000,188000000000
2016-07-26 00:03:48.000000,301000000000
2016-07-26 00:03:48.000000,451500000000
2016-07-26 00:05:41.000000,75000000000
2016-07-26 00:05:41.000000,188000000000
2016-07-26 00:05:41.000000,338500000000
2016-07-26 00:06:56.000000,113000000000
2016-07-26 00:06:56.000000,263500000000
2016-07-26 00:08:49.000000,150500000000
2016-07-16 00:00:21.000000,75000000000
2016-07-16 00:00:21.000000,152000000000
2016-07-16 00:00:21.000000,266000000000
2016-07-16 00:00:21.000000,379000000000
2016-07-16 00:00:21.000000,456000000000
2016-07-16 00:00:21.000000,607000000000
2016-07-16 00:01:36.000000,77000000000
2016-07-16 00:01:36.000000,191000000000
2016-07-16 00:01:36.000000,304000000000
2016-07-16 00:01:36.000000,381000000000
2016-07-16 00:01:36.000000,532000000000
2016-07-16 00:02:53.000000,114000000000
2016-07-16 00:02:53.000000,227000000000
2016-07-16 00:02:53.000000,304000000000
2016-07-16 00:02:53.000000,455000000000
2016-07-16 00:04:47.000000,113000000000
2016-07-16 00:04:47.000000,190000000000
2016-07-16 00:04:47.000000,341000000000
2016-07-16 00:06:40.000000,77000000000
2016-07-16 00:06:40.000000,228000000000
2016-07-16 00:07:57.000000,151000000000
2016-07-18 00:00:31.000000,76000000000
2016-07-18 00:00:31.000000,157000000000
2016-07-18 00:00:31.000000,270500000000
2016-07-18 00:00:31.000000,383000000000
2016-07-18 00:00:31.000000,458000000000
2016-07-18 00:00:31.000000,533000000000
2016-07-18 00:00:31.000000,644500000000
2016-07-18 00:00:31.000000,757000000000
2016-07-18 00:00:31.000000,873500000000
2016-07-18 00:00:31.000000,985000000000
2016-07-18 00:01:47.000000,81000000000
2016-07-18 00:01:47.000000,194500000000
2016-07-18 00:01:47.000000,307000000000
2016-07-18 00:01:47.000000,382000000000
2016-07-18 00:01:47.000000,457000000000
2016-07-18 00:01:47.000000,568500000000
2016-07-18 00:01:47.000000,681000000000
2016-07-18 00:01:47.000000,797500000000
2016-07-18 00:01:47.000000,909000000000
2016-07-18 00:03:08.000000,113500000000
2016-07-18 00:03:08.000000,226000000000
2016-07-18 00:03:08.000000,301000000000
2016-07-18 00:03:08.000000,376000000000
2016-07-18 00:03:08.000000,487500000000
2016-07-18 00:03:08.000000,600000000000
2016-07-18 00:03:08.000000,716500000000
2016-07-18 00:03:08.000000,828000000000
2016-07-18 00:05:01.500000,112500000000
2016-07-18 00:05:01.500000,187500000000
2016-07-18 00:05:01.500000,262500000000
2016-07-18 00:05:01.500000,374000000000
2016-07-18 00:05:01.500000,486500000000
2016-07-18 00:05:01.500000,603000000000
2016-07-18 00:05:01.500000,714500000000
2016-07-18 00:06:54.000000,75000000000
2016-07-18 00:06:54.000000,150000000000
2016-07-18 00:06:54.000000,261500000000
2016-07-18 00:06:54.000000,374000000000
2016-07-18 00:06:54.000000,490500000000
2016-07-18 00:06:54.000000,602000000000
2016-07-18 00:08:09.000000,75000000000
2016-07-18 00:08:09.000000,186500000000
2016-07-18 00:08:09.000000,299000000000
2016-07-18 00:08:09.000000,415500000000
2016-07-18 00:08:09.000000,527000000000
2016-07-18 00:09:24.000000,111500000000
2016-07-18 00:09:24.000000,224000000000
2016-07-18 00:09:24.000000,340500000000
2016-07-18 00:09:24.000000,452000000000
2016-07-18 00:11:15.500000,112500000000
2016-07-18 00:11:15.500000,229000000000
2016-07-18 00:11:15.500000,340500000000
2016-07-18 00:13:08.000000,116500000000
2016-07-18 00:13:08.000000,228000000000
2016-07-18 00:15:04.500000,111500000000
2016-07-28 00:00:49.000000,114500000000
2016-07-28 00:00:49.000000,229000000000
2016-07-28 00:00:49.000000,306000000000
2016-07-28 00:00:49.000000,382000000000
2016-07-28 00:00:49.000000,496000000000
2016-07-28 00:00:49.000000,611000000000
2016-07-28 00:00:49.000000,687000000000
2016-07-28 00:02:43.500000,114500000000
2016-07-28 00:02:43.500000,191500000000
2016-07-28 00:02:43.500000,267500000000
2016-07-28 00:02:43.500000,381500000000
2016-07-28 00:02:43.500000,496500000000
2016-07-28 00:02:43.500000,572500000000
2016-07-28 00:04:38.000000,77000000000
2016-07-28 00:04:38.000000,153000000000
2016-07-28 00:04:38.000000,267000000000
2016-07-28 00:04:38.000000,382000000000
2016-07-28 00:04:38.000000,458000000000
2016-07-28 00:05:55.000000,76000000000
2016-07-28 00:05:55.000000,190000000000
2016-07-28 00:05:55.000000,305000000000
2016-07-28 00:05:55.000000,381000000000
2016-07-28 00:07:11.000000,114000000000
2016-07-28 00:07:11.000000,229000000000
2016-07-28 00:07:11.000000,305000000000
2016-07-28 00:09:05.000000,115000000000
2016-07-28 00:09:05.000000,191000000000
2016-07-28 00:11:00.000000,76000000000
2016-12-07 00:01:21.000000,75000000000
2016-12-07 00:01:21.000000,150000000000
2016-12-07 00:01:21.000000,225000000000
2016-12-07 00:01:21.000000,300000000000
2016-12-07 00:01:21.000000,375000000000
2016-12-07 00:01:21.000000,449000000000
2016-12-07 00:01:21.000000,524000000000
2016-12-07 00:01:21.000000,598000000000
2016-12-07 00:02:36.000000,75000000000
2016-12-07 00:02:36.000000,150000000000
2016-12-07 00:02:36.000000,225000000000
2016-12-07 00:02:36.000000,300000000000
2016-12-07 00:02:36.000000,374000000000
2016-12-07 00:02:36.000000,449000000000
2016-12-07 00:02:36.000000,523000000000
2016-12-07 00:03:51.000000,75000000000
2016-12-07 00:03:51.000000,150000000000
2016-12-07 00:03:51.000000,225000000000
2016-12-07 00:03:51.000000,299000000000
2016-12-07 00:03:51.000000,374000000000
2016-12-07 00:03:51.000000,448000000000
2016-12-07 00:05:06.000000,75000000000
2016-12-07 00:05:06.000000,150000000000
2016-12-07 00:05:06.000000,224000000000
2016-12-07 00:05:06.000000,299000000000
2016-12-07 00:05:06.000000,373000000000
2016-12-07 00:06:21.000000,75000000000
2016-12-07 00:06:21.000000,149000000000
2016-12-07 00:06:21.000000,224000000000
2016-12-07 00:06:21.000000,298000000000
2016-12-07 00:07:36.000000,74000000000
2016-12-07 00:07:36.000000,149000000000
2016-12-07 00:07:36.000000,223000000000
2016-12-07 00:08:50.000000,75000000000
2016-12-07 00:08:50.000000,149000000000
2016-12-07 00:10:05.000000,74000000000
2016-07-20 00:02:13.000000,342500000000
2016-07-20 00:02:13.000000,608000000000
2016-07-20 00:07:55.500000,265500000000
2016-07-22 00:00:51.000000,111000000000
2016-07-22 00:00:51.000000,222000000000
2016-07-22 00:00:51.000000,296000000000
2016-07-22 00:00:51.000000,370000000000
2016-07-22 00:00:51.000000,444000000000
2016-07-22 00:00:51.000000,518000000000
2016-07-22 00:00:51.000000,592000000000
2016-07-22 00:00:51.000000,666000000000
2016-07-22 00:02:42.000000,111000000000
2016-07-22 00:02:42.000000,185000000000
2016-07-22 00:02:42.000000,259000000000
2016-07-22 00:02:42.000000,333000000000
2016-07-22 00:02:42.000000,407000000000
2016-07-22 00:02:42.000000,481000000000
2016-07-22 00:02:42.000000,555000000000
2016-07-22 00:04:33.000000,74000000000
2016-07-22 00:04:33.000000,148000000000
2016-07-22 00:04:33.000000,222000000000
2016-07-22 00:04:33.000000,296000000000
2016-07-22 00:04:33.000000,370000000000
2016-07-22 00:04:33.000000,444000000000
2016-07-22 00:05:47.000000,74000000000
2016-07-22 00:05:47.000000,148000000000
2016-07-22 00:05:47.000000,222000000000
2016-07-22 00:05:47.000000,296000000000
2016-07-22 00:05:47.000000,370000000000
2016-07-22 00:07:01.000000,74000000000
2016-07-22 00:07:01.000000,148000000000
2016-07-22 00:07:01.000000,222000000000
2016-07-22 00:07:01.000000,296000000000
2016-07-22 00:08:15.000000,74000000000
2016-07-22 00:08:15.000000,148000000000
2016-07-22 00:08:15.000000,222000000000
2016-07-22 00:09:29.000000,74000000000
2016-07-22 00:09:29.000000,148000000000
2016-07-22 00:10:43.000000,74000000000
2016-08-07 00:00:24.000000,75000000000
2016-08-07 00:00:24.000000,150000000000
2016-08-07 00:00:24.000000,225000000000
2016-08-07 00:00:24.000000,301000000000
2016-08-07 00:00:24.000000,376000000000
2016-08-07 00:00:24.000000,488500000000
2016-08-07 00:00:24.000000,601000000000
2016-08-07 00:00:24.000000,679000000000
2016-08-07 00:01:39.000000,75000000000
2016-08-07 00:01:39.000000,150000000000
2016-08-07 00:01:39.000000,226000000000
2016-08-07 00:01:39.000000,301000000000
2016-08-07 00:01:39.000000,413500000000
2016-08-07 00:01:39.000000,526000000000
2016-08-07 00:01:39.000000,604000000000
2016-08-07 00:02:54.000000,75000000000
2016-08-07 00:02:54.000000,151000000000
2016-08-07 00:02:54.000000,226000000000
2016-08-07 00:02:54.000000,338500000000
2016-08-07 00:02:54.000000,451000000000
2016-08-07 00:02:54.000000,529000000000
2016-08-07 00:04:09.000000,76000000000
2016-08-07 00:04:09.000000,151000000000
2016-08-07 00:04:09.000000,263500000000
2016-08-07 00:04:09.000000,376000000000
2016-08-07 00:04:09.000000,454000000000
2016-08-07 00:05:25.000000,75000000000
2016-08-07 00:05:25.000000,187500000000
2016-08-07 00:05:25.000000,300000000000
2016-08-07 00:05:25.000000,378000000000
2016-08-07 00:06:40.000000,112500000000
2016-08-07 00:06:40.000000,225000000000
2016-08-07 00:06:40.000000,303000000000
2016-08-07 00:08:32.500000,112500000000
2016-08-07 00:08:32.500000,190500000000
2016-08-07 00:10:25.000000,78000000000
2016-11-07 00:01:41.000000,113000000000
2016-11-07 00:01:41.000000,188000000000
2016-11-07 00:01:41.000000,263000000000
2016-11-07 00:01:41.000000,339000000000
2016-11-07 00:01:41.000000,414000000000
2016-11-07 00:01:41.000000,489000000000
2016-11-07 00:01:41.000000,565000000000
2016-11-07 00:01:41.000000,640000000000
2016-11-07 00:01:41.000000,716000000000
2016-11-07 00:01:41.000000,792000000000
2016-11-07 00:01:41.000000,874000000000
2016-11-07 00:01:41.000000,950000000000
2016-11-07 00:03:34.000000,75000000000
2016-11-07 00:03:34.000000,150000000000
2016-11-07 00:03:34.000000,226000000000
2016-11-07 00:03:34.000000,301000000000
2016-11-07 00:03:34.000000,376000000000
2016-11-07 00:03:34.000000,452000000000
2016-11-07 00:03:34.000000,527000000000
2016-11-07 00:03:34.000000,603000000000
2016-11-07 00:03:34.000000,679000000000
2016-11-07 00:03:34.000000,761000000000
2016-11-07 00:03:34.000000,837000000000
2016-11-07 00:04:49.000000,75000000000
2016-11-07 00:04:49.000000,151000000000
2016-11-07 00:04:49.000000,226000000000
2016-11-07 00:04:49.000000,301000000000
2016-11-07 00:04:49.000000,377000000000
2016-11-07 00:04:49.000000,452000000000
2016-11-07 00:04:49.000000,528000000000
2016-11-07 00:04:49.000000,604000000000
2016-11-07 00:04:49.000000,686000000000
2016-11-07 00:04:49.000000,762000000000
2016-11-07 00:06:04.000000,76000000000
2016-11-07 00:06:04.000000,151000000000
2016-11-07 00:06:04.000000,226000000000
2016-11-07 00:06:04.000000,302000000000
2016-11-07 00:06:04.000000,377000000000
2016-11-07 00:06:04.000000,453000000000
2016-11-07 00:06:04.000000,529000000000
2016-11-07 00:06:04.000000,611000000000
2016-11-07 00:06:04.000000,687000000000
2016-11-07 00:07:20.000000,75000000000
2016-11-07 00:07:20.000000,150000000000
2016-11-07 00:07:20.000000,226000000000
2016-11-07 00:07:20.000000,301000000000
2016-11-07 00:07:20.000000,377000000000
2016-11-07 00:07:20.000000,453000000000
2016-11-07 00:07:20.000000,535000000000
2016-11-07 00:07:20.000000,611000000000
2016-11-07 00:08:35.000000,75000000000
2016-11-07 00:08:35.000000,151000000000
2016-11-07 00:08:35.000000,226000000000
2016-11-07 00:08:35.000000,302000000000
2016-11-07 00:08:35.000000,378000000000
2016-11-07 00:08:35.000000,460000000000
2016-11-07 00:08:35.000000,536000000000
2016-11-07 00:09:50.000000,76000000000
2016-11-07 00:09:50.000000,151000000000
2016-11-07 00:09:50.000000,227000000000
2016-11-07 00:09:50.000000,303000000000
2016-11-07 00:09:50.000000,385000000000
2016-11-07 00:09:50.000000,461000000000
2016-11-07 00:11:06.000000,75000000000
2016-11-07 00:11:06.000000,151000000000
2016-11-07 00:11:06.000000,227000000000
2016-11-07 00:11:06.000000,309000000000
2016-11-07 00:11:06.000000,385000000000
2016-11-07 00:12:21.000000,76000000000
2016-11-07 00:12:21.000000,152000000000
2016-11-07 00:12:21.000000,234000000000
2016-11-07 00:12:21.000000,310000000000
2016-11-07 00:13:37.000000,76000000000
2016-11-07 00:13:37.000000,158000000000
2016-11-07 00:13:37.000000,234000000000
2016-11-07 00:14:53.000000,82000000000
2016-11-07 00:14:53.000000,158000000000
2016-11-07 00:16:15.000000,76000000000
2016-07-19 00:01:12.000000,75000000000
2016-07-19 00:01:12.000000,151000000000
2016-07-19 00:01:12.000000,226000000000
2016-07-19 00:01:12.000000,303000000000
2016-07-19 00:01:12.000000,379000000000
2016-07-19 00:01:12.000000,454000000000
2016-07-19 00:01:12.000000,530000000000
2016-07-19 00:01:12.000000,605000000000
2016-07-19 00:01:12.000000,680000000000
2016-07-19 00:02:27.000000,76000000000
2016-07-19 00:02:27.000000,151000000000
2016-07-19 00:02:27.000000,228000000000
2016-07-19 00:02:27.000000,304000000000
2016-07-19 00:02:27.000000,379000000000
2016-07-19 00:02:27.000000,455000000000
2016-07-19 00:02:27.000000,530000000000
2016-07-19 00:02:27.000000,605000000000
2016-07-19 00:03:43.000000,75000000000
2016-07-19 00:03:43.000000,152000000000
2016-07-19 00:03:43.000000,228000000000
2016-07-19 00:03:43.000000,303000000000
2016-07-19 00:03:43.000000,379000000000
2016-07-19 00:03:43.000000,454000000000
2016-07-19 00:03:43.000000,529000000000
2016-07-19 00:04:58.000000,77000000000
2016-07-19 00:04:58.000000,153000000000
2016-07-19 00:04:58.000000,228000000000
2016-07-19 00:04:58.000000,304000000000
2016-07-19 00:04:58.000000,379000000000
2016-07-19 00:04:58.000000,454000000000
2016-07-19 00:06:15.000000,76000000000
2016-07-19 00:06:15.000000,151000000000
2016-07-19 00:06:15.000000,227000000000
2016-07-19 00:06:15.000000,302000000000
2016-07-19 00:06:15.000000,377000000000
2016-07-19 00:07:31.000000,75000000000
2016-07-19 00:07:31.000000,151000000000
2016-07-19 00:07:31.000000,226000000000
2016-07-19 00:07:31.000000,301000000000
2016-07-19 00:08:46.000000,76000000000
2016-07-19 00:08:46.000000,151000000000
2016-07-19 00:08:46.000000,226000000000
2016-07-19 00:10:02.000000,75000000000
2016-07-19 00:10:02.000000,150000000000
2016-07-19 00:11:17.000000,75000000000
2016-11-07 00:07:57.583333,497416667000
2016-11-07 00:07:57.583333,573416667000
2016-11-07 00:16:15.000000,76000000000
2016-12-07 00:00:42.000000,114000000000
2016-12-07 00:00:42.000000,264000000000
2016-12-07 00:02:36.000000,150000000000
2016-07-25 00:07:33.300000,421700000000
2016-07-25 00:07:33.300000,497700000000
2016-07-25 00:07:33.300000,573700000000
2016-07-25 00:14:35.000000,76000000000
2016-07-25 00:14:35.000000,152000000000
2016-07-25 00:15:51.000000,76000000000
2016-07-17 00:00:39.000000,75000000000
2016-07-17 00:00:39.000000,150000000000
2016-07-17 00:00:39.000000,225000000000
2016-07-17 00:00:39.000000,300000000000
2016-07-17 00:00:39.000000,374000000000
2016-07-17 00:00:39.000000,449000000000
2016-07-17 00:00:39.000000,524000000000
2016-07-17 00:00:39.000000,635500000000
2016-07-17 00:00:39.000000,747000000000
2016-07-17 00:01:54.000000,75000000000
2016-07-17 00:01:54.000000,150000000000
2016-07-17 00:01:54.000000,225000000000
2016-07-17 00:01:54.000000,299000000000
2016-07-17 00:01:54.000000,374000000000
2016-07-17 00:01:54.000000,449000000000
2016-07-17 00:01:54.000000,560500000000
2016-07-17 00:01:54.000000,672000000000
2016-07-17 00:03:09.000000,75000000000
2016-07-17 00:03:09.000000,150000000000
2016-07-17 00:03:09.000000,224000000000
2016-07-17 00:03:09.000000,299000000000
2016-07-17 00:03:09.000000,374000000000
2016-07-17 00:03:09.000000,485500000000
2016-07-17 00:03:09.000000,597000000000
2016-07-17 00:04:24.000000,75000000000
2016-07-17 00:04:24.000000,149000000000
2016-07-17 00:04:24.000000,224000000000
2016-07-17 00:04:24.000000,299000000000
2016-07-17 00:04:24.000000,410500000000
2016-07-17 00:04:24.000000,522000000000
2016-07-17 00:05:39.000000,74000000000
2016-07-17 00:05:39.000000,149000000000
2016-07-17 00:05:39.000000,224000000000
2016-07-17 00:05:39.000000,335500000000
2016-07-17 00:05:39.000000,447000000000
2016-07-17 00:06:53.000000,75000000000
2016-07-17 00:06:53.000000,150000000000
2016-07-17 00:06:53.000000,261500000000
2016-07-17 00:06:53.000000,373000000000
2016-07-17 00:08:08.000000,75000000000
2016-07-17 00:08:08.000000,186500000000
2016-07-17 00:08:08.000000,298000000000
2016-07-17 00:09:23.000000,111500000000
2016-07-17 00:09:23.000000,223000000000
2016-07-17 00:11:14.500000,111500000000
2016-07-24 00:00:30.000000,76000000000
2016-07-24 00:00:30.000000,152000000000
2016-07-24 00:00:30.000000,227000000000
2016-07-24 00:00:30.000000,303000000000
2016-07-24 00:00:30.000000,378000000000
2016-07-24 00:00:30.000000,454000000000
2016-07-24 00:00:30.000000,529000000000
2016-07-24 00:00:30.000000,605000000000
2016-07-24 00:00:30.000000,680000000000
2016-07-24 00:00:30.000000,755000000000
2016-07-24 00:01:46.000000,76000000000
2016-07-24 00:01:46.000000,151000000000
2016-07-24 00:01:46.000000,227000000000
2016-07-24 00:01:46.000000,302000000000
2016-07-24 00:01:46.000000,378000000000
2016-07-24 00:01:46.000000,453000000000
2016-07-24 00:01:46.000000,529000000000
2016-07-24 00:01:46.000000,604000000000
2016-07-24 00:01:46.000000,679000000000
2016-07-24 00:03:02.000000,75000000000
2016-07-24 00:03:02.000000,151000000000
2016-07-24 00:03:02.000000,226000000000
2016-07-24 00:03:02.000000,302000000000
2016-07-24 00:03:02.000000,377000000000
2016-07-24 00:03:02.000000,453000000000
2016-07-24 00:03:02.000000,528000000000
2016-07-24 00:03:02.000000,603000000000
2016-07-24 00:04:17.000000,76000000000
2016-07-24 00:04:17.000000,151000000000
2016-07-24 00:04:17.000000,227000000000
2016-07-24 00:04:17.000000,302000000000
2016-07-24 00:04:17.000000,378000000000
2016-07-24 00:04:17.000000,453000000000
2016-07-24 00:04:17.000000,528000000000
2016-07-24 00:05:33.000000,75000000000
2016-07-24 00:05:33.000000,151000000000
2016-07-24 00:05:33.000000,226000000000
2016-07-24 00:05:33.000000,302000000000
2016-07-24 00:05:33.000000,377000000000
2016-07-24 00:05:33.000000,452000000000
2016-07-24 00:06:48.000000,76000000000
2016-07-24 00:06:48.000000,151000000000
2016-07-24 00:06:48.000000,227000000000
2016-07-24 00:06:48.000000,302000000000
2016-07-24 00:06:48.000000,377000000000
2016-07-24 00:08:04.000000,75000000000
2016-07-24 00:08:04.000000,151000000000
2016-07-24 00:08:04.000000,226000000000
2016-07-24 00:08:04.000000,301000000000
2016-07-24 00:09:19.000000,76000000000
2016-07-24 00:09:19.000000,151000000000
2016-07-24 00:09:19.000000,226000000000
2016-07-24 00:10:35.000000,75000000000
2016-07-24 00:10:35.000000,150000000000
2016-07-24 00:11:50.000000,75000000000
2016-07-26 00:01:17.000000,113000000000
2016-07-26 00:01:17.000000,189000000000
2016-07-26 00:03:10.000000,76000000000

@prcastro
Copy link
Contributor Author

Using the data sample above as sample.csv, I get:

>>> df = pd.read_csv('sample.csv', parse_dates=['timestamp'], index_col='timestamp')


>>> df.groupby(pd.Grouper(freq='H')).agg({'value': 'mean'})
                            value
timestamp
2016-06-07 00:00:00  2.729111e+11
2016-06-07 01:00:00           NaN
2016-06-07 02:00:00           NaN
2016-06-07 03:00:00           NaN
2016-06-07 04:00:00           NaN
...



>>> df.groupby([pd.Grouper(freq='H')]).agg({'value': 'mean'})
                   value
timestamp
2016-06-07  2.729111e+11
2016-07-07  2.509444e+11
2016-07-13  2.775778e+11
2016-07-15  2.490556e+11
2016-07-16  2.676190e+11
...

@jreback
Copy link
Contributor

jreback commented Jun 28, 2017

ok, your example repros. If you can step thru both the working and non-working cases (as well as df.resample('H').value.mean() these should all be the same, and see if you can pinpoint the problem.

@jreback
Copy link
Contributor

jreback commented Sep 15, 2017

closing as duplicate of #17530

@jreback jreback closed this as completed Sep 15, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants