Pandas groupby.median casts to int when possible

[simon-mo]

groupby.median() will return floats dataframe in most cases; however, when possible, it might return ints dataframe as well:

In [40]: df = pd.DataFrame(np.random.randint(0, 8, size=(100, 4)),
    ...:                                      columns=list('ABCD'))
    ...: df.groupby(df['A'].tolist()).median()
    ...:
Out[40]:
     A    B    C    D
0  0.0  4.0  4.0  4.0
1  1.0  3.0  1.0  1.0
2  2.0  6.0  4.0  4.0
3  3.0  3.0  5.5  3.0
4  4.0  4.0  4.0  5.0
5  5.0  4.0  4.0  3.0
6  6.0  5.0  2.5  2.0
7  7.0  2.5  3.0  4.5

In [41]: df = pd.DataFrame(np.random.randint(0, 8, size=(100, 4)),
    ...:                                      columns=list('ABCD'))
    ...: df.groupby(df['A'].tolist()).median()
    ...:
Out[41]:
   A  B  C  D
0  0  3  4  2
1  1  3  4  2
2  2  2  2  3
3  3  4  3  2
4  4  1  4  4
5  5  1  3  2
6  6  2  4  5
7  7  5  4  4

We need to address this in our groupby.

[simon-mo]

We will skip the test for this at this moment.

[kunalgosar]

I am reopening this issue as it needs a permanent fix

[dchigarev]

It seems that this problem is already fixed, at least this reproduced works as expected in the current master:

reproducer

if __name__ == "__main__":
    import pandas
    import modin.pandas as pd

    future_int = {
        "A": ["a", "b", "c", "d"],
        "B": [0, 1, 2, 3],
        "C": [4, 5, 6, 7],
        "D": [8, 9, 10, 11],
    }
    md_df, pd_df = pd.DataFrame(future_int), pandas.DataFrame(future_int)

    print("pd dtypes:\n", pd_df.groupby("A").median().dtypes, sep="")
    print("md dtypes:\n", md_df.groupby("A").median().dtypes, sep="")

    future_float = {
        "A": ["a", "b", "b", "d"],
        "B": [0, 1, 2, 3],
        "C": [4, 5, 6, 7],
        "D": [8, 9, 10, 11],
    }
    md_df, pd_df = pd.DataFrame(future_float), pandas.DataFrame(future_float)
    print("\n\npd dtypes:\n", pd_df.groupby("A").median().dtypes, sep="")
    print("md dtypes:\n", md_df.groupby("A").median().dtypes, sep="")

Output:

pd dtypes:
B    int64
C    int64
D    int64
dtype: object
md dtypes:
B    int64
C    int64
D    int64
dtype: object


pd dtypes:
B    float64
C    float64
D    float64
dtype: object
md dtypes:
B    float64
C    float64
D    float64
dtype: object

Since for groupby aggregation we use pandas version under the hood and it encapsulates all logic about downcasting dtypes in it, we'll always gonna get same dtypes as pandas gives.

So @devin-petersohn what do you think about closing that issue?

[devin-petersohn]

Yes, I have standardized the type inference in Modin, so this should be closed.