BUG: Unexpected/undocumented behaviour of sum and mean aggregations on object dtypes

[MJafarMashhadi]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

import pandas as pd

# Sum, looks kinda OK
>>> pd.Series('a b c d'.split()).sum()
'abcd'
>>> pd.Series('1 2 3 4'.split()).sum()
'1234'

# Mean, inconsistent behaviour
>>> pd.Series('a b c d'.split()).mean()
...
TypeError: Could not convert abcd to numeric
>>> pd.Series('1 2 3 4'.split()).mean()
308.5

I saw the last example in a #26927 comment, scanned the code to see what's wrong and eventually decided to open this issue.

Problem description

IMO When calculating the mean, data type conversion should either happen on each element of the series before applying sum or should happen in the end. Here, to calculate the mean of ['1', '2', '3', '4'], first a sum concatenates all the elements keeping the dtype, then it's converted to float (1234) and then divided by 4 which results in 308.5.

Expected Output

I have 3 options in mind:

1. an element-wise type conversion, which means:

>>> pd.Series('a b c d'.split()).sum()  # and mean
Type Error
>>> pd.Series('1 2 3 4'.split()).sum()
10
>>> pd.Series('1 2 3 4'.split()).mean()
2.5

2. throwing an error in these cases or adding a note in docs for consequences of performing these aggregations on an object dtype. For example, considering that np.sum(np.array('a b c d'.split())) fails with a TypeError it's not intuitive that the same function in pandas performs a concatenation.

pandas/pandas/core/generic.py

Line 10410 in d9fff27

"This is equivalent to the method ``numpy.sum``.",

just like what happens when dtype is explicitly set as 'string':

>>> pd.Series('1 2 3 4'.split(), dtype='string').mean()
...
TypeError: Cannot perform reduction 'mean' with string dtype

3. No type conversions in the middle of calculating the mean: (It's not a good option IMHO for all the side effects it will have)

>>> pd.Series('a b c d'.split()).sum()
'abcd'  
>>> pd.Series('a b c d'.split()).mean()
TypeError while trying to compute 'abcd' / 4, but not because 'abcd' cannot be converted to a number, but bc a string cannot be divided by a number
>>> pd.Series('1 2 3 4'.split()).sum()
'1234'
>>> pd.Series('1 2 3 4'.split()).mean()
TypeError while trying to '1234' / 4

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 4014a60
python : 3.8.3.final.0
python-bits : 64
OS : Linux
OS-release : 5.3.0-40-generic
Version : #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC 2020
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_CA.UTF-8
LOCALE : en_CA.UTF-8
pandas : 1.1.0.dev0+2171.g4014a6035
numpy : 1.18.5
pytz : 2020.1
dateutil : 2.8.1
pip : 20.1.1
setuptools : 47.3.1.post20200616
Cython : 0.29.20
pytest : 5.4.3
hypothesis : 5.16.1
sphinx : 3.1.1
blosc : None
feather : None
xlsxwriter : 1.2.9
lxml.etree : 4.5.1
html5lib : 1.0.1
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.15.0
pandas_datareader: None
bs4 : 4.9.1
bottleneck : 1.3.2
fsspec : 0.7.4
fastparquet : 0.4.0
gcsfs : None
matplotlib : 3.2.1
numexpr : 2.7.1
odfpy : None
openpyxl : 3.0.3
pandas_gbq : None
pyarrow : 0.17.1
pytables : None
pyxlsb : None
s3fs : 0.4.2
scipy : 1.4.1
sqlalchemy : 1.3.17
tables : 3.6.1
tabulate : 0.8.7
xarray : 0.15.1
xlrd : 1.2.0
xlwt : 1.3.0
numba : 0.48.0

[jreback]

pls look for a duplicate issue - this has been discussed multiple times

[MJafarMashhadi]

It looked like something that should've already been reported, but couldn't find a duplicate at first

[simonjayhawkins]

duplicate of #34671