BUG: lib.infer_type broken for IntervalArray / PeriodArray

[h-vetinari]

While working on #23167, I found a regression in IntervalArray / PeriodArray - I'm guessing it's related to the whole EA effort. @jorisvandenbossche @TomAugspurger

>>> import pandas as pd
>>> import pandas._libs.lib as lib
>>> s = pd.Series([pd.Period(2013), pd.Period(2018)], dtype='category')
>>> lib.infer_dtype(s.values.categories)
Traceback (most recent call last):
  File "pandas\_libs\lib.pyx", line 1166, in pandas._libs.lib.infer_dtype
    values = getattr(value, '_values', getattr(value, 'values', value))
TypeError: Cannot convert PeriodArray to numpy.ndarray
>>>
>>> t = pd.Series([pd.Interval(0, 1), pd.Interval(0, 2)], dtype='category')
>>> lib.infer_dtype(t.values.categories)
Traceback (most recent call last):
  File "pandas\_libs\lib.pyx", line 1166, in pandas._libs.lib.infer_dtype
    values = getattr(value, '_values', getattr(value, 'values', value))
TypeError: Cannot convert IntervalArray to numpy.ndarray

This was still working in v0.23.4:

>>> import pandas as pd
>>> import pandas._libs.lib as lib
>>> pd.__version__
'0.23.4'
>>> s = pd.Series([pd.Period(2013), pd.Period(2018)], dtype='category')
>>> lib.infer_dtype(s.values.categories)
'period'
>>>
>>> t = pd.Series([pd.Interval(0, 1), pd.Interval(0, 2)], dtype='category')
>>> lib.infer_dtype(t.values.categories)
'interval'

This is due to the fact that lib.infer_dtype tries
values = getattr(value, '_values', getattr(value, 'values', value))
and both IntervalArray / PeriodArray do not return a numpy array for _values anymore. For PeriodArray, it would work if _values and values were switched, but for IntervalArray, both _values and values return the same (non-numpy-array) result.

[gfyoung]

While technically not a regression for us (regression means that it broke in the most recent release but was working in the one before that), this should be "restored" to what it was before we release 0.24.0.

A git bisect would be good to determine the cause.

[jorisvandenbossche]

Not that I want to discuss semantics too much, but this is for sure a regression :)
Just one that does not need to end up in an actual release if we fix it now, thanks to @h-vetinari discovering it on master.

[jorisvandenbossche]

I am not very familiar with the infer_dtype code, but I would think we should add 'period' (and the names of our other internal dtypes as well) to _TYPE_MAP, so the ExtensionArray will be properly recognized, just as Categorical is already recognized.

[TomAugspurger]

What's the expected dtype for

>>> lib.infer_dtype(integer_array([1, np.nan]))

Should we basically expect infer_dtype to throw away .dtype information, and treat it the same as [1, np.nan] (so mixed-integer-float)? Or do we add a new return value for integer-na?

cc @jreback I suspect you're the most familiar with this.

[jreback]

so for this example we would return Int64 (because it’s an actual array) but for a list i would expect the same as now (mixed integer float)

[TomAugspurger]

Would you say 'integer-na' or 'nullable-integer', rather than Int64 to be more consistent with the other return values, which aren't dtypes (no itemsize)?

[jreback]

i think we could eventually add those
but the inference is used in indexing so i am not sure if changing it now would work

[h-vetinari]

milestone ping pong, haha. ;-)

Would this still be acceptable during RC phase? I might be able to make some time.