Fix (Series|DataFrame).interpolate for datetime dtypes #19291
Conversation
pandas/core/internals.py
Outdated
| mask = np.atleast_2d(mask) | ||
| data = data.astype(np.float64) | ||
| data[mask] = np.nan | ||
|
|
There was a problem hiding this comment.
ths is a hodgepodge. use the hierarchy to make this code readable. IOW put the datetime stuff in the datetimeblock.
There was a problem hiding this comment.
This is in the DatetimeBlock. I'm confused.
There was a problem hiding this comment.
then why would you need is_datetime.
There was a problem hiding this comment.
this needs to be be way less if/then stuff.
There was a problem hiding this comment.
then why would you need is_datetime.
It's specifically is_datetimetz; the tzaware case gets slightly different handling. If you're suggesting a separate implementation for DatetimeTZBlock vs DatetimeBlock I guess that's OK, but pretty sure there'll be complaints about code duplication that way.
Codecov Report
@@ Coverage Diff @@
## master #19291 +/- ##
==========================================
+ Coverage 91.53% 91.57% +0.03%
==========================================
Files 150 148 -2
Lines 48738 48913 +175
==========================================
+ Hits 44611 44790 +179
+ Misses 4127 4123 -4
Continue to review full report at Codecov.
|
| dtype_counts = _maybe_transposed_self._data.get_dtype_counts() | ||
| if ('object' in dtype_counts and | ||
| dtype_counts.get('object') == len(_maybe_transposed_self.T)): | ||
| # Checking for 'object' lets us avoid sometimes-fragile tranpose |
There was a problem hiding this comment.
huh? where are you testing this
There was a problem hiding this comment.
This is a workaround until #19198 is fixed:
dti = pd.date_range('2016-01-01', periods=3, tz='US/Pacific').insert(1, pd.NaT)
ser = pd.Series(dti)
df = ser.to_frame()
>>> df.interpolate()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pandas/core/generic.py", line 5155, in interpolate
'object') == len(_maybe_transposed_self.T):
File "pandas/core/frame.py", line 1941, in transpose
return super(DataFrame, self).transpose(1, 0, **kwargs)
File "pandas/core/generic.py", line 616, in transpose
new_values = self.values.transpose(axes_numbers)
File "pandas/core/base.py", line 701, in transpose
nv.validate_transpose(args, kwargs)
File "pandas/compat/numpy/function.py", line 54, in __call__
self.defaults)
File "pandas/util/_validators.py", line 218, in validate_args_and_kwargs
validate_kwargs(fname, kwargs, compat_args)
File "pandas/util/_validators.py", line 157, in validate_kwargs
_check_for_default_values(fname, kwds, compat_args)
File "pandas/util/_validators.py", line 69, in _check_for_default_values
format(fname=fname, arg=key)))
ValueError: the 'axes' parameter is not supported in the pandas implementation of transpose()
(ser.interpolate doesn't raise, just forgets to interpolate)
You complained about the existence of |
|
i am suggesting a refactor completely of interpolate actually is very hacky now - this is just making it worse to be honest i would prob not touch this now many many other fish to fry |
|
OK. I'll put a pin in this until I can come up with something more elegant. Mind keeping it open for now so it doesn't fall off my radar screen? |
git diff upstream/master -u -- "*.py" | flake8 --diff