REGR: Panel.where

[max-sixty]

This used to work in 0.16.2:

In [78]: panel=pd.Panel(pd.np.random.rand(5,4,3))

In [80]: panel[0]
Out[80]: 
          0         1         2
0  0.191947  0.324781  0.070749
1  0.396628  0.966418  0.120066
2  0.551517  0.643907  0.230434
3  0.245471  0.485606  0.315219


In [79]: panel.where(panel>0.5, None)[0]
Out[79]: 
           0          1     2
0       None       None  None
1       None  0.9664181  None
2  0.5515173  0.6439069  None
3       None       None  None

But now in 0.17.0:

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
<ipython-input-13-5edbb4ece05d> in <module>()
----> 1 bs_raw.where(bs_raw, 3)

/usr/local/lib/python2.7/dist-packages/pandas/core/generic.pyc in where(self, cond, other, inplace, axis, level, try_cast, raise_on_error)
   3802 
   3803         if isinstance(cond, NDFrame):
-> 3804             cond, _ = cond.align(self, join='right', broadcast_axis=1)
   3805         else:
   3806             if not hasattr(cond, 'shape'):

/usr/local/lib/python2.7/dist-packages/pandas/core/panel.pyc in align(self, other, **kwargs)
    630 
    631     def align(self, other, **kwargs):
--> 632         raise NotImplementedError
    633 
    634     def dropna(self, axis=0, how='any', inplace=False):

NotImplementedError: 

I think this is as a result of #11095 but @terrytangyuan / @jreback may know the status?

[max-sixty]

Actually it may be here: #10283
CC @mortada

[jreback]

Aligning between a panel and other things (meaning lower dims) has not been implemented, though this is same dim alignment.

so this will work. I guess this is a regression, but I suspect not tested.

In [27]: p.where((p>0.5).values)
Out[27]: 
<class 'pandas.core.panel.Panel'>
Dimensions: 5 (items) x 4 (major_axis) x 3 (minor_axis)
Items axis: 0 to 4
Major_axis axis: 0 to 3
Minor_axis axis: 0 to 2

In [28]: p.where((p>0.5).values).values
Out[28]: 
array([[[        nan,         nan,         nan],
        [ 0.51946856,         nan,         nan],
        [ 0.98561153,  0.55156741,         nan],
        [ 0.7817365 ,  0.98288951,         nan]],

       [[        nan,         nan,  0.52340917],
        [        nan,         nan,  0.88230812],
        [ 0.84609837,  0.81249447,         nan],
        [        nan,         nan,  0.87110282]],

       [[ 0.60153362,         nan,  0.75575728],
        [        nan,         nan,         nan],
        [ 0.52964145,         nan,         nan],
        [        nan,         nan,  0.82057933]],

       [[        nan,  0.94693997,         nan],
        [        nan,  0.94250928,  0.89039156],
        [ 0.70035531,         nan,         nan],
        [        nan,         nan,         nan]],

       [[ 0.56797997,  0.90667059,  0.83789881],
        [        nan,  0.57919084,  0.87088217],
        [ 0.51711024,         nan,         nan],
        [        nan,         nan,         nan]]])

[max-sixty]

OK, thanks.

Is there a working alternative? I'm currently iterating over 2D numpy arrays and putting those back into the panel with [:, :, i], which feels like it can't be the most efficient way.

[jreback]

my example above

[max-sixty]

OIC, thank you

[terrytangyuan]

@jreback Could you close this?

[jreback]

hmm, actually I think this needs a trivial implementation for .align, e.g. if all of the axes are the same then it should just return.

[jorisvandenbossche]

Since this is a regression, it would be nice to fix this for 0.19.0. However, I am not familiar with Panel, @MaximilianR or @jreback is this an easy fix?

[jreback]

closing as Panels are deprecated