Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError(s) if .loc[] is passed a list with missing labels on a non-unique index of Sparse object #20766

Closed
toobaz opened this issue Apr 20, 2018 · 1 comment · Fixed by #28425
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Sparse Sparse Data Type

Comments

@toobaz
Copy link
Member

toobaz commented Apr 20, 2018

Code Sample, a copy-pastable example if possible

In [2]: s = pd.Series(-1, index=list('aba')).to_sparse()

In [3]: s.loc[['a', 'c']]
/usr/bin/ipython3:1: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  #! /bin/sh
Out[3]: ---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~/.local/lib/python3.5/site-packages/IPython/core/formatters.py in __call__(self, obj)
    700                 type_pprinters=self.type_printers,
    701                 deferred_pprinters=self.deferred_printers)
--> 702             printer.pretty(obj)
    703             printer.flush()
    704             return stream.getvalue()

~/.local/lib/python3.5/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    393                             if callable(meth):
    394                                 return meth(obj, self, cycle)
--> 395             return _default_pprint(obj, self, cycle)
    396         finally:
    397             self.end_group()

~/.local/lib/python3.5/site-packages/IPython/lib/pretty.py in _default_pprint(obj, p, cycle)
    508     if _safe_getattr(klass, '__repr__', None) is not object.__repr__:
    509         # A user-provided repr. Find newlines and replace them with p.break_()
--> 510         _repr_pprint(obj, p, cycle)
    511         return
    512     p.begin_group(1, '<')

~/.local/lib/python3.5/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    699     """A pprint that just redirects to the normal repr function."""
    700     # Find newlines and replace them with p.break_()
--> 701     output = repr(obj)
    702     for idx,output_line in enumerate(output.splitlines()):
    703         if idx:

/home/nobackup/repo/pandas/pandas/core/base.py in __repr__(self)
     80         Yields Bytestring in Py2, Unicode String in py3.
     81         """
---> 82         return str(self)
     83 
     84 

/home/nobackup/repo/pandas/pandas/core/base.py in __str__(self)
     59 
     60         if compat.PY3:
---> 61             return self.__unicode__()
     62         return self.__bytes__()
     63 

/home/nobackup/repo/pandas/pandas/core/sparse/series.py in __unicode__(self)
    261         series_rep = Series.__unicode__(self)
    262         rep = '{series}\n{index!r}'.format(series=series_rep,
--> 263                                            index=self.sp_index)
    264         return rep
    265 

/home/nobackup/repo/pandas/pandas/core/generic.py in __getattr__(self, name)
   4376             if name in self._info_axis:
   4377                 return self[name]
-> 4378             return object.__getattribute__(self, name)
   4379 
   4380     def __setattr__(self, name, value):

/home/nobackup/repo/pandas/pandas/core/sparse/series.py in sp_index(self)
    200     @property
    201     def sp_index(self):
--> 202         return self.block.sp_index
    203 
    204     @property

AttributeError: 'FloatBlock' object has no attribute 'sp_index'

In [4]: df = pd.DataFrame(-1, index=list('abc'), columns=list('xyx')).to_sparse()

In [5]: df.loc[:, ['x', 'w']]
/usr/bin/ipython3:1: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  #! /bin/sh
/home/nobackup/repo/pandas/pandas/core/indexing.py:1368: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  return self._getitem_tuple(key)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-56a2111d1155> in <module>()
----> 1 df.loc[:, ['x', 'w']]

/home/nobackup/repo/pandas/pandas/core/indexing.py in __getitem__(self, key)
   1366             except (KeyError, IndexError):
   1367                 pass
-> 1368             return self._getitem_tuple(key)
   1369         else:
   1370             # we by definition only have the 0th axis

/home/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_tuple(self, tup)
    875                 continue
    876 
--> 877             retval = getattr(retval, self.name)._getitem_axis(key, axis=i)
    878 
    879         return retval

/home/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1829                     raise ValueError('Cannot index with multidimensional key')
   1830 
-> 1831                 return self._getitem_iterable(key, axis=axis)
   1832 
   1833             # nested tuple slicing

/home/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_iterable(self, key, axis)
   1156                     result = result._reindex_with_indexers(
   1157                         {axis: [new_target, new_indexer]},
-> 1158                         copy=True, allow_dups=True)
   1159 
   1160                 else:

/home/nobackup/repo/pandas/pandas/core/sparse/frame.py in _reindex_with_indexers(self, reindexers, method, fill_value, limit, copy, allow_dups)
    731                                                 fill_value=fill_value)
    732             else:
--> 733                 new_arrays[col] = self[col]
    734 
    735         return self._constructor(new_arrays, index=index,

/home/nobackup/repo/pandas/pandas/core/sparse/frame.py in __getitem__(self, key)
    439             return self._getitem_array(key)
    440         else:
--> 441             return self._get_item_cache(key)
    442 
    443     def get_value(self, index, col, takeable=False):

/home/nobackup/repo/pandas/pandas/core/generic.py in _get_item_cache(self, item)
   2482         if res is None:
   2483             values = self._data.get(item)
-> 2484             res = self._box_item_values(item, values)
   2485             cache[item] = res
   2486             res._set_as_cached(item, self)

/home/nobackup/repo/pandas/pandas/core/frame.py in _box_item_values(self, key, values)
   3047         items = self.columns[self.columns.get_loc(key)]
   3048         if values.ndim == 2:
-> 3049             return self._constructor(values.T, columns=items, index=self.index)
   3050         else:
   3051             return self._box_col_values(values, items)

AttributeError: 'BlockManager' object has no attribute 'T'

Problem description

Despite the two different errors, I guess they are related.

Probably related to #14427

Might be also related to

Expected Output

In [6]: s.to_dense().loc[['a', 'c']]
/usr/bin/ipython3:1: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  #! /bin/sh
Out[6]: 
a   -1.0
a   -1.0
c    NaN
dtype: float64

In [10]: df.to_dense().loc[:, ['x', 'w']]
/usr/bin/ipython3:1: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  #! /bin/sh
/home/nobackup/repo/pandas/pandas/core/indexing.py:1368: FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
  return self._getitem_tuple(key)
Out[10]: 
   x  x   w
a -1 -1 NaN
b -1 -1 NaN
c -1 -1 NaN

Output of pd.show_versions()

INSTALLED VERSIONS

commit: d04b746
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-6-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: it_IT.UTF-8

pandas: 0.23.0.dev0+754.gd04b7464d
pytest: 3.5.0
pip: 9.0.1
setuptools: 39.0.1
Cython: 0.25.2
numpy: 1.14.1
scipy: 0.19.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.5.6
patsy: 0.5.0
dateutil: 2.7.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.0dev
tables: 3.3.0
numexpr: 2.6.1
feather: 0.3.1
matplotlib: 2.0.0
openpyxl: 2.3.0
xlrd: 1.0.0
xlwt: 1.3.0
xlsxwriter: 0.9.6
lxml: 4.1.1
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: 0.2.1

@toobaz toobaz added Sparse Sparse Data Type Indexing Related to indexing on series/frames, not to indexes themselves labels Apr 20, 2018
@toobaz toobaz changed the title AttributeError(s) if .loc[] is passed a list with missing labels on a non-unique index of Sparse AttributeError(s) if .loc[] is passed a list with missing labels on a non-unique index of Sparse object Apr 20, 2018
@TomAugspurger
Copy link
Contributor

Can't reproduce on master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Sparse Sparse Data Type
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants