Segmentation fault: 11 on nosetests

[vfilimonov]

Nosetests results in the segmentation fault error:

$ nosetests pandas
................/Users/v/.virtual_envs/system/lib/python2.7/site-packages/pandas/core/index.py:910: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  result.sort()
................../Users/v/.virtual_envs/system/lib/python2.7/site-packages/numpy/core/fromnumeric.py:788: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  a.sort(axis, kind, order)
..................................................................................................................................................................................................................................F....................................S...S.....S..........................................SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS..................................................................................................................................................................................................................................................................................................................................................................S....................................................................................................................................S..........S......................................................................................................................................SSSS............................S................................................................SSSSSSSSSS......................SS.....................................................................................................................................................S...Segmentation fault: 11

It crashes on the following test:

...
test_binop_maybe_preserve_name (pandas.sparse.tests.test_sparse.TestSparseSeries) ... ok
test_combine_first (pandas.sparse.tests.test_sparse.TestSparseSeries) ... ok
test_combine_first_dt64 (pandas.sparse.tests.test_sparse.TestSparseSeries) ... ok
test_combine_first_name (pandas.sparse.tests.test_sparse.TestSparseSeries) ... Segmentation fault: 11

Operating system: OSX 10.9.1 Mavericks. Library versions:

>>> show_versions()

INSTALLED VERSIONS
------------------
Python: 2.7.6.final.0
OS: Darwin
Release: 13.0.0
Processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8

pandas: 0.13.0
Cython: 0.20
Numpy: 1.9.0.dev-783b96d
Scipy: 0.13.2
statsmodels: 0.5.0
    patsy: 0.2.1
scikits.timeseries: Not installed
dateutil: 2.1
pytz: 2013.9
bottleneck: 0.8.0
PyTables: 3.0.0
    numexpr: 2.2.2
matplotlib: 1.1.1
openpyxl: 1.8.2
xlrd: 0.9.2
xlwt: Not installed
xlsxwriter: Not installed
sqlalchemy: 0.9.1
lxml: 3.2.5
bs4: 4.3.2
html5lib: 0.999
bigquery: Not installed
apiclient: Not installed

[jreback]

related to #5997

dev numpy is not stable yet

u r adventurous for trying it!

pls follow that issue and report the same

[vfilimonov]

@jreback, right - 1.9-dev is unstable.

I was trying to chase one segfault in my own code (resampling empty dataframe that was retrieved from hdf5 file) which looks very similar to this issue
https://stackoverflow.com/questions/19722580/segfault-11-with-pandas-with-python-v2-7-6-rc1-on-mac-os-x-10-9
and the solution was to update to dev version of numpy.

p.s. just tested on numpy-1.8.0 - there's no segfaults

[jreback]

ok....will close this issue (but I cross-linked)...thanks for reporting....numpy apparently changed some things which happen to show up in pandas tests as segfaults!

[jreback]

@vfilimonov

ps resample of empty frame should work, but I suspect that the dtypes are wrong (when retrieving from HDF)

can you provide a working example (open a new issue) - its prob not setting the empty dtypes correctly

[vfilimonov]

@jreback, sure. I'm trying to create a minimal-example to reproduce the problem. When I have it I will definitely open an issue.

In fact, I'm not 100% sure yet that it is about resample. Segfault is all about the environment - and slight restructuring changes the error (e.g. in my case even some prints result in passing those iterations where program crashed before). So I'm trying to locate the problem to report it.

[juliantaylor]

a recent improvement to np.where in numpy 1.9.dev breaks for object arrays, it should be fixed by this PR which will hopefully be merged soon:
numpy/numpy#4215
maybe you can already try if it works for you.

[jtratner]

Does it fail with non-dev numpy?

[vfilimonov]

@jtratner no - with numpy-1.8.0 there's no segfaults for nosetests

[jtratner]

that email was apparently queued for hours past where it made sense - sorry
about that!