Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicitly set dtype of np.lexsort in group_rank #19679

Merged
merged 1 commit into from
Feb 13, 2018

Conversation

WillAyd
Copy link
Member

@WillAyd WillAyd commented Feb 13, 2018

xref #19481

I didn't see any other instance in group_rank where the dtype was open to interpretation, so I'm wondering if np.lexsort is returning a plain int on 32 bit systems. Hoping that explicitly getting a view of that np.lexsort result to match the dtype of _as will resolve the test issues

@@ -531,7 +531,7 @@ def group_rank_{{name}}(ndarray[float64_t, ndim=2] out,
# each label corresponds to a different group value,
# the mask helps you differentiate missing values before
# performing sort on the actual values
_as = np.lexsort(order)
_as = np.lexsort(order).view(dtype=np.int64)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this prob returns a platform int. this is ok

@jreback jreback added Dtype Conversions Unexpected or buggy dtype conversions Compat pandas objects compatability with Numpy or Python functions labels Feb 13, 2018
@jreback jreback added this to the 0.23.0 milestone Feb 13, 2018
@jreback
Copy link
Contributor

jreback commented Feb 13, 2018

ping on green.

@codecov
Copy link

codecov bot commented Feb 13, 2018

Codecov Report

Merging #19679 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #19679   +/-   ##
=======================================
  Coverage   91.58%   91.58%           
=======================================
  Files         150      150           
  Lines       48864    48864           
=======================================
  Hits        44752    44752           
  Misses       4112     4112
Flag Coverage Δ
#multiple 89.95% <ø> (ø) ⬆️
#single 41.75% <ø> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df38f66...fe5245b. Read the comment docs.

@WillAyd
Copy link
Member Author

WillAyd commented Feb 13, 2018

@jreback should be good to go. Couple failures on the optional Travis tests that appear unrelated

@jreback jreback merged commit 49812cf into pandas-dev:master Feb 13, 2018
@jreback
Copy link
Contributor

jreback commented Feb 13, 2018

yeah these are actually NOT optional. they seem to be failing very recently. @TomAugspurger any idea what is happening?

@WillAyd WillAyd deleted the rank-int-issue branch February 13, 2018 23:57
@jreback
Copy link
Contributor

jreback commented Feb 14, 2018

This is segaulting: https://travis-ci.org/MacPython/pandas-wheels/jobs/341233134

can you install a 32-bit version of miniconda and see if you can track the problem?

dl from https://repo.continuum.io/miniconda/

@TomAugspurger
Copy link
Contributor

https://travis-ci.org/pandas-dev/pandas/jobs/341072093 for the failure about

    Exception: Cython-generated file 'pandas/_libs/algos.c' not found.
                    Cython is required to compile pandas from a development branch.
                    Please install Cython or download a release package of pandas.

I don't know why https://travis-ci.org/pandas-dev/pandas/jobs/341072091 timed out. Something in our build step seems to sometimes take a way longer time.

@WillAyd
Copy link
Member Author

WillAyd commented Feb 14, 2018

I installed a 32 bit version of conda on my system, but can't seem to import pandas to debug. When trying to build I get the same error that Tom is getting above, and on import I get the below - is there an easy way to configure the development environment for a 32 bit installation to match what the Travis builds are doing?

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/__init__.py", line 19, in <module>
    "Missing required dependencies {0}".format(missing_dependencies))
ImportError: Missing required dependencies ['pytz']
# packages in environment at /Users/williamayd/miniconda3_32/envs/py27_32:
#
alabaster                 0.7.3                    py27_0  
babel                     2.0                      py27_0  
cython                    0.23.1                   py27_0  
docutils                  0.12                     py27_0  
jinja2                    2.8                      py27_0  
markupsafe                0.23                     py27_0  
numpy                     1.9.2                    py27_0  
openssl                   1.0.1k                        1  
ortools-examples          6.4.4495                  <pip>
pip                       7.1.2                    py27_0  
py                        1.5.2                      py_0  
pygments                  2.0.2                    py27_0  
pytest                    3.2.2                      py_0  
python                    2.7.10                        0  
python-dateutil           2.4.2                    py27_0  
pytz                      2018.3                     py_0  
readline                  6.2                           2  
setuptools                18.1                     py27_0  
six                       1.9.0                    py27_0  
snowballstemmer           1.2.0                    py27_0  
sphinx                    1.3.1                    py27_0  
sphinx-rtd-theme          0.1.7                     <pip>
sphinx_rtd_theme          0.1.7                    py27_0  
sqlite                    3.8.4.1                       1  
tk                        8.5.18                        0  
wheel                     0.24.0                   py27_0  
zlib                      1.2.8                         0

@WillAyd
Copy link
Member Author

WillAyd commented Feb 14, 2018

Alright I was able to get an environment configured using pip instead of conda. FWIW I don't think the latter is a possibility to use. At the very least, the latest osx32 python-dateutil package on anaconda is 2.4.2 but pandas requires at least 2.5.0

@jreback
Copy link
Contributor

jreback commented Feb 14, 2018

right but can alwyas pip install dateutil

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Feb 14, 2018 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants