Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression in tests from #95 - numpy 1.11.0 is no longer matched to manylinux #107

Closed
ehashman opened this issue Oct 18, 2018 · 2 comments · Fixed by #110
Closed

Regression in tests from #95 - numpy 1.11.0 is no longer matched to manylinux #107

ehashman opened this issue Oct 18, 2018 · 2 comments · Fixed by #110
Assignees

Comments

@ehashman
Copy link
Member

The external libraries showing up in this map with "null" keypairs is causing legitimate wheels to be misidentified as non-manylinux. We missed this because the tests appeared to be passing, due to #105.

@ehashman
Copy link
Member Author

Just to test my theory that #95 was the source of the regression, I reverted the merge commit and ran the tests against that. They pass again, so something in the logic of that PR broke things. I'm going to use that knowledge to try to hunt down and fix the bug, but if I can't find it are you okay with me reverting the PR @lpsinger? Mainline auditwheel seems to be broken, I'm not sure why no one noticed since I released the changes in 1.9...

@ehashman
Copy link
Member Author

Okay, I think I've tracked this down. For context, the test failure looks like:

numpy-1.11.0-cp35-cp35m-manylinux1_x86_64.whl is consistent with the
following platform tag: "linux_x86_64".

The wheel references external versioned symbols in these system-
provided shared libraries: libc.so.6 with versions {'GLIBC_2.2.5',
'GLIBC_2.4', 'GLIBC_2.3.2', 'GLIBC_2.3'}, libm.so.6 with versions
{'GLIBC_2.2.5'}, libpthread.so.0 with versions {'GLIBC_2.2.5'}

The following external shared libraries are required by the wheel:
{
    "libatlas-dd66cbe8.so.3.0": null,
    "libc.so.6": "/lib64/libc-2.5.so",
    "libcblas-de16e930.so.3.0": null,
    "libf77blas-60dfc405.so.3.0": null,
    "libgcc_s.so.1": "/lib64/libgcc_s-4.1.2-20080825.so.1",
    "libgfortran-bfa58d52.so.1.0.0": null,
    "libm.so.6": "/lib64/libm-2.5.so",
    "libpthread.so.0": "/lib64/libpthread-2.5.so"
}

In order to achieve the tag platform tag "manylinux1_x86_64" the
following shared library dependencies will need to be eliminated:

libatlas-dd66cbe8.so.3.0, libcblas-de16e930.so.3.0,
libf77blas-60dfc405.so.3.0, libgfortran-bfa58d52.so.1.0.0

Since the result of #95 is that auditwheel looks at all ELF files in the wheel, it seems that we end up misprocessing the repaired ELF files that we grafted during the repair step into .libs. So we end up walking something like this:

DEBUG:auditwheel.wheel_abi:{
    "numpy/.libs/libptf77blas-536e8f3e.so.3.0": {
        "rpath": [],
        "libs": {
            "libc.so.6": {
                "needed": [
                    "ld-linux-x86-64.so.2"
                ],
                "realpath": "/lib64/libc-2.5.so",
                "path": "/lib64/libc.so.6"
            },
            "libgfortran-bfa58d52.so.1.0.0": {
                "needed": [],
                "realpath": null,
                "path": null
            },
            "libatlas-dd66cbe8.so.3.0": {
                "needed": [],
                "realpath": null,
                "path": null
            },
            "ld-linux-x86-64.so.2": {
                "needed": [],
                "realpath": "/lib64/ld-2.5.so",
                "path": "/lib64/ld-linux-x86-64.so.2"
            }
        },
        "realpath": "numpy/.libs/libptf77blas-536e8f3e.so.3.0",
        "path": "numpy/.libs/libptf77blas-536e8f3e.so.3.0",
        "needed": [
            "libatlas-dd66cbe8.so.3.0",
            "libgfortran-bfa58d52.so.1.0.0",
            "libc.so.6"
        ],
        "runpath": [],
        "interp": null
    },

And as you can see, the realpath and path of our vendored libraries are null, causing these to be misidentified as failing policy with the mysterious null values indicated above. I'm not quite sure how this happens yet, but perhaps the solution is to ignore things we know to have been repaired by auditwheel or similar...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant