Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix rpath setting on external libraries #454

Closed
wants to merge 2 commits into from
Closed

Fix rpath setting on external libraries #454

wants to merge 2 commits into from

Conversation

licy183
Copy link

@licy183 licy183 commented Sep 21, 2023

External libraries will be copied to sdir during the process. If the external library contains rpath, rpath will be removed and reset.

Since commit a9b4794, rpath will be set to the dest_dir if an external library originally contains rpath. It is quite weird and will let the loader (on Android) gives a warning message.

Original library:

$ readelf -d libopenblas.so
 0x000000000000001d (RUNPATH)            Library runpath: [/data/data/com.termux/files/usr/lib]

Library after processing:

$ readelf -d libopenblas-2e48b161.so
 0x000000000000001d (RUNPATH)            Library runpath: [numpy-libs]

The intended behavior:

$ readelf -d libopenblas-xxxxxx.so
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/.]

@codecov
Copy link

codecov bot commented Sep 21, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (af2c5a5) 92.30% compared to head (5c63fce) 92.30%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #454   +/-   ##
=======================================
  Coverage   92.30%   92.30%           
=======================================
  Files          23       23           
  Lines        1286     1287    +1     
  Branches      304      304           
=======================================
+ Hits         1187     1188    +1     
  Misses         57       57           
  Partials       42       42           
Files Changed Coverage Δ
src/auditwheel/repair.py 86.71% <100.00%> (+0.10%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lkollar
Copy link
Contributor

lkollar commented Sep 25, 2023

$ readelf -d libopenblas-2e48b161.so
0x000000000000001d (RUNPATH) Library runpath: [numpy-libs]

This definitely looks incorrect, but I don't fully understand the problem. Could you provide a full reproducer, like the unrepaired wheel as input?

@licy183
Copy link
Author

licy183 commented Sep 27, 2023

It is originally found on Termux (an Linux emulator on Android), but still reproducible on GNU/Linux.

  1. Compile OpenBLAS and set RPATH for it (to simulate a library with RPATH)
WORK_DIR=$(pwd)
mkdir -p test-prefix
wget https://github.com/OpenMathLib/OpenBLAS/releases/download/v0.3.24/OpenBLAS-0.3.24.tar.gz
tar xf OpenBLAS-0.3.24.tar.gz
pushd OpenBLAS-0.3.24
mkdir -p build
pushd build
cmake -DCMAKE_INSTALL_PREFIX=$WORK_DIR/test-prefix -DBUILD_SHARED_LIBS=ON ..
make install
popd # build
popd # OpenBLAS-0.3.24
patchelf --set-rpath "\$ORIGIN/../lib" ./test-prefix/lib/libopenblas.so
  1. Build numpy with custom openblas
wget https://github.com/numpy/numpy/releases/download/v1.26.0/numpy-1.26.0.tar.gz
tar xf numpy-1.26.0.tar.gz
cd numpy-1.26.0/
echo "[openblas]
libraries = openblas
library_dirs = $WORK_DIR/test-prefix/lib
include_dirs = $WORK_DIR/test-prefix/include
runtime_library_dirs = $WORK_DIR/test-prefix/lib" > site.cfg
python -m pip install cython
python setup.py bdist_wheel -v
  1. Audit wheel
python -m pip install auditwheel
python -m auditwheel repair ./dist/*.whl  --plat linux_x86_64
  1. See the result of numpy.libs/libopenblas-xxx.so and ./wheelhouse/numpy_xxx.whl
$ readelf -d libopenblas-f40dce46.so.0.3
Dynamic section at offset 0x1170000 contains 30 entries:
  Tag        Type                         Name/Value
 0x000000000000000f (RPATH)              Library rpath: [numpy.libs]
 0x0000000000000001 (NEEDED)             Shared library: [libgfortran-df5d6f8d.so.5.0.0]
$ readelf -d libopenblas.so

Dynamic section at offset 0x1133000 contains 30 entries:
  Tag        Type                         Name/Value
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib]
 0x0000000000000001 (NEEDED)             Shared library: [libgfortran.so.5]

The unrepaired wheel file: numpy-1.26.1.26.0-cp310-cp310-linux_x86_64.whl.zip

@mayeut
Copy link
Member

mayeut commented Feb 3, 2024

duplicated by #477 / #478

@mayeut mayeut closed this Feb 3, 2024
@licy183 licy183 deleted the fix-rpath branch February 4, 2024 04:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants