RPATH goes missing when using both install_rpath and an internal shared library dependency

[rgommers]

This is a case that probably hasn't been exercised before: a Python extension module that links against both a shared library that's part of the package and a shared library built in a subproject that's folded into the wheel by meson-python.

I ran into this in SciPy when adding a subproject. This worked as advertised, except for this scipy.special._ufuncs extension module, which depends on this libsf_error shared library as well as on the library in the subproject.
What one sees then is that the installed wheel can no longer find libsf_error.so, because the RPATH entry for install_rpath: '$ORIGIN' goes missing.

With a default SciPy build (no subproject), the Library rpath|runpath entries on the _ufuncs target in the build and install directories looks like this (has $ORIGIN as it should):

$ readelf -d build/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so | rg "Library r"
 0x000000000000000f (RPATH)              Library rpath: [/home/rgommers/mambaforge/envs/scipy-dev-py312/lib:$ORIGIN/]

$ readelf -d build-install/lib/python3.12/site-packages/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so | rg "Library r"
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN:/home/rgommers/mambaforge/envs/scipy-dev-py312/lib]

On the branch with the subproject, we see this instead:

$ readelf -d build-whl/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so | rg Library
 0x000000000000001d (RUNPATH)            Library runpath: [/home/rgommers/code/pixi-dev-scipystack/scipy/.pixi/envs/openblas-src/lib:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs]

$ readelf -d site-packages/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so | rg "Library r"
 0x000000000000001d (RUNPATH)            Library runpath: [/home/rgommers/code/pixi-dev-scipystack/scipy/.pixi/envs/openblas-src/lib:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs:$ORIGIN/../../.scipy.mesonpy.libs]

The duplication of the .scipy.mesonpy.libs entries is a minor stylistic issue, we may be able to de-duplicate but it doesn't matter. What does matter is that $ORIGIN is no longer present.

This looks like a bug in the fix_rpath implementation at:

meson-python/mesonpy/_rpath.py

Lines 28 to 36 in 91d64d1

	def fix_rpath(filepath: Path, libs_relative_path: str) -> None:
	old_rpath = _get_rpath(filepath)
	new_rpath = []
	for path in old_rpath:
	if path.startswith('$ORIGIN/'):
	path = '$ORIGIN/' + libs_relative_path
	new_rpath.append(path)
	if new_rpath != old_rpath:
	_set_rpath(filepath, new_rpath)

The problem seems clear: all RPATH entries are cleared, and the ones that are added back all get libs_relative_path appended (i.e. to the subproject-relocated location). Any existing RPATHs within the package, like '$ORIGIN', will get lost.

[rgommers]

Adding the relevant content from the build-whl/meson-info/intro-install_plan.json install metadata file:

{
  "targets": {
    "/home/.../build-whl/subprojects/openblas/lapack-netlib/SRC/libscipy_lapack.so": {
      "destination": "{libdir_shared}/libscipy_lapack.so",
      "tag": "runtime",
      "subproject": "openblas",
      "install_rpath": null
    },

    "/home/.../build-whl/scipy/special/libsf_error_state.so": {
      "destination": "{py_platlib}/scipy/special/libsf_error_state.so",
      "tag": "runtime",
      "subproject": null,
      "install_rpath": null
    },

    "/home/.../build-whl/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so": {
      "destination": "{py_platlib}/scipy/special/_ufuncs.cpython-312-x86_64-linux-gnu.so",
      "tag": "runtime",
      "subproject": null,
      "install_rpath": "$ORIGIN"
    },

[rgommers]

I think the correct behavior for fix_rpath is:

1. If the install target has no RPATH entries, do nothing

2. If the install target has >=1 RPATH entries, then:

   • Add a single new relative entry pointing to .<project-name>.mesonpy.libs
   • Remove entries starting with $ORIGIN but pointing outside of the package's own tree and into subprojects (e.g., $ORIGIN/../../subprojects). Those are the ones being relocated.
   • Keep other entries that start with $ORIGIN unchanged
   • Keep other entries that don't start with $ORIGIN also unchanged (that is already the case)