--replace-needed calls corrupt these executables

[sztomi]

patchelf version: Built from 27ffe8a

In our build process, we try to remove all version numbers from the shared objects names of our various dependencies. We use patchelf to fix the referenced sonames everywhere. This seems to work well for the most part, but we noticed one particular example where it corrupts the executable.

In the first case, there were no version numbers in the sonames, so the rewriting is redundant (i.e. we can avoid it). Download the binary here

$ patchelf --print-needed ./ffmpeg
libavfilter.so
libavformat.so
libavcodec.so
libswresample.so
libswscale.so
libavutil.so
libm.so
libc.so

$ patchelf --replace-needed 'libavfilter.so' 'libavfilter.so' --replace-needed 'libavformat.so' 'libavformat.so' --replace-needed 'libavcodec.so' 'libavcodec.so' --replace-needed 'libswresample.so' 'libswresample.so' --replace-needed 'libswscale.so' 'libswscale.so' --replace-needed 'libavutil.so' 'libavutil.so' ./ffmpeg

$ patchelf --print-needed ./ffmpeg
libavutil.so
bavcodec.so
avformat.so
libswresample.so
bswscale.so
bavfilter.so
libm.so
libc.so

Notice how libavcodec.so became bavcodec.so etc.

In the second case, there are numbers to strip. Download the binary here

$ patchelf --print-needed ./ffmpeg2
libavfilter.so.7
libavformat.so.58
libavcodec.so.58
libswresample.so.3
libswscale.so.5
libavutil.so.56
libm.so.6
libpthread.so.0
libc.so.6

$ patchelf --replace-needed 'libavfilter.so.7' 'libavfilter.so' --replace-needed 'libavformat.so.58' 'libavformat.so' --replace-needed 'libavcodec.so.58' 'libavcodec.so' --replace-needed 'libswresample.so.3' 'libswresample.so' --replace-needed 'libavutil.so.56' 'libavutil.so' --replace-needed 'libm.so.6' 'libm.so' --replace-needed 'libpthread.so.0' 'libpthread.so' --replace-needed 'libc.so.6' 'libc.so' ./ffmpeg2

$ patchelf --print-needed ./ffmpeg2
libavfilter.so
libpthread.so
ibavcodec.so
ibswresample.so
libswscale.so.5
ibc.so
til.so
ibm.so
rmat.so

As you can see, the replacements are all over the place in this case. The issue is 100% reproducible with the binaries I posted. Let me know if I can provide more information.

[sztomi]

It seems that the released version (https://github.com/NixOS/patchelf/releases/tag/0.9) doesn't reproduce this issue. That should help with the troubleshooting, it's probably possible to bisect the offending commit.

We upgraded because we were experiencing other issues:

Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion 'needed != NULL' failed!

(I don't have the full build output now, unfortunately). This latter problem doesn't happen with the latest commit.

[sztomi]

It also works (with 27ffe8a) if I invoke patchelf multiple times with single --replace-needed parameters. But this has an unwanted side effect of shifting the headers which makes the binary unusable on FreeBSD due to this upstream bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229708

[Jikstra]

Tried to patch a weird .node elf library whatever and got a similiar error Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assertion \needed != NULL' failed!`. I'm using version 0.9 from arch repos if that helps, can provide binaries of course too.

[sztomi]

@Jikstra try the commit I mentioned, maybe it will work in your case. Unfortunately it seems it has other issues in some cases (as in this very bug report). I don't know how active patchelf is, it seems there are no responses to the issues unfortunately. I recommend checking out the LIEF framework, it has an easy Python interface and can accomplish all of what patchelf does. I did run into a couple issues with that as well. It seems like this is just a really hard problem to get right. We steered away from binary patching as much as we could and instead we patch build systems to emit the right thing (when needed).

[Jikstra]

@sztomi thanks for your reply (it has been some days, came back to the exact same problem 2 weeks ago ^^) but sadly this commit also doesn't help.

[domenkozar]

It would be great if someone can try with latest master and report back what is left to fix here.

[sztomi]

@domenkozar the binaries I shared are still downloadable through the links in my first post. Happy to check for you, but it's probably more effective if you try those yourself (especially if the bug is still present).

[domenkozar]

Can still reproduce.

[sztomi]

@domenkozar in that case I suggest bisecting from the 0.9 tag (because that one didn't have this issue).

[mayeut]

Does #335 effectively fixes this (PR that replaced #237) ?

It seems to resolve a similar issue in pypa/auditwheel#401