r17 standalone toolchain's -static-libstdc++ flag links arm32 libdl statically (breaks ASAN, dlopen)

[rprichard]

This error seems to happen in a very specific situation:

test.cpp:

int main() {}

Commands:

export NDK=/ssd2/android-ndk-r17
$NDK/build/tools/make_standalone_toolchain.py --arch arm --api 16 --install-dir stand --force
stand/bin/clang++ -pie -fpie -fsanitize=address -std=c++11 hello.cpp  -static-libstdc++
adb push a.out $NDK/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/6.0.2/lib/linux/libclang_rt.asan-arm-android.so /data/local/tmp
adb shell 'LD_LIBRARY_PATH=/data/local/tmp /data/local/tmp/a.out'

Output from adb:

==7530==AddressSanitizer CHECK failed: /usr/local/google/buildbot/src/android/llvm-r316199/toolchain/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:6280 "((__interception::real_memcpy)) != (0)" (0x0, 0x0)
    <empty stack>

Segmentation fault 

• NDK Version: 17.0.4754217 (r16b works)
• Build system: standalone toolchain (ndk-build works)
• Host OS: gLinux
• Compiler: Clang
• ABI: armeabi-v7a (arm64 works)
• STL: c++_static (c++_shared works)
• NDK API level: 16 (26 also fails)
• Device API level: 26

---

Edit: It doesn't work on device API 16, but the error there is different:

reloc_library[1306]: 12941 cannot locate '__register_atfork'...
CANNOT LINK EXECUTABLE

[enh]

more useful as an internal bug so you can assign it to eugenis?

[DanAlbert]

@eugenis might be assignable here too. If not, add him to the org.

[DanAlbert]

(he might prefer a buganizer bug anyway since that's probably his usual workflow)

[eugenis]

Here is fine.

ASan never supported api 16: https://github.com/google/sanitizers/wiki/AddressSanitizerOnAndroid
Supported started from api 19, but I'm not sure how KitKat is doing these days. Realistically, I'd start with api 22.

I'll take a look at the problem on api 26 device.

[DanAlbert]

Yeah, the reason for the __register_atfork failure is because the sanitizer libs are built against a more recent API level. Although that function was actually added in M, so maybe the minimum supported version for ASAN has moved up?

[eugenis]

We support at least L.

…

On Mon, May 14, 2018 at 3:05 PM, Dan Albert ***@***.***> wrote: Yeah, the reason for the __register_atfork failure is because the sanitizer libs are built against a more recent API level. Although that function was actually added in M, so maybe the minimum supported version for ASAN has moved up? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#692 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAZuShZkfruvVVZMCv5iBlctHsM1rcK2ks5tyf_HgaJpZM4T6uaj> .

[DanAlbert]

The asan runtime agrees:

$ ./parse_elfnote.py ~/src/android-ndk-r17/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/6.0.2/lib/linux/libclang_rt.asan-arm-android.so 
----------ABI INFO----------
ABI_ANDROID_API: 14
ABI_NDK_VERSION: r16b
ABI_NDK_BUILD_NUMBER: 4479499

@rprichard: not sure how you got that error, but it may be worth investigating, as it hints that it was built for M+.

[rprichard]

I tried my test case again, and I don't see the __register_atfork error anymore on my API 16 device. I still see the __interception::real_memcpy error, though, on an API 26 device.

[rprichard]

From r16b to r17, we started including -ldl in the libstdc++.a text file in the standalone toolchain for libc++.

Clang passes these arguments to the linker: -Bstatic -lstdc++ -Bdynamic.

The linker then pulls in libdl functions from libdl_static.a, which are exported from the executable.

 .text.dlopen   0x000022ec        0x4 stand/bin/../sysroot/usr/lib/../lib/libdl.a(libdl_static.o)
                0x00000001                dlopen

$ nm -D hello | grep dl
000022fd T dladdr
00002301 T dlclose
000022f1 T dlerror
000022ed T dlopen
000022f5 T dlsym
00002309 T dl_unwind_find_exidx
000022f9 T dlvsym

The dl* functions are also exported if I build a shared object using the standalone toolchain.

I think the libdl APIs are broken in this configuration:

• standalone toolchain
• --stl=libc++ (the default)
• -static-libstdc++
• arm32

Test case:

dltest.cpp:

#include <dlfcn.h>
#include <stdio.h>
int main() {
  void *lib = dlopen("libc.so", RTLD_NOW);
  printf("lib=%p\n", lib);
  return 0;
}

export NDK=/ssd2/android-ndk-r17
$NDK/build/tools/make_standalone_toolchain.py --arch arm --api 16 --install-dir stand --force
stand/bin/clang++ -pie -fpie dltest.cpp  -static-libstdc++ -o dltest
adb push dltest /data/local/tmp
adb shell 'LD_LIBRARY_PATH=/data/local/tmp /data/local/tmp/dltest'

Output:

lib=0x0

The pointer should be non-NULL, and it is with NDK r16b.

IIRC, arm32's libunwind needs libdl, so that's why we're specifying it, but the driver includes libdl automatically, so maybe we could omit it.

[rprichard]

The -ldl flag was added here: https://android-review.googlesource.com/c/platform/ndk/+/609153.

[DanAlbert]

https://android-review.googlesource.com/c/platform/ndk/+/691335