ld reports relocation error with -fPIC and -fno-integrated-as on MIPS64

[yan12125]

Description

I'm building cpython, which uses libffi for calling external C functions. On NDK r15 beta 2, building for MIPS64 with -fintegrated-as fails during linking

/home/travis/android-ndk-r15-beta2/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -shared -target mips64el-none-linux-android -gcc-toolchain /home/travis/android-ndk-r15-beta2/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64 --sysroot=/home/travis/android-ndk-r15-beta2/platforms/android-21/arch-mips64/usr -pie -L/home/travis/build/yan12125/python3-android/build/target/usr/lib build/temp.linux-mips64el-3.7/home/travis/pybuild-src/cpython/Modules/_ctypes/_ctypes.o build/temp.linux-mips64el-3.7/home/travis/pybuild-src/cpython/Modules/_ctypes/callbacks.o build/temp.linux-mips64el-3.7/home/travis/pybuild-src/cpython/Modules/_ctypes/callproc.o build/temp.linux-mips64el-3.7/home/travis/pybuild-src/cpython/Modules/_ctypes/stgdict.o build/temp.linux-mips64el-3.7/home/travis/pybuild-src/cpython/Modules/_ctypes/cfield.o -L/home/travis/build/yan12125/python3-android/build/target/usr/lib -lffi -o build/lib.linux-mips64el-3.7/_ctypes.cpython-37m.so
clang: �[0;1;35mwarning: �[0margument unused during compilation: '-pie' [-Wunused-command-line-argument]�[0m
/home/travis/android-ndk-r15-beta2/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin/ld: /home/travis/build/yan12125/python3-android/build/target/usr/lib/libffi.a(n32.o): relocation (null) against `ffi_closure_mips_inner_N32' can not be used when making a shared object; recompile with -fPIC
/home/travis/build/yan12125/python3-android/build/target/usr/lib/libffi.a: error adding symbols: Bad value
clang: �[0;1;31merror: �[0mlinker command failed with exit code 1 (use -v to see invocation)�[0m

I indeed built libffi with -fPIC:

libtool: compile:  /home/travis/android-ndk-r15-beta2/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -target mips64el-none-linux-android -gcc-toolchain /home/travis/android-ndk-r15-beta2/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64 --sysroot=/home/travis/android-ndk-r15-beta2/sysroot/usr -I/home/travis/android-ndk-r15-beta2/sysroot/usr/include/mips64el-linux-android -D__ANDROID_API__=21 -I/home/travis/build/yan12125/python3-android/build/target/usr/include -I. -I../include -Iinclude -I../src -fPIC -fintegrated-as -MT src/mips/n32.lo -MD -MP -MF src/mips/.deps/n32.Tpo -c ../src/mips/n32.S -o src/mips/n32.o

If I rebuild everything with -fno-integrated-as, linking is fine.

Complete log can be fetched from https://travis-ci.org/yan12125/python3-android/jobs/233650249. I haven't had time to create a standalone test case. If you need one, please tell me.
The offending symbol reference can be found in https://github.com/libffi/libffi/blob/master/src/mips/n32.S, line 451 and line 485.

I'm not sure whether it's a regression after the clang update or not. I didn't use -fno-integrated-as or -fintegrated-as for MIPS64 before #399 hit me.

Environment Details

Not all of these will be relevant to every bug, but please provide as much
information as you can.

• NDK Version: 15.0.3996722-beta2
• Build sytem: custom
• Host OS: Arch Linux x86_64
• Compiler: Clang
• ABI: MIPS64
• STL: N/A
• NDK API level: 21
• Device API level: N/A; building issues

[yan12125]

OK, I got a standalone test case. First create a file n32.S:

 .abicalls

 .set mips4
 .text
 .align 2
 .globl ffi_go_closure_N32
 .ent ffi_go_closure_N32
ffi_go_closure_N32:
.LFB1:
 .frame $sp, (20 * 8), $31
 .mask 0x90000000,-((20 * 8) - (1 * 8))
 .fmask 0x00000000,0
 dsubu $sp, (20 * 8)
.LCFI10:
 .cpsetup $25, (0 * 8), ffi_go_closure_N32
 sd $31, (1 * 8)($sp) # Save return address
.LCFI11:

 sd $4, (12 * 8)($sp)
 sd $5, (13 * 8)($sp)
 sd $6, (14 * 8)($sp)
 sd $7, (15 * 8)($sp)
 sd $8, (16 * 8)($sp)
 sd $9, (17 * 8)($sp)

 # Call ffi_closure_mips_inner_N32 to do the real work.

 dla $25, ffi_closure_mips_inner_N32
 ld $4, 8($15) # cif
 ld $5, 16($15) # fun
 move $6, $15 # userdata=$10
 daddu $7, $sp, (2 * 8) # rvalue
 daddu $8, $sp, (12 * 8) # ar
 daddu $9, $sp, (4 * 8) # fpr

 b $do_closure

.LFE1:
 .end ffi_go_closure_N32

And compile this file with:

$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -target mips64el-none-linux-android -gcc-toolchain $NDK/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64 --sysroot=$NDK/platforms/android-21/arch-mips64/usr -shared -fPIC n32.S -fintegrated-as -o n32.so

On my machine the result is:

$ /opt/android-ndk-beta/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -v -target mips64el-none-linux-android -gcc-toolchain /opt/android-ndk-beta/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64 --sysroot=/opt/android-ndk-beta/platforms/android-21/arch-mips64/usr -shared -fPIC n32.S -fintegrated-as -o n32.so
Android clang version 5.0.300080  (based on LLVM 5.0.300080)
Target: mips64el-none-linux-android
Thread model: posix
InstalledDir: /opt/android-ndk-beta/toolchains/llvm/prebuilt/linux-x86_64/bin
Found candidate GCC installation: /opt/android-ndk-beta/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/mips64el-linux-android/4.9.x
Selected GCC installation: /opt/android-ndk-beta/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/mips64el-linux-android/4.9.x
Candidate multilib: .;@march=mips64r6
Candidate multilib: 32/mips-r1;@march=mips32
Candidate multilib: 32/mips-r2;@march=mips32r2
Candidate multilib: 32/mips-r6;@march=mips32r6
Selected multilib: .;@march=mips64r6
 "/opt/android-ndk-beta/toolchains/llvm/prebuilt/linux-x86_64/bin/clang" -cc1 -triple mips64el-none-linux-android -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name n32.S -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fmath-errno -mconstructor-aliases -fuse-init-array -target-cpu mips64r6 -target-abi n64 -mfloat-abi hard -target-linker-version 2.24 -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /opt/android-ndk-beta/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/5.0.300080 -isysroot /opt/android-ndk-beta/platforms/android-21/arch-mips64/usr -internal-isystem /opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/usr/local/include -internal-isystem /opt/android-ndk-beta/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/5.0.300080/include -internal-externc-isystem /opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/include -internal-externc-isystem /opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/usr/include -fdebug-compilation-dir /home/yen/Projects/python3-android/src/libffi/mips64el-unknown-linux-android -ferror-limit 19 -fmessage-length 136 -femulated-tls -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/n32-f5521b.s -x assembler-with-cpp n32.S
clang -cc1 version 5.0.300080 based upon LLVM 5.0.300080 default target x86_64-unknown-linux
ignoring nonexistent directory "/opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/usr/local/include"
ignoring nonexistent directory "/opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/android-ndk-beta/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/5.0.300080/include
 /opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/include
End of search list.
 "/opt/android-ndk-beta/toolchains/llvm/prebuilt/linux-x86_64/bin/clang" -cc1as -triple mips64el-none-linux-android -filetype obj -main-file-name n32.S -target-cpu mips64r6 -fdebug-compilation-dir /home/yen/Projects/python3-android/src/libffi/mips64el-unknown-linux-android -dwarf-debug-producer Android clang version 5.0.300080  (based on LLVM 5.0.300080) -dwarf-version=4 -mrelocation-model pic -target-abi n64 -o /tmp/n32-9d561c.o /tmp/n32-f5521b.s
 "/opt/android-ndk-beta/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin/ld" --sysroot=/opt/android-ndk-beta/platforms/android-21/arch-mips64/usr --sysroot=/opt/android-ndk-beta/platforms/android-21/arch-mips64/usr --eh-frame-hdr -m elf64ltsmip -shared -o n32.so /opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/lib/../lib64/crtbegin_so.o -L/opt/android-ndk-beta/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/5.0.300080/lib/linux/mips64el -L/opt/android-ndk-beta/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/mips64el-linux-android/4.9.x -L/opt/android-ndk-beta/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/lib/../lib64 -L/opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/lib/../lib64 -L/opt/android-ndk-beta/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/lib -L/opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/lib /tmp/n32-9d561c.o -lgcc -ldl -lc -lgcc -ldl /opt/android-ndk-beta/platforms/android-21/arch-mips64/usr/lib/../lib64/crtend_so.o
/opt/android-ndk-beta/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin/ld: /tmp/n32-9d561c.o: relocation (null) against `ffi_closure_mips_inner_N32' can not be used when making a shared object; recompile with -fPIC
/tmp/n32-9d561c.o: error adding symbols: Bad value
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[DanAlbert]

MIPS is deprecated as of r16.

[yan12125]

Good news. Maintaining MIPS/MIPS64 builds is a real headache.

Seems https://android.googlesource.com/platform/ndk.git/+/f90c6865de087c9b0cfd948124871c4982d62283 should be cherry-picked to ndk-release-r16 branch too?

[DanAlbert]

Yep, already on it.