Unicorn ARM backend problems

[MagaTailor]

I've just managed to build unicorn on ARM by using just a single architecture:

UNICORN_ARCHS="x86" make.sh

However, while building the samples we immediately get a hint something's wrong:

LINK    sample_x86
../libunicorn.so: error: undefined reference to 'qemu_getauxval'
../libunicorn.so: error: undefined reference to 'pthread_mutexattr_init'
../libunicorn.so: error: undefined reference to 'pthread_mutexattr_settype'
../libunicorn.so: error: undefined reference to 'pthread_mutexattr_destroy'
../libunicorn.so: error: undefined reference to 'pthread_mutex_trylock'
../libunicorn.so: error: undefined reference to 'sem_init'
../libunicorn.so: error: undefined reference to 'sem_destroy'
../libunicorn.so: error: undefined reference to 'sem_post'
../libunicorn.so: error: undefined reference to 'sem_trywait'
../libunicorn.so: error: undefined reference to 'sem_timedwait'
../libunicorn.so: error: undefined reference to 'sem_wait'
../libunicorn.so: error: undefined reference to 'pthread_sigmask'
../libunicorn.so: error: undefined reference to 'pthread_create'
../libunicorn.so: error: undefined reference to 'pthread_join'
../libunicorn.so: error: undefined reference to 'pthread_kill'

For some reason LDFLAGS=-lpthread needs to be exported manually which leaves:

undefined reference to 'qemu_getauxval'

Naturally, the installed library doesn't work:

/tmp/unicorn/bindings/python/sample_x86.py
Traceback (most recent call last):
  File "/tmp/unicorn/bindings/python/sample_x86.py", line 5, in <module>
    from unicorn import *
  File "/tmp/unicorn/bindings/python/unicorn/__init__.py", line 4, in <module>
    from .unicorn import Uc, uc_version, uc_arch_supported, version_bind, debug, UcError
  File "/tmp/unicorn/bindings/python/unicorn/unicorn.py", line 81, in <module>
    raise ImportError("ERROR: fail to load the dynamic library.")
ImportError: ERROR: fail to load the dynamic library.

What gives?

[aquynh]

i removed file qemu/util/getauxval.c from Qemu when i forked Unicorn, but apparently this is needed for ARM backend. so you should recover this file from Qemu 2.2.1 version and try to put it back.

you can recover the other missing files if you still cannot compile.

thanks.

[MagaTailor]

Hooray, I've got it working on ARM!

I added the missing file and the following line to qemu/util/Makefile.objs:

util-obj-y += getauxval.o

and that was enough. Thanks.

[aquynh]

Nice, so all samples/sample_xxx compiled and worked on ARM without any issue?

[MagaTailor]

We still have the original problem of not being able to link all architectures in at the same time. I worked around it by enabling just x86 and that works as expected. Plus all samples compile and execute fine as well.

I'm pretty sure this should work for any of the remaining archs separately.

[aquynh]

can you paste the output error when compiling on ARM with default option? (just run ./make.sh is enough)

[MagaTailor]

Sure, it seems it manages to get to the linking stage nowadays:

 GEN     ./libunicorn.so.0
/usr/bin/ld.gold.real: error: qemu/arm-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/arm-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/m68k-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/m68k-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/aarch64-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/aarch64-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/mips-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/mips-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/mipsel-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/mipsel-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/mips64-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/mips64-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/mips64el-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/mips64el-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/sparc-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/sparc-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/sparc64-softmmu/tcg/tcg.o: multiple definition of 'use_idiv_instructions'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here
/usr/bin/ld.gold.real: error: qemu/sparc64-softmmu/tcg/tcg.o: multiple definition of 'tcg_target_deposit_valid'
/usr/bin/ld.gold.real: qemu/x86_64-softmmu/tcg/tcg.o: previous definition here

[aquynh]

can you pull the new branch "arm" (https://github.com/unicorn-engine/unicorn/tree/arm) and try to compile again on ARM to see if the problem is fixed?

[MagaTailor]

Yes, it does compile and link now, but doesn't work:

 LINK    sample_arm
../libunicorn.so: error: undefined reference to 'use_idiv_instructions_mips'
../libunicorn.so: error: undefined reference to 'use_idiv_instructions_mipsel'

[aquynh]

can you pull from the "arm" branch now and try again?

[MagaTailor]

Now it's:

 LINK    sample_arm
../libunicorn.so: error: undefined reference to 'use_idiv_instructions_rt_mips'
../libunicorn.so: error: undefined reference to 'use_idiv_instructions_rt_mipsel'

[aquynh]

can you try again now?

[MagaTailor]

That did it - all python tests pass and earlier I'd already built usercorn with it. Well done and thanks!

[MagaTailor]

Oh, one last thing - please have a look the samples' Makefile and the missing -lpthread.

Cheers!

[aquynh]

missing "-lpthread"? why did you still compile and run the samples successfully?
what is the output error?

[aquynh]

i merged the branch "arm" into "master", and deleted "arm".

[MagaTailor]

As I pointed out in the first comment, I had to pass LDFLAGS manually.

[aquynh]

see 69b784f. can you pull & try to compile again without your trick now?

[MagaTailor]

Indeed, the missing link has just got filled :)

[aquynh]

now if you can try to see if ARM64 backend works & report the result, that would be great!

[MagaTailor]

Can't help with that as I don't have the hardware. Hopefully someone on e.g. Tegra will hear your plea soon.

[aquynh]

i added ARM64 backend to the TODO list: https://github.com/unicorn-engine/unicorn/wiki/TODO