Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

link static library for TA #1399

Closed
yuhsien-chen opened this issue Mar 9, 2017 · 28 comments
Closed

link static library for TA #1399

yuhsien-chen opened this issue Mar 9, 2017 · 28 comments

Comments

@yuhsien-chen
Copy link

Hello, I'm new to optee, and I want to do a work with link a prebuilt static library to TA.
When my TA call functions in that prebuilt static library, it results in panic, and I don't understand the root cause.
Any advice will be appreciate.
Here is the log from Trusted OS.

ERROR: TEE-CORE:
ERROR: TEE-CORE: user TA data-abort at address 0x4008d0b4 (alignment fault)
ERROR: TEE-CORE: esr 0x92000061 ttbr0 0x100003f0790a0 ttbr1 0x00000000 cidr 0x0
ERROR: TEE-CORE: cpu #5 cpsr 0x40000100
ERROR: TEE-CORE: x0 000000004008d0b4 x1 0000000040000e70
ERROR: TEE-CORE: x2 0000004300000808 x3 0000001400000079
ERROR: TEE-CORE: x4 0000000000000010 x5 0000000000000000
ERROR: TEE-CORE: x6 8080808080808080 x7 0000000000000008
ERROR: TEE-CORE: x8 7f7f7f7f7f7f7f7f x9 fefefefefefefeff
ERROR: TEE-CORE: x10 7f7f7f7f7f7f7f7f x11 0101010101010101
ERROR: TEE-CORE: x12 0000000000000000 x13 0000000040000fb0
ERROR: TEE-CORE: x14 0000000000000000 x15 0000000000000001
ERROR: TEE-CORE: x16 000000003f00ec08 x17 0000000000000000
ERROR: TEE-CORE: x18 0000000000000000 x19 0000000000000000
ERROR: TEE-CORE: x20 000000004008d070 x21 0000000000000001
ERROR: TEE-CORE: x22 0000000040000e60 x23 0000000000000002
ERROR: TEE-CORE: x24 000000004008d0f0 x25 0000000000000001
ERROR: TEE-CORE: x26 0000000040000e50 x27 0000000000000001
ERROR: TEE-CORE: x28 000000004008d0f0 x29 0000000040000e90
ERROR: TEE-CORE: x30 0000000040004604 elr 0000000040004614
ERROR: TEE-CORE: sp_el0 0000000040000db0
ERROR: TEE-CORE: Status of TA 956173ce-a08e-4628-af83-ca9a5ed6260e (0x3f05bcf0) (active)
ERROR: TEE-CORE: - load addr : 0x40001000 ctx-idr: 1
ERROR: TEE-CORE: - stack: 0x40000000 4096
ERROR: TEE-CORE: sect 0 : va 0x40000000 pa 0x3f28d000 0x1000
ERROR: TEE-CORE: sect 1 : va 0x40001000 pa 0x3f200000 0x5f000
ERROR: TEE-CORE: sect 2 : va 0x40060000 pa 0x3f25f000 0x22000
ERROR: TEE-CORE: sect 3 : va 0x40082000 pa 0x3f281000 0xc000
ERROR: TEE-CORE: sect 4 : va 0 pa 0 0
ERROR: TEE-CORE: sect 5 : va 0 pa 0 0
ERROR: TEE-CORE: sect 6 : va 0 pa 0 0
ERROR: TEE-CORE: sect 7 : va 0 pa 0 0

@jforissier
Copy link
Contributor

Hi @yuhsien-chen,
Your TA is trying to write a 64-bit value to a memory address that is not 8-byte aligned. Try re-building your static library with -mno-unaligned-access if you can.

@yuhsien-chen
Copy link
Author

I don't have the source code for the library, so I can't re-build it.
Very appreciate for your reply.
I'll try other way to do it. Thanks.

@yuhsien-chen
Copy link
Author

yuhsien-chen commented Mar 13, 2017

I was updated the version of static library(vendor provided), and I encountered another problem.
When my TA call functions in the static library,
the Trust OS show message "TEE-CORE: Unknown relocation type 257", and the CA received an error code "0xffff0005" means "TEEC_ERROR_BAD_FORMAT".
I found that code in /optee_os/core/arch/arm/kernel/elf_load.c.
What's the message means?
Thanks.

@jenswi-linaro
Copy link
Contributor

Relocation types are described in http://infocenter.arm.com/help/topic/com.arm.doc.ihi0056b/IHI0056B_aaelf64.pdf
In this case it's a R_AARCH64_ABS64. I'm not sure what code would result in this. I haven't run across it in our TAs. Possibly it's due to how that library is compiled.

It shouldn't be very hard to implement, considering that we support R_ARM_ABS32 for 32-bit ELFs.

@jforissier
Copy link
Contributor

jforissier commented Mar 14, 2017

@yuhsien-chen could you try this patch, please? jforissier@74e20a1
As I said in the commit message, it's completely untested, but if you can confirm it works for you then I will create a pull request.
Update 29-Mar-2017: this patch is incorrect, see #1419 for the (hopefully) correct version.

@yuhsien-chen
Copy link
Author

Hi @jforissier ,
It works for me to execute my TA, even it going to "alignment fault", but your patch addressed "R_AARCH64_ABS64 not support" problem, thanks!
I'll try other way to address the "alignment fault". Thanks.

jforissier added a commit to jforissier/optee_os that referenced this issue Mar 15, 2017
…ng TAs

If a 64-bit TA contains relocations of type R_AARCH64_ABS64, OP-TEE
refuses to load it and logs the following error:

  ERROR:   TEE-CORE: Unknown relocation type 257

This relocation type does not seem to happen in our test applications,
but someone has experienced the issue after linking a TA against a
third-party static library [1]. I could reporoduce the issue by adding
the following assembly code to a TA:

          .pushsection .rodata
          .global _reloc__text_start
  _reloc__text_start:
          .quad __text_start
          .popsection

(this adds a 64-bit value into the .rodata section of the TA, exported
as a global symbol called _reloc__text_start, its value is set to the
__text_start address).

This commit adds the necessary code to support R_AARCH64_ABS64.

[1] OP-TEE#1399

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Tested-by: Jerome Forissier <jerome.forissier@linaro.org> (qemu_v8)
jforissier added a commit to jforissier/optee_os that referenced this issue Mar 15, 2017
…ng TAs

If a 64-bit TA contains relocations of type R_AARCH64_ABS64, OP-TEE
refuses to load it and logs the following error:

  ERROR:   TEE-CORE: Unknown relocation type 257

This relocation type does not seem to happen in our test applications,
but someone has experienced the issue after linking a TA against a
third-party static library [1]. I could reporoduce the issue by adding
the following assembly code to a TA:

          .pushsection .rodata
          .global _reloc__text_start
  _reloc__text_start:
          .quad __text_start
          .popsection

(this adds a 64-bit value into the .rodata section of the TA, exported
as a global symbol called _reloc__text_start, its value is set to the
__text_start address).

This commit adds the necessary code to support R_AARCH64_ABS64.

[1] OP-TEE#1399

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Tested-by: Jerome Forissier <jerome.forissier@linaro.org> (qemu_v8)
jforissier added a commit to jforissier/optee_os that referenced this issue Mar 15, 2017
If a 64-bit TA contains relocations of type R_AARCH64_ABS64, OP-TEE
refuses to load it and logs the following error:

  ERROR:   TEE-CORE: Unknown relocation type 257

This relocation type does not seem to happen in our test applications,
but someone has experienced the issue after linking a TA against a
third-party static library [1]. I could reporoduce the issue by adding
the following assembly code to a TA:

          .pushsection .rodata
          .global _reloc__text_start
  _reloc__text_start:
          .quad __text_start
          .popsection

(this adds a 64-bit value into the .rodata section of the TA, exported
as a global symbol called _reloc__text_start, its value is set to the
__text_start address).

This commit adds the necessary code to support R_AARCH64_ABS64.

[1] OP-TEE#1399

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Tested-by: Jerome Forissier <jerome.forissier@linaro.org> (qemu_v8)
jforissier added a commit to jforissier/optee_os that referenced this issue Mar 15, 2017
If a 64-bit TA contains relocations of type R_AARCH64_ABS64, OP-TEE
refuses to load it and logs the following error:

  ERROR:   TEE-CORE: Unknown relocation type 257

This relocation type does not seem to happen in our test applications,
but someone has experienced the issue after linking a TA against a
third-party static library [1]. I could reporoduce the issue by adding
the following assembly code to a TA:

          .pushsection .rodata
          .global _reloc__text_start
  _reloc__text_start:
          .quad __text_start
          .popsection

(this adds a 64-bit value into the .rodata section of the TA, exported
as a global symbol called _reloc__text_start, its value is set to the
__text_start address).

This commit adds the necessary code to support R_AARCH64_ABS64.

[1] OP-TEE#1399

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Tested-by: Jerome Forissier <jerome.forissier@linaro.org> (qemu_v8)
@jforissier
Copy link
Contributor

#1419

@yuhsien-chen
Copy link
Author

I implemented a simple library to cause alignment fault.
And I tried build with "-mno-unaligned-access" option, got
"error: unrecognized command line option '-mno-alignment-access".
I found it was the gcc option for ARM, and the option for ARM64 is "-mstrict-align".
But I built with it, and still get alignment fault.
Any advice will be appreciate.

Below is the simple library code.

libMM.h

#ifndef _LIB_MM_H
#define _LIB_MM_H

int test_unalign_access2(uint8_t a);

#endif

libMM.c

#include <stdint.h>

#include "libMM.h"

int test_unalign_access2(uint8_t a)
{
	uint32_t* pMyPointer = (uint32_t*)(&a);

	*pMyPointer += 2;

	return *pMyPointer;
}

makefile

ANDROID_NDK_BIN := /tmp/my-android-toolchain/arm64/bin

CC := $(ANDROID_NDK_BIN)/aarch64-linux-android-gcc
CPP := $(ANDROID_NDK_BIN)/aarch64-linux-android-g++
AR := $(ANDROID_NDK_BIN)/aarch64-linux-android-ar

CFLAGS := -I../include -I/tmp/my-android-toolchain/arm64/sysroot/usr/include
CFLAGS += -static -Wall -O3 -mstrict-align

LIBS := -L/tmp/my-android-toolchain/arm64/sysroot/usr/lib

# Rule for make c++ object file
.cpp.o:
	$(CPP) $(CFLAGS) $(LIBS) -c $<

# Rule for make c object file
.c.o:
	$(CC) $(CFLAGS) $(LIBS) -c $<

STATIC_LIB_SRC = libMM.c
STATIC_LIB_OBJ = libMM.o
OUT = ../lib/arm64/libMM.a
$(OUT): $(STATIC_LIB_OBJ)
	$(AR) rcs $(OUT) $(STATIC_LIB_OBJ)

@yuhsien-chen yuhsien-chen reopened this Mar 22, 2017
@jenswi-linaro
Copy link
Contributor

Shouldn't it be -no-mstrict-align?

@yuhsien-chen
Copy link
Author

I had tried the options list below.
-mno-strict-align => compiler report "unrecognized command line option '-mno-strict-align'"
-mstrict-align => compile ok, but still get alignment fault in execution.
-no-mstrict-align => compiler report "unrecognized command line option '-no-mstrict-align'"

@jenswi-linaro
Copy link
Contributor

jenswi-linaro commented Mar 22, 2017

So it's -mno-strict-align then. But it may not help in this case, at least not from what the man page says:

       -mstrict-align
           Do not assume that unaligned memory references are handled by the
           system.

It's the hardware catching the unaligned access. See the A bit in SCTLR_EL1 in the ARMv8 reference manual.

@yuhsien-chen
Copy link
Author

yuhsien-chen commented Mar 22, 2017

I had download manual and search the content with "SCTLR_EL1".
I got
The exception masking bits

  • D Debug exception mask bit. When EL0 is enabled to modify the mask bits, this bit is
    visible and can be modified. However, this bit is architecturally ignored at EL0.
  • A SError interrupt mask bit.
  • I IRQ interrupt mask bit.
  • F FIQ interrupt mask bit.

For each bit, the values are:

  • 0 Exception not masked.
  • 1 Exception masked.

Does the A bit means "A SError interrupt mask bit."?
Can I modify the A bit value in optee_os?
My develop environment is AOSP 7.1.1 with optee 2.3.0

@jenswi-linaro
Copy link
Contributor

Search further, the bits you found are from CPSR/SPSR...

@yuhsien-chen
Copy link
Author

@jenswi-linaro

I found the exactly one.
It's in B2.4 Aligmment Support.
The A bit of SCTLR_EL1 seems to be set in arm-trusted-firmware(u-boot).
I'll try to modify it.
Thanks!

jforissier added a commit to jforissier/optee_os that referenced this issue Mar 22, 2017
If a 64-bit TA contains relocations of type R_AARCH64_ABS64, OP-TEE
refuses to load it and logs the following error:

  ERROR:   TEE-CORE: Unknown relocation type 257

This relocation type does not seem to happen in our test applications,
but someone has experienced the issue after linking a TA against a
third-party static library [1]. I could reproduce the issue by
compiling the hello_world TA with -fPIC instead of -fpie. This simple
change generates *one* R_AARCH64_ABS64 in the TA ELF file.

This commit adds the necessary code to support R_AARCH64_ABS64.

[1] OP-TEE#1399

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Tested-by: Jerome Forissier <jerome.forissier@linaro.org> (qemu_v8)
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
@yuhsien-chen
Copy link
Author

yuhsien-chen commented Mar 23, 2017

I tried reset value in "arm-trusted-firmware/bl1/aarch64/bl1_entrypoint.S".

        /* ---------------------------------------------
	 * Enable the instruction cache, stack pointer
	 * and data access alignment checks
	 * ---------------------------------------------
	 */
	mov	x1, #(SCTLR_I_BIT | SCTLR_A_BIT | SCTLR_SA_BIT)
	mrs	x0, sctlr_el3
	//orr	x0, x0, x1
        // reset values
        bic    x0, x0, x1
	msr	sctlr_el3, x0
	isb

Bit it still catch the align access, and report exception.
Did I look for the wrong direction?
Or the SCTLR_EL1.A bit shouldn't be modified?
Thanks.

@yuhsien-chen yuhsien-chen reopened this Mar 23, 2017
@yuhsien-chen
Copy link
Author

yuhsien-chen commented Mar 23, 2017

Finally, I found "optee_os/core/arch/arm/kernel/generic_entry_a64.S"

        mrs	x0, sctlr_el1
	mov	x1, #(SCTLR_I | SCTLR_A | SCTLR_SA)
	//orr	x0, x0, x1
        // reset values
	bic x0, x0, x1
	msr	sctlr_el1, x0
	isb

It works!
I tried my static library, and it works too.
Thanks for your help.
Very appreciate.

@jforissier
Copy link
Contributor

@jenswi-linaro @yuhsien-chen do you think we should make this configurable at build time? Such as, in core/arch/arm/arm.mk:

# Allow SEL0/SEL1 unaligned access?
CFG_UNALIGNED_ACCESS=n

@jenswi-linaro
Copy link
Contributor

@jforissier sounds good, we need to be careful with how it's enabled, not all platforms may support this.

@yuhsien-chen
Copy link
Author

@jforissier it would be good for users.
But I agree @jenswi-linaro 's opinion.
When I search relative data for "unaligned access", I found some platform does not support it.
Even if I currently disabled the config and makes my static library call works.
But It still occurs alignment fault exception sometimes.
So, I think the good way is to fix static library, to avoid unaligned access.
Thanks for your help. :)

@jforissier
Copy link
Contributor

OK, thanks for your feedback.

@acalb
Copy link

acalb commented Mar 28, 2017

Hi,

I am using HiKey 8GB board with AOSP + OP-TEE.
I am trying to load a TA from an Android application in AOSP Permissive mode. It successfully loads the TA but getting failed at TEEC_OpenSession with error code 0xffff0005 (TEEC_ERROR_BAD_FORMAT).

Just before the TEEC_OpenSession TEE output log shows that Unknown relocation type 257.

I've pulled the latest version of optee_os and optee_test repositories.
AOSP 7.1.1
Kernel Version: 4.9.10

Here is the debug log:

ERROR:   TEE-CORE: Unknown relocation type 257
03-28 09:37:18.557  2720  2720 D log   : TEEC_OpenSession failed: 0xffff0005
03-28 09:37:18.560  2720  2720 F libc    : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 2720 (test.app)
03-28 09:37:18.693  2936  2936 I crash_dump64: obtaining output fd from tombstoned
03-28 09:37:18.695  1877  1877 I /system/bin/tombstoned: received crash request for pid 2720
03-28 09:37:18.700  2936  2936 I crash_dump64: performing dump of process 2720 (target tid = 2720)
03-28 09:37:18.701  2936  2936 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
03-28 09:37:18.701  2936  2936 F DEBUG   : Build fingerprint: 'Android/hikey/hikey:7.1.1/NYC/alican03280747:userdebug/test-keys'
03-28 09:37:18.701  2936  2936 F DEBUG   : Revision: '0'
03-28 09:37:18.701  2936  2936 F DEBUG   : ABI: 'arm64'
03-28 09:37:18.701  2936  2936 F DEBUG   : pid: 2720, tid: 2720, name: test.app  >>> org.test.app <<<
03-28 09:37:18.701  2936  2936 F DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
03-28 09:37:18.701  2936  2936 F DEBUG   : Cause: null pointer dereference
03-28 09:37:18.701  2936  2936 F DEBUG   :     x0   0000000000000000  x1   000000008010a403  x2   0000007fffffa370  x3   0000007fffffa410
03-28 09:37:18.701  2936  2936 F DEBUG   :     x4   0000007fffffa440  x5   006869746166ffff  x6   0000000000000000  x7   0000000000000080
03-28 09:37:18.701  2936  2936 F DEBUG   :     x8   0000000000000000  x9   0000007f957a82c8  x10  0000000000000002  x11  0000000000000000
03-28 09:37:18.701  2936  2936 F DEBUG   :     x12  0101010101010101  x13  0000000000000038  x14  ffffffffffffffff  x15  000015c9ccaaa1be
03-28 09:37:18.701  2936  2936 F DEBUG   :     x16  0000007f957aafa0  x17  0000007fb781e6f0  x18  0000000000000020  x19  0000000000000000
03-28 09:37:18.701  2936  2936 F DEBUG   :     x20  0000007fffffa558  x21  0000000000000000  x22  0000007f95686008  x23  0000007fffffa458
03-28 09:37:18.701  2936  2936 F DEBUG   :     x24  0000000000000004  x25  0000007fb4496a98  x26  0000000000000000  x27  0000000000000000
03-28 09:37:18.702  2936  2936 F DEBUG   :     x28  0000000000000001  x29  0000007fffffa510  x30  0000007f957a86dc
03-28 09:37:18.702  2936  2936 F DEBUG   :     sp   0000007fffffa370  pc   0000007f957a870c  pstate 0000000060000000
03-28 09:37:18.707  2936  2936 F DEBUG   : 
03-28 09:37:18.707  2936  2936 F DEBUG   : backtrace:
03-28 09:37:18.707  2936  2936 F DEBUG   :     #00 pc 000000000000170c  /system/app/Test/test.apk (offset 0x2c5e000)
03-28 09:37:18.707  2936  2936 F DEBUG   :     #01 pc 0000000000002784  /system/app/Test/test.apk (offset 0x2c55000)
03-28 09:37:18.707  2936  2936 F DEBUG   :     #02 pc 0000000001a97bbc  /system/app/Test/oat/arm64/Test.odex (offset 0x933000)

Thank you so much for your time.

@etienne-lms
Copy link
Contributor

@acalb, I think this has been recently fixed: see #1419, related to issue #1399.

@acalb
Copy link

acalb commented Mar 28, 2017

@etienne-lms, I see that fixes and pulled the latest commits of optee_os repository, however it still gives the same error.

@etienne-lms
Copy link
Contributor

@acalb, that's strange: try to rebuild optee-os from scratch. Maybe the fix fails on your relocation but at least you should not get the trace "ERROR: TEE-CORE: Unknown relocation type 257".

@jforissier
Copy link
Contributor

Agreed, I fail to see how you can still have the "Unknown relocation" trace if you are running the correct version. @acalb, can you please make sure the correct TEE binary is loaded by checking the "Initializing" trace? It looks like this:

INFO:    TEE-CORE: Initializing (2.3.0-150-gb96f67d #3 Tue Mar 28 08:12:44 UTC 2017 aarch64)

The version/commit ID and the build date should be enough to know for sure if it's indeed the expected binary that's running.

@yuhsien-chen
Copy link
Author

yuhsien-chen commented Mar 28, 2017

@acalb Did you rebuild optee_os with arm-trusted-firmware?

In Hikey, optee_os is built into fip.bin and you should replace by fastboot.

$ sudo fastboot flash fastboot fip.bin

@vchong
Copy link
Contributor

vchong commented Mar 28, 2017

@acalb optee_os/fip.bin is built separately from the main aosp sources so you'll need to rebuild and flash it again as advised by all the good people above.

@acalb
Copy link

acalb commented Mar 29, 2017

@jforissier, @yuhsien-chen, @vchong, @etienne-lms thanks for your answers. I've rebuild fip.bin and flashed it again. Now it works as expected.

takuya-sakata pushed a commit to renesas-rcar/optee_os that referenced this issue Dec 22, 2017
If a 64-bit TA contains relocations of type R_AARCH64_ABS64, OP-TEE
refuses to load it and logs the following error:

  ERROR:   TEE-CORE: Unknown relocation type 257

This relocation type does not seem to happen in our test applications,
but someone has experienced the issue after linking a TA against a
third-party static library [1]. I could reproduce the issue by
compiling the hello_world TA with -fPIC instead of -fpie. This simple
change generates *one* R_AARCH64_ABS64 in the TA ELF file.

This commit adds the necessary code to support R_AARCH64_ABS64.

[1] OP-TEE/optee_os#1399

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Tested-by: Jerome Forissier <jerome.forissier@linaro.org> (qemu_v8)
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants