DMSG/IMSG from TA/STA is not seen

[toddkuhreng]

Hi all,

Problem I am facing is that I can't find any Debug messages from TA and STA on UART3. I am kind of lost here. Could someone please help with this? (I am trying this on Hikey board)

All I see in UART3 is following:

DEBUG:   [0x0] TEE-CORE:spi_init:150: enable SPI clock
DEBUG:   [0x0] TEE-CORE:spi_init:158: PERI_SC_PERIPH_CLKEN3: 0x0
DEBUG:   [0x0] TEE-CORE:spi_init:161: PERI_SC_PERIPH_CLKSTAT3: 0x2403c1
DEBUG:   [0x0] TEE-CORE:spi_init:172: configure gpio6_{0,1,3} as SPI
DEBUG:   [0x0] TEE-CORE:spi_init:173: configure gpio6_2 as GPIO
DEBUG:   [0x0] TEE-CORE:spi_init:179: configure gpio6_{0:3} as nopull
INFO:    TEE-CORE: Initialized
DEBUG:   [0x0] TEE-CORE:init_primary_helper:628: DMSG Primary CPU switching to normal world boot
DEBUG:   TEE-CORE:generic_boot_cpu_on_handler:662: cpu 1: a0 0x3f000f24
DEBUG:   TEE-CORE:init_secondary_helper:647: Secondary CPU Switching to normal world boot
DEBUG:   TEE-CORE:generic_boot_cpu_on_handler:662: cpu 2: a0 0x3f000f24
DEBUG:   TEE-CORE:init_secondary_helper:647: Secondary CPU Switching to normal world boot
DEBUG:   TEE-CORE:generic_boot_cpu_on_handler:662: cpu 3: a0 0x3f000f24
DEBUG:   TEE-CORE:init_secondary_helper:647: Secondary CPU Switching to normal world boot
DEBUG:   TEE-CORE:generic_boot_cpu_on_handler:662: cpu 4: a0 0x3f000f24
DEBUG:   TEE-CORE:init_secondary_helper:647: Secondary CPU Switching to normal world boot
DEBUG:   TEE-CORE:generic_boot_cpu_on_handler:662: cpu 5: a0 0x3f000f24
DEBUG:   TEE-CORE:init_secondary_helper:647: Secondary CPU Switching to normal world boot
DEBUG:   TEE-CORE:generic_boot_cpu_on_handler:662: cpu 6: a0 0x3f000f24
DEBUG:   TEE-CORE:init_secondary_helper:647: Secondary CPU Switching to normal world boot
DEBUG:   TEE-CORE:generic_boot_cpu_on_handler:662: cpu 7: a0 0x3f000f24
DEBUG:   TEE-CORE:init_secondary_helper:647: Secondary CPU Switching to normal world boot

Following is that I have in my config, and make Makefiles.

Inside my build/hikey.mk file I have following:

CFG_NW_CONSOLE_UART ?= 0
CFG_SW_CONSOLE_UART ?= 3
OPTEE_OS_COMMON_FLAGS += PLATFORM=hikey
CFG_CONSOLE_UART=$(CFG_SW_CONSOLE_UART)
OPTEE_OS_CLEAN_COMMON_FLAGS += PLATFORM=hikey

CFG_TEE_CORE_LOG_LEVEL ?= 3 # 0=none 1=err 2=info 3=debug 4=flow
OPTEE_OS_COMMON_FLAGS += CFG_TEE_CORE_LOG_LEVEL=$(CFG_TEE_CORE_LOG_LEVEL)
OPTEE_OS_COMMON_FLAGS += DEBUG=1
OPTEE_OS_COMMON_FLAGS += CFG_WITH_PAGER=n
OPTEE_OS_COMMON_FLAGS += CFG_TEE_TA_LOG_LEVEL=3

inside hello_world/ta/Makefile I have following:

CFG_TEE_TA_LOG_LEVEL ?= 3
CPPFLAGS += -DCFG_TEE_TA_LOG_LEVEL=$(CFG_TEE_TA_LOG_LEVEL)

Inside ./optee_os/mk/config.mk I have following:

CFG_TEE_CORE_DEBUG ?= y
CFG_TEE_CORE_LOG_LEVEL ?= 3
CFG_TEE_TA_LOG_LEVEL ?= 3
CFG_TEE_CORE_TA_TRACE ?= y

Inside STA I have following:

static TEE_Result test_trace(uint32_t param_types __unused,
                        TEE_Param params[TEE_NUM_PARAMS] __unused)
{

        IMSG("static TA \"%s\" says \"Hello world !\"", TA_NAME);
        return TEE_SUCCESS;
}

Inside TA I have following:

TEE_Result TA_CreateEntryPoint(void)
{
  DMSG("TA Create entry point has been called");
  return TEE_SUCCESS;
}

And following is how I am building the whole image:
make CFG_TEE_TA_LOG_LEVEL=3 CFG_TEE_CORE_LOG_LEVEL=3 DEBUG=1

I am invoking STA and TA from NW. I don't see DMSG from TA and IMSG/DMSG from STA on UART3.
What am I missing here? Could you please help?

[jforissier]

Hi @toddkuhreng ,

Your settings look good...

Regarding the pseudo TA (static TA), I suppose your test_trace() function is not called. Otherwise, you would definitely see some output on UART3, because you do have some debug output from TEE core already (INFO: TEE-CORE: Initialized), so the debug settings must be OK.
As for the user TA, the settings look good too.
Please make sur you are indeed running the binaries you think you are running:

• Check the git version string and build count in the OP-TEE version string. For instance, if you see: INFO: TEE-CORE: Initializing (2.3.0-169-g77bef78 #4 Sat Apr 8 13:40:34 UTC 2017 aarch64), then git describe in optee_os should print 2.3.0-169-g77bef78 and cat optee_os/out/arm/core/.buildcount should show 4.
• Use md5sum to check that the .ta file is identical on the board and in your build environment.

[toddkuhreng]

I see following on the UART3:
INFO: TEE-CORE: IMSG Initializing (2.3.0-16-ged1993b-dev #1 zo apr 16 12:38:01 UTC 2017 arm) and git describe in optee_os shows 2.3.0-16-ged1993b

And cat optee_os/out/arm/core/.buildcount prints 1. What does that mean?

I checked md5sum of .ta they matches.

[jforissier]

Build count is printed after # in ident string, and incremented each time you rebuild tee.bin. It helps check if the running binary matches the current sources in a dev environment.

I have no idea what's wrong.

[toddkuhreng]

I am not sure what is the issue. But if I give CFG_NW_CONSOLE_UART and CFG_SW_CONSOLE_UART both 3, then I see the DMSG and IMSG on UART3.

[jforissier]

That's interesting, thanks for the information. I will try to reproduce the issue and debug it.

[jforissier]

For the record: the issue when CFG_NW_CONSOLE_UART is different from CFG_SW_CONSOLE_UART is caused by the kernel doing something wrong with the UART device when it's enabled in the DT but not used as the console by the kernel.

I came to this conclusion after I observed that things work as expected if I do this:

diff --git a/arch/arm64/boot/dts/hisilicon/hi6220-hikey.dts b/arch/arm64/boot/dts/hisilicon/hi6220-hikey.dts
index 6753e63..42f389f 100644
--- a/arch/arm64/boot/dts/hisilicon/hi6220-hikey.dts
+++ b/arch/arm64/boot/dts/hisilicon/hi6220-hikey.dts
@@ -105,7 +105,7 @@
                };
 
                uart3: uart@f7113000 {
-                       status = "ok";
+                       status = "disabled";
                };
 
                /*

...then build with CFG_NW_CONSOLE_UART=0 CFG_SW_CONSOLE_UART=3. In this case the OP-TEE output goes to UART3 while Linux uses UART0 as expected.

For now, I will propose a patch to add a comment to the Makefile, documenting the fact that the DT has to be consistent with the build settings. A better long-term solution would be to get rid of the compile-time configuration and rely only on the DT to configure the UARTs and the secure/non-secure consoles.