Common Trace Format - Failed to produce correct trace output

[starmachine24]

Describe the bug
We wanted to produce a CTF trace via UART, so we enabled it in Kconfig and used minicom to capture the output. When trying to open the file in Trace compass it fails to import the file when the metadata file is used, otherwise it can import the file but can't understand it. It fails to be correctly parsed when run through babeltrace.

The output is from running the test program in tests/subsys/tracing. The print statements in the test program was removed.

The error message from babeltrace is:

[error] Event id 4 is unknown.
[error] Reading event failed.
Error printing trace.

Looking through the metadata file no such event is defined.

What have you tried to diagnose or workaround this issue?
We started to add event handlers for each of the events that babeltrace complained about. The number of events added quickly grew, so we wrote a script to add all events between 0 - 255 that were not already defined. All the events we added are defined as follows:

event {
        name = event3;
        id = 0x3;
        fields := struct {
                uint32_t thread_id;
        };
};

When using this modified metadata file we get the following output:
babeltrace output.txt

As you can see there are a lot of events captured that are not defined in the default metadata file. As well as this warning with differing values:

[warning] Unknown value 3694505765 in enum. 

These three warnings are most likely related to us shutting down the board:

[error] Unexpected end of packet. Either the trace data stream is corrupted or metadata description does not match data layout.
[error] Reading event failed.
Error printing trace.

To Reproduce
Steps to reproduce the behavior:

1. Enable CTF tracing with UART backend
2. Build the test program for the up_squared board
3. Capture the output via minicom
4. Try opening the trace using Trace compass or babeltrace.

Expected behavior
By running the supplied test program we expected to be able to produce a correct trace file and analyze it in Trace compass or babeltrace.

Impact
We can't progress with our Thesis since we need to be able to analyze the trace output.

Environment (please complete the following information):

• OS: Ubuntu: 18.04.4 LTS
• Toolchain: Zephyr SDK 0.11.2
• Zephyr: 2.2.99

[wentongwu]

@starmachine24 please make sure the uart used to trace ctf has no other output, if anything wrong the tool(Trace compass or babeltrace) will stop the parse. BTW, can you please try the script in https://github.com/zephyrproject-rtos/zephyr/blob/master/scripts/tracing/trace_capture_uart.py

[starmachine24]

We managed to capture the trace output using the script you provided. We think that the problem was that console redirection was enabled so the output file contained the splash screen and the zephyr boot line of text. We think that the output file got corrupted when we tried to remove these lines from the file. However, we are now able to parse the trace output in babeltrace and trace compass using the provided metadata file.

Thank you!

[wentongwu]

@starmachine24 thanks, can we close this now?

[starmachine24]

Yes i would say that you can close the issue

[smrtos]

@starmachine24 I got a similar issue #49889. Could you share more details about how you fixed your issue?