Architecture

Architecture

Traces uses clang to expand and instrument traced code. Traced processes write trace records to shared memory, which is dumped by an external process, the trace_dumper.

Trace point generation

During compile time, the traced code is passed through the Traces code instrumentor. The instrumentor expands explicit trace points, instrumenting function entry/leave points and compiles type-information into the executable.

Linking

The traced executable is explicitly linked against libtraces. When the traced application is run, the libtraces constructor allocates two regions in shared memory through the shm API. One region is dedicated for holding static metadata (types and trace point format). The second region is a cyclic buffer holding trace records as they are written by the application. Writing a trace record from the traced application's perspective is a matter of writing data to memory. If the buffer is full, the traced application will write trace records in place of the oldest entries in the cyclic buffer.

Dumping records to disk

The traced application itself does not write the records to disk. The record writing logic is implemented in trace_dumper. trace_dumper is a standalone application that maps the shared memory areas of traced applications and dumps their records to disk. The trace_dumper has two modes of operation which can be simultaneously used:

• Online mode - Print traces to stdout as they are generated by applications
• Journal mode - Write the records to a file for later inspection

Separating the writing of traces to memory and their dumping to disk to different processes has several advantages:

• When traced processes crash, the trace_dumper will dump every record written prior to the crash, allowing the developer to see exactly what happened prior to the crash
• Central way of controlling which processes' records are dumped
• All records and metadata are written to a single file - no need to manage many log files for multiple