LevelZero.md

Runtime API Tracing for oneAPI Level Zero (Level Zero)

Overview

Level Zero API tracing allows to intercept standard API calls by injecting user-defined callbacks to function enter and exit points, as well as to read and modify traced function arguments and to read return value.

API tracing is implemented as one of the layers in Level Zero loader.

Supported Runtimes:

• Intel(R) Graphics Compute Runtime for oneAPI Level Zero and OpenCL(TM) Driver

Supported OS:

• Linux
• Windows

Supported HW:

• Intel(R) Processor Graphics GEN9+

Needed Headers:

• zet_api.h
• zel_tracing_api.h

Needed Libraries:

• oneAPI Level Zero libraries

How To Use

1. Before any work with Level Zero one must initialize its core and tools API:

ze_result_t status = ZE_RESULT_SUCCESS;
status = zeInit(ZE_INIT_FLAG_GPU_ONLY);
assert(status == ZE_RESULT_SUCCESS);

2. For each traced function on need to define the callbacks that will be called along with the traced API on enter and on exit, e.g. for zeCommandListAppendLaunchKernel:

void OnEnterCommandListAppendLaunchKernel(
    ze_command_list_append_launch_kernel_params_t* params,
    ze_result_t result, void* global_user_data, void** instance_user_data) {
  std::cout << "Function zeCommandListAppendLaunchKernel is called on enter" <<
    std::endl;
}

void OnExitCommandListAppendLaunchKernel(
    ze_command_list_append_launch_kernel_params_t* params,
    ze_result_t result, void* global_user_data, void** instance_user_data) {
  std::cout << "Function zeCommandListAppendLaunchKernel is called on exit" <<
    std::endl;
}

3. Create a tracer handle for the target device:

  zel_tracer_desc_t tracer_desc = {
      ZEL_STRUCTURE_TYPE_TRACER_DESC, nullptr,
      nullptr /* global user data */};

  zel_tracer_handle_t tracer = nullptr;
  status = zelTracerCreate(context, &tracer_desc, &tracer);
  assert(status == ZE_RESULT_SUCCESS);

4. Set callbacks to Level Zero API functions that should be traced, e.g.:

  zet_core_callbacks_t prologue_callbacks = {};
  zet_core_callbacks_t epilogue_callbacks = {};
  prologue_callbacks.CommandList.pfnAppendLaunchKernelCb =
    OnEnterCommandListAppendLaunchKernel;
  epilogue_callbacks.CommandList.pfnAppendLaunchKernelCb =
    OnExitCommandListAppendLaunchKernel;

  status = zelTracerSetPrologues(tracer_, &prologue_callbacks);
  assert(status == ZE_RESULT_SUCCESS);
  status = zelTracerSetEpilogues(tracer, &epilogue_callbacks);
  assert(status == ZE_RESULT_SUCCESS);

5. Enable tracing session. After this call you will be notified on every traced API call:

  status = zelTracerSetEnabled(tracer, true);
  assert(status == ZE_RESULT_SUCCESS);

6. To stop tracing, disable it. This call is non-blocking so some callbacks may be executing even after this function successfully returned:

  status = zelTracerSetEnabled(tracer, false);
  assert(status == ZE_RESULT_SUCCESS);

7. Destroy tracing handle to release all the resources and wait for all callbacks, which may be still in the process of execution:

  status = zelTracerDestroy(tracer);
  assert(status == ZE_RESULT_SUCCESS);

Build and Run

To make tracing work one need to link the application with Level Zero ICD library (e.g. libze_loader.so) and run it as following:

ZE_ENABLE_TRACING_LAYER=1 ./<application>

Usage Details

• refer to oneAPI Level Zero API Tracing documentation to learn more

Samples

• Level Zero Hot Functions
• Level Zero Hot Kernels
• Level Zero Debug Info
• Level Zero Metric Query

Tools

• Level Zero Tracer
• Tracing and Profiling Tool for Data Parallel C++ (DPC++)
• GPU Metrics Collection Tool for Data Parallel C++ (DPC++)