From 573ad96594263c2017830ca6ef38e7b97507e814 Mon Sep 17 00:00:00 2001 From: pytorchbot Date: Wed, 16 Oct 2024 17:37:53 -0700 Subject: [PATCH] LLM manual fixes on profiling part (#6315) LLM manual fixes on etdump generation (#6299) Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/6299 Fix the problem that LLM Manual profiling part cannot generate ETDump Reviewed By: Gasoonjia Differential Revision: D64485281 fbshipit-source-id: 864dda92112f50850feb94234658fdf6ca3b5922 (cherry picked from commit ec1c431c71d65e927533bd1f4a46e87ed7cbf9a2) Co-authored-by: Olivia Liu --- docs/source/llm/getting-started.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/docs/source/llm/getting-started.md b/docs/source/llm/getting-started.md index a74c4bd196..30356dd9b5 100644 --- a/docs/source/llm/getting-started.md +++ b/docs/source/llm/getting-started.md @@ -360,6 +360,7 @@ set(CMAKE_CXX_STANDARD 17) set(CMAKE_CXX_STANDARD_REQUIRED True) # Set options for executorch build. +option(EXECUTORCH_ENABLE_LOGGING "" ON) option(EXECUTORCH_BUILD_EXTENSION_DATA_LOADER "" ON) option(EXECUTORCH_BUILD_EXTENSION_MODULE "" ON) option(EXECUTORCH_BUILD_EXTENSION_TENSOR "" ON) @@ -515,6 +516,7 @@ set(CMAKE_CXX_STANDARD 17) set(CMAKE_CXX_STANDARD_REQUIRED True) # Set options for executorch build. +option(EXECUTORCH_ENABLE_LOGGING "" ON) option(EXECUTORCH_BUILD_EXTENSION_DATA_LOADER "" ON) option(EXECUTORCH_BUILD_EXTENSION_MODULE "" ON) option(EXECUTORCH_BUILD_EXTENSION_TENSOR "" ON) @@ -760,6 +762,8 @@ An ETRecord is an artifact generated at the time of export that contains model g In your export script, after calling `to_edge()` and `to_executorch()`, call `generate_etrecord()` with the `EdgeProgramManager` from `to_edge()` and the `ExecuTorchProgramManager` from `to_executorch()`. Make sure to copy the `EdgeProgramManager`, as the call to `to_backend()` mutates the graph in-place. ``` +# export_nanogpt.py + import copy from executorch.devtools import generate_etrecord @@ -810,23 +814,24 @@ if (result.buf != nullptr && result.size > 0) { Additionally, update CMakeLists.txt to build with Developer Tools and enable events to be traced and logged into ETDump: ``` +option(EXECUTORCH_ENABLE_EVENT_TRACER "" ON) option(EXECUTORCH_BUILD_DEVTOOLS "" ON) # ... target_link_libraries( - nanogpt_runner - PRIVATE - executorch - extension_module_static # Provides the Module class - optimized_native_cpu_ops_lib # Provides baseline cross-platform kernels - xnnpack_backend # Provides the XNNPACK CPU acceleration backend + # ... omit existing ones etdump) # Provides event tracing and logging target_compile_options(executorch PUBLIC -DET_EVENT_TRACER_ENABLED) target_compile_options(portable_ops_lib PUBLIC -DET_EVENT_TRACER_ENABLED) ``` -Run the runner, you will see “etdump.etdp” generated. +Build and run the runner, you will see a file named “etdump.etdp” is generated. (Note that this time we build in release mode to get around a flatccrt build limitation.) +```bash +(rm -rf cmake-out && mkdir cmake-out && cd cmake-out && cmake -DCMAKE_BUILD_TYPE=Release ..) +cmake --build cmake-out -j10 +./cmake-out/nanogpt_runner +``` #### Analyze with Inspector APIs