Skip to content

Commit

Permalink
LLM manual fixes on profiling part (#6315)
Browse files Browse the repository at this point in the history
LLM manual fixes on etdump generation (#6299)

Summary:
Pull Request resolved: #6299

Fix the problem that LLM Manual profiling part cannot generate ETDump

Reviewed By: Gasoonjia

Differential Revision: D64485281

fbshipit-source-id: 864dda92112f50850feb94234658fdf6ca3b5922
(cherry picked from commit ec1c431)

Co-authored-by: Olivia Liu <olivialpx@meta.com>
  • Loading branch information
pytorchbot and Olivia-liu authored Oct 17, 2024
1 parent a060791 commit 573ad96
Showing 1 changed file with 12 additions and 7 deletions.
19 changes: 12 additions & 7 deletions docs/source/llm/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,7 @@ set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)
# Set options for executorch build.
option(EXECUTORCH_ENABLE_LOGGING "" ON)
option(EXECUTORCH_BUILD_EXTENSION_DATA_LOADER "" ON)
option(EXECUTORCH_BUILD_EXTENSION_MODULE "" ON)
option(EXECUTORCH_BUILD_EXTENSION_TENSOR "" ON)
Expand Down Expand Up @@ -515,6 +516,7 @@ set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)
# Set options for executorch build.
option(EXECUTORCH_ENABLE_LOGGING "" ON)
option(EXECUTORCH_BUILD_EXTENSION_DATA_LOADER "" ON)
option(EXECUTORCH_BUILD_EXTENSION_MODULE "" ON)
option(EXECUTORCH_BUILD_EXTENSION_TENSOR "" ON)
Expand Down Expand Up @@ -760,6 +762,8 @@ An ETRecord is an artifact generated at the time of export that contains model g
In your export script, after calling `to_edge()` and `to_executorch()`, call `generate_etrecord()` with the `EdgeProgramManager` from `to_edge()` and the `ExecuTorchProgramManager` from `to_executorch()`. Make sure to copy the `EdgeProgramManager`, as the call to `to_backend()` mutates the graph in-place.

```
# export_nanogpt.py
import copy
from executorch.devtools import generate_etrecord
Expand Down Expand Up @@ -810,23 +814,24 @@ if (result.buf != nullptr && result.size > 0) {
Additionally, update CMakeLists.txt to build with Developer Tools and enable events to be traced and logged into ETDump:

```
option(EXECUTORCH_ENABLE_EVENT_TRACER "" ON)
option(EXECUTORCH_BUILD_DEVTOOLS "" ON)
# ...
target_link_libraries(
nanogpt_runner
PRIVATE
executorch
extension_module_static # Provides the Module class
optimized_native_cpu_ops_lib # Provides baseline cross-platform kernels
xnnpack_backend # Provides the XNNPACK CPU acceleration backend
# ... omit existing ones
etdump) # Provides event tracing and logging
target_compile_options(executorch PUBLIC -DET_EVENT_TRACER_ENABLED)
target_compile_options(portable_ops_lib PUBLIC -DET_EVENT_TRACER_ENABLED)
```
Run the runner, you will see “etdump.etdp” generated.
Build and run the runner, you will see a file named “etdump.etdp” is generated. (Note that this time we build in release mode to get around a flatccrt build limitation.)
```bash
(rm -rf cmake-out && mkdir cmake-out && cd cmake-out && cmake -DCMAKE_BUILD_TYPE=Release ..)
cmake --build cmake-out -j10
./cmake-out/nanogpt_runner
```

#### Analyze with Inspector APIs

Expand Down

0 comments on commit 573ad96

Please sign in to comment.