From 924ff5a3514943f613dbbc310ca38fba602fc797 Mon Sep 17 00:00:00 2001 From: Taekyung Heo <7621438+TaekyungHeo@users.noreply.github.com> Date: Thu, 9 May 2024 14:19:59 -0400 Subject: [PATCH] Update USER_GUIDE.md --- INSTALL.md | 137 -------------------------------------------------- USER_GUIDE.md | 122 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 122 insertions(+), 137 deletions(-) delete mode 100644 INSTALL.md create mode 100644 USER_GUIDE.md diff --git a/INSTALL.md b/INSTALL.md deleted file mode 100644 index f0f3d51b..00000000 --- a/INSTALL.md +++ /dev/null @@ -1,137 +0,0 @@ -# Chakra -## Installation -We use `pip` to install/uninstall the `chakra` package: -```shell -# Install package from source -$ pip install . - -# Install latest from github -$ pip install https://github.com/mlcommons/chakra/archive/refs/heads/main.zip - -# Install specific revision from github -$ pip install https://github.com/mlcommons/chakra/archive/ae7c671db702eb1384015bb2618dc753eed787f2.zip - -# Uninstall package -$ pip uninstall chakra -``` - -## Execution Trace Converter (et_converter) -This tool converts execution traces into the Chakra format. -This converter supports three types of formats: ASTRA-sim text files, FlexFlow, and PyTorch. - -You can use the following commands for each input type. - -### ASTRA-sim Text Files -```shell -$ python -m chakra.et_converter.et_converter\ - --input_type Text\ - --input_filename \ - --output_filename \ - --num_npus \ - --num_passes -``` - -### FlexFlow Execution Graphs -```shell -$ python -m chakra.et_converter.et_converter\ - --input_type FlexFlow\ - --input_filename \ - --output_filename \ - --npu_frequency -``` - -### PyTorch Execution Graphs -```shell -$ python -m chakra.et_converter.et_converter\ - --input_type PyTorch\ - --input_filename \ - --output_filename -``` - -## Execution Trace Generator (et_generator) -This is an execution trace generator that generates synthetic execution traces. -A user can define a new function in the generator to generate new synthetic execution traces. -You can follow the commands below to run the generator. -```shell -$ python -m chakra.et_generator.et_generator\ - --num_npus -``` - -## Execution Trace Visualizer (et_visualizer) -This tool visualizes a given execution trace (ET) by converting the ET to a graph in various supported formats: PDF, Graphviz (dot), or GraphML. -The output format is determined by the file extension (postfix) of the output filename. -For PDF and Graphviz formats, use ".pdf" and ".dot" extensions respectively. -For GraphML, suitable for visualizing large-scale graphs, use the ".graphml" extension. -The PDF and Graphviz formats are generated using the Graphviz library, while the GraphML format is generated using the NetworkX library. -Graphviz files can be visualized with a Graphviz viewer such as https://dreampuf.github.io/GraphvizOnline/. -For visualizing GraphML files, you can use Gephi (https://gephi.org/). - -Run the tool with the following command: -```shell -$ python -m chakra.et_visualizer.et_visualizer\ - --input_filename \ - --output_filename -``` - -The input_filename is the path to the execution trace you want to visualize, and the output_filename is the name of the output file you want to create. -Remember to specify the correct extension in the output_filename to select the desired output format. - -## Timeline Visualizer (timeline_visualizer) -This tool visualizes the execution timeline of a given execution trace (ET). - -You can run this timeline visualizer with the following command. -```shell -$ python -m chakra.timeline_visualizer.timeline_visualizer\ - --input_filename \ - --output_filename \ - --num_npus \ - --npu_frequency -``` - -The input file is an execution trace file in csv, and the output file is a json file. -The input file format is shown below. -```csv -issue,=npu_id,=curr_cycle,=node_id,=node_name -callback,=npu_id,=curr_cycle,=node_id,=node_name -issue,=npu_id,=curr_cycle,=node_id,=node_name -issue,=npu_id,=curr_cycle,=node_id,=node_name -callback,=npu_id,=curr_cycle,=node_id,=node_name -callback,=npu_id,=curr_cycle,=node_id,=node_name -... -``` -As this tool requires an execution trace of an ET, a simulator has to print out execution traces. -The output json file is chrome-tracing-compatible. -When you open the file with `chrome://tracing`, you will see an execution timeline like the one below. -![](doc/timeline_visualizer.png) - -## Execution Trace Feeder (et_feeder) -This is a trace feeder that feeds dependency-free nodes to a simulator. -Therefore, a simulator has to import this feeder as a library. -Currently, ASTRA-sim is the only simulator that supports the trace feeder. -You can run execution traces on ASTRA-sim with the following commands. -``` -$ git clone --recurse-submodules git@github.com:astra-sim/astra-sim.git -$ cd astra-sim -$ git checkout Chakra -$ git submodule update --init --recursive -$ cd extern/graph_frontend/chakra/ -$ git checkout main -$ cd - -$ ./build/astra_analytical/build.sh -c - -$ cd extern/graph_frontend/chakra/ -$ python -m chakra.et_generator.et_generator\ - --num_npus - -$ cd - -$ ./run.sh -``` - -## Execution Trace Jsonizer (et_jsonizer) -This tool prints the nodes within execution traces for better comprehension. -The printed information includes the node's id, name, type, and any associated metadata, which are all outputted in a user-friendly text format. -``` -$ python -m chakra.et_jsonizer.et_jsonizer\ - --input_filename \ - --output_filename -``` diff --git a/USER_GUIDE.md b/USER_GUIDE.md new file mode 100644 index 00000000..2a1b95fb --- /dev/null +++ b/USER_GUIDE.md @@ -0,0 +1,122 @@ +# Chakra User Guide +## Installation +### Step 1: Set up a Virtual Environment +It's advisable to create a virtual environment using Python 3.10.2. + +```bash +# Create a virtual environment +$ python3 -m venv chakra_env + +# Activate the virtual environment +$ source chakra_env/bin/activate +``` + +### Step 2: Install Chakra +With the virtual environment activated, install the Chakra package using pip. + +```bash +# Install package from source +$ pip install . + +# Install latest from GitHub +$ pip install https://github.com/mlcommons/chakra/archive/refs/heads/main.zip + +# Install specific revision from GitHub +$ pip install https://github.com/mlcommons/chakra/archive/ae7c671db702eb1384015bb2618dc753eed787f2.zip +``` + +### Step 3: Install PARAM +Installing PARAM is necessary for Chakra to function properly as it imports essential components from it. + +```bash +$ git clone git@github.com:facebookresearch/param.git +$ cd param/train/compute/python/ +$ git checkout c83ce8429110a86549c40fec5a01acbd9fbd54a4 +$ pip install . +``` + +### Step 4: Uninstalling Chakra +To uninstall Chakra, use the following command within the virtual environment. + +```bash +$ pip uninstall chakra +``` + +## Tools Overview +### Execution Trace Link (chakra_trace_link) +Merge PyTorch Chakra host trace and Kineto trace to encode GPU operators into the output execution trace. + +```bash +$ chakra_trace_link \ + --pytorch-et-file /path/to/pytorch_et \ + --kineto-file /path/to/kineto \ + --output-file /path/to/merged_et +``` + +### Execution Trace Converter (chakra_converter) +Converts the merged execution traces into the Chakra schema. + +```bash +$ chakra_converter \ + --input_filename /path/to/merged_et \ + --output_filename /path/to/chakra_et \ + --input_type +``` + +### Execution Trace Feeder (et_feeder) +The Execution Trace Feeder (et_feeder) is a C++ library designed to feed Chakra traces into any compatible C++ simulator. This library specifically provides dependency-free nodes to a simulator, which must import the feeder as a library. Currently, ASTRA-sim is the only simulator that supports this trace feeder. Below are the commands to run execution traces on ASTRA-sim: +```bash +$ git clone --recurse-submodules git@github.com:astra-sim/astra-sim.git +$ cd astra-sim +$ git checkout Chakra +$ git submodule update --init --recursive +$ cd extern/graph_frontend/chakra/ +$ git checkout main +$ cd - +$ ./build/astra_analytical/build.sh -c + +$ cd extern/graph_frontend/chakra/ +$ python -m chakra.et_generator.et_generator\ + --num_npus + +$ cd - +$ ./run.sh +``` + +### Execution Trace Visualizer (chakra_visualizer) +This tool visualizes execution traces in various formats. Here is an example command: + +```bash +$ chakra_visualizer \ + --input_filename /path/to/chakra_et + --output_filename /path/to/output.[graphml|pdf|dot] +``` + +### Execution Trace Jsonizer (chakra_jsonizer) +Provides a readable JSON format of execution traces: + +```bash +$ chakra_jsonizer \ + --input_filename /path/to/chakra_et \ + --output_filename /path/to/output_json +``` + +### Timeline Visualizer (chakra_timeline_visualizer) +Visualizes the execution timeline of traces. This tool serves as a reference implementation for visualizing the simulation of Chakra traces. After simulating Chakra traces, you can visualize the timeline of operator executions. Update the simulator to present when operators are issued and completed. Below is the format needed: +```csv +issue,=npu_id,=curr_cycle,=node_id,=node_name +callback,=npu_id,=curr_cycle,=node_id,=node_name +... +``` + +You can visualize the timeline with the command below. +```bash +$ chakra_timeline_visualizer \ + --input_filename /path/to/input.csv \ + --output_filename /path/to/output.json \ + --num_npus 4 \ + --npu_frequency 1.5GHz +``` + +When you open the output file with `chrome://tracing`, you will see an execution timeline like the one below. +![](doc/timeline_visualizer.png)