Skip to content

Latest commit

 

History

History
93 lines (57 loc) · 3.88 KB

weights-benchmarking.md

File metadata and controls

93 lines (57 loc) · 3.88 KB
Raw

Weights benchmarking

What is the reference machine?

For now (09/2022), it's a Raspberry Pi 4 Model B - 4GB with an SSD connected via USB3.

To cross-compile the benchmarks binary for armv7:

./scripts/cross-build-arm.sh --features runtime-benchmarks

The cross compiled binary is generated here: target/armv7-unknown-linux-gnueabihf/release/duniter

How to benchmarks weights of a Call/Hook/Pallet

  1. Create the benchmarking tests. See commit f5f2ae969ac592ba9957b0e40e18d6e4b0048473 for a complete real example.

  2. Run the benchmark test on your local machine:

cargo test -p <pallet> --features runtime-benchmarks
  1. If the benchmark tests compiles and pass, compile the binary with benchmarks on your local machine:
cargo build --release --features runtime-benchmarks
  1. Run the benchmarks on your local machine (to test if it work with a real runtime). See 0d1232cd0d8b5809e1586b48376f8952cebc0d27 for a complete real example. The command is:
duniter benchmark pallet --chain=CHAINSPEC --steps=50 --repeat=20 --pallet=<pallet> --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/common/src/weights/

  1. Use the generated file content to create the WeightInfo trait and the () dummy implementation in pallets/<pallet>/src/weights.rs. Then use the WeightInfo trait in the real code of the pallet. See 62dcc17f2c0b922e883fbc6337a9e7da97fc3218 for a complete real example.

  2. Redo steps 3. and 4. on the reference machine.

  3. Use the runtime/common/src/weights/pallet_<pallet>.rs generated on the reference machine in the runtimes configuration. See af62a3b9cfc42d6653b3a957836f58540c18e65a for a complete real example.

Note 1: Use relevant chainspec for the benchmarks in place of CHAINSPEC, for example --chain=dev.

Note 2: If the reference machine does not support wasmtime, you should replace --wasm-execution=compiled by --wasm-execution=interpreted-i-know-what-i-do.

Generate base block benchmarking

  1. Build binary for reference machine and copy it on reference machine.
  2. Run base block benchmarks command:
duniter benchmark overhead --chain=dev --execution=wasm --wasm-execution=compiled --weight-path=./runtime/common/src/weights/ --warmup=10 --repeat=100
  1. Commit changes and open an MR.

Generate storage benchmarking

  1. Build binary for reference machine and copy it on reference machine.
  2. Copy a DB on reference machine (on ssd), example: scp -r -P 37015 tmp/t1 pi@192.168.1.188:/mnt/ssd1/duniter-v2s/
  3. Run storage benchmarks command, example:
duniter benchmark storage -d=/mnt/ssd1/duniter-v2s/t1 --chain=gdev --mul=2 --weight-path=. --state-version=1
  1. Copy the generated file paritydb_weights.rs in the codebase in folder runtime/common/src/weights/.
  2. Commit changes and open an MR.

How to Write Benchmarks

Calls

Ensure that any extrinsic call is benchmarked using the most computationally intensive path, i.e., the worst-case scenario.

Hooks

Benchmark each hook to determine the weight consumed by it; hence, it is essential to benchmark all possible paths.

Handlers and Internal Functions

When designing handlers and internal functions, it is advisable to avoid having them return weight for the following reasons:

  1. Simplified Benchmarking: Writing benchmarks for hooks or calls where handlers and internal functions are utilized becomes more straightforward.
  2. Reduced Benchmarking Complexity: By directly measuring execution and overhead in a single pass, the number of benchmarks is minimized.
  3. Enhanced Readability: Understanding that weight accounting occurs at the outermost level improves the overall readability of the code.

One notable exception is the internal functions called in hooks like on_idle or on_initialize that can be easier to benchmark separately when the hook contains numerous branching.