feat(perf): Flamegraphs for test program execution benchmarks

[aakoshh]

Description

Problem*

Resolves #6186

Summary*

• Breaks up the nargo_cli/benches/criterion.rs into two steps: 1) a compilation step which happens only once and 2) the execution of the already compiled program.
• Limits the code in the benchmark loop to be only the circuit execution, excluding the parsing of the program and its inputs and the saving of results as the execute command would do.

These are the timings of just the circuit execution:

eddsa_execute           time:   [111.13 ms 111.19 ms 111.25 ms]
regression_execute      time:   [151.27 µs 151.84 µs 152.66 µs]
struct_execute          time:   [602.39 ns 606.05 ns 609.77 ns]

The flame graphs look as follows (you can right click and use the "Save Linked File As..." to download it and open it in a browser to make it interactive):

• eddsa:
• regression:
• struct:

To generate them run the following commands:

./scripts/benchmark_start.sh
cargo bench -p nargo_cli --bench criterion -- --profile-time=30
./scripts/benchmark_stop.sh

Additional Context

The problem with the current nargo_cli/benches/criterion.rs was that it executed the nargo execute --program-dir ... --force command as a sub-process, and the profiler that creates the Flamegraph only sampled the criterion executor itself, not what the subprocess was doing.

My first idea to get the flame graphs to include the actual execution was to turn nargo_cli into a library and a binary (af19dfc), so that we can import the commands into the benchmark and run them in the same process. This resulted in the following flame graphs:

• eddsa:
• regression:
• struct:

These include the entire ExecuteCommand::run command, which unfortunately results in the flame graph dominated by parsing logic, reading inputs and writing outputs to files. In fact in all but the eddsa example the execute_circuit was so fast I can't even find it on the flamegraph.

These are the timings for the command execution per test program:

eddsa_execute           time:   [984.12 ms 988.74 ms 993.95 ms]
regression_execute      time:   [71.240 ms 71.625 ms 71.957 ms]
struct_execute          time:   [68.447 ms 69.414 ms 70.438 ms]

For this reason I rolled back the library+binary change and broke up the execution further by parsing the program into a CompiledProgram once and calling execute_program in the benchmark loop without saving the results.

I copied two utility functions to read the program artefact and the input map. Not sure these are worth moving around, even the errors they raise a CLI specific.

Documentation*

Check one:

• No documentation needed.
• Documentation included in this PR.
• [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt on default settings.

[TomAFrench]

For this reason I'll undo the library+binary change and try to break up the execution further by parsing the program into a CompiledProgram once and calling execute_program in the benchmark loop without saving the results.

Agreed, the usage of rayon is making these pretty difficult to gain insights from.

[michaeljklein]

Tiny change, otherwise LGTM

By the way, I see some explicit bench Noir programs:

• test_programs/benchmarks/*
• execution_success/bench_2_to_17
• test_programs/execution_success/bench_ecdsa_secp256k1

Would it make sense to add these to the selected_tests?

[aakoshh]

By the way, I see some explicit bench Noir programs:
Would it make sense to add these to the selected_tests?

Thanks, I haven't noticed these! Added all programs from the benchmarks directory to the selection. The execution_success/bench_* ones seem very similar to those.

[michaeljklein]

LGTM