Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: measure plonk rounds (AztecProtocol#3065)
Features: - Adds round benchmarking executables - Adds benchmarking circuits of size `2**13` to `2**18` - Refactoring ultrahonk flame graph: Reference with 'Ultra Honk Round Breakdown' below <img width="1512" alt="Screenshot 2023-10-26 at 8 03 47 PM" src="https://github.com/AztecProtocol/aztec-packages/assets/163993/ae391982-34d0-47c8-bed6-77c3cb9cd816"> ultraplonk flame graph: Reference with 'Ultra Plonk Round Breakdown' below To read this one, you have to count plonk_round from round 1 through 6. <img width="1501" alt="Screenshot 2023-10-26 at 8 24 40 PM" src="https://github.com/AztecProtocol/aztec-packages/assets/163993/504e7dd7-ba6a-473e-880c-0362a3a73bc9"> Current stats: ultrahonk ~43-49% faster than ultraplonk on 10x ECDSA, ~25% faster on 2**18 circuit basic arithmetic circuit Example output of scripts/benchmarks.sh: MACBOOK STATS 10 cores <details><summary>Standard Plonk</summary> ``` Unable to determine clock rate from sysctl: hw.cpufrequency: No such file or directory This does not affect benchmark measurements, only the metadata output. 2023-10-26T19:40:15-04:00 Running ./bin/standard_plonk_bench Run on (10 X 24.1218 MHz CPU s) CPU Caches: L1 Data 64 KiB L1 Instruction 128 KiB L2 Unified 4096 KiB (x10) Load Average: 63.73, 26.17, 14.93 --------------------------------------------------------------------------------- Benchmark Time CPU Iterations --------------------------------------------------------------------------------- construct_proof_standard_power_of_2/13 82.0 ms 70.9 ms 9 construct_proof_standard_power_of_2/14 132 ms 117 ms 6 construct_proof_standard_power_of_2/15 223 ms 202 ms 3 construct_proof_standard_power_of_2/16 413 ms 370 ms 2 construct_proof_standard_power_of_2/17 746 ms 716 ms 1 construct_proof_standard_power_of_2/18 1418 ms 1364 ms 1 ``` </details> <details><summary>Ultra Honk Round Breakdown</summary> ``` Unable to determine clock rate from sysctl: hw.cpufrequency: No such file or directory This does not affect benchmark measurements, only the metadata output. 2023-10-26T19:40:24-04:00 Running ./bin/ultra_honk_rounds_bench Run on (10 X 24.121 MHz CPU s) CPU Caches: L1 Data 64 KiB L1 Instruction 128 KiB L2 Unified 4096 KiB (x10) Load Average: 59.43, 25.90, 14.90 --------------------------------------------------------------------------------------- Benchmark Time CPU Iterations --------------------------------------------------------------------------------------- ROUND_PREAMBLE/iterations:1 16.7 ms 16.6 ms 1 ROUND_WIRE_COMMITMENTS/iterations:1 205 ms 169 ms 1 ROUND_SORTED_LIST_ACCUMULATOR/iterations:1 189 ms 169 ms 1 ROUND_GRAND_PRODUCT_COMPUTATION/iterations:1 452 ms 419 ms 1 ROUND_RELATION_CHECK 1426 ms 1407 ms 1 ROUND_ZEROMORPH 1204 ms 1158 ms 1 ``` </details> <details><summary>Ultra Plonk Round Breakdown</summary> ``` Unable to determine clock rate from sysctl: hw.cpufrequency: No such file or directory This does not affect benchmark measurements, only the metadata output. 2023-10-26T19:40:56-04:00 Running ./bin/ultra_plonk_rounds_bench Run on (10 X 24.1204 MHz CPU s) CPU Caches: L1 Data 64 KiB L1 Instruction 128 KiB L2 Unified 4096 KiB (x10) Load Average: 33.61, 23.16, 14.35 ------------------------------------------------------------------------------------ Benchmark Time CPU Iterations ------------------------------------------------------------------------------------ ROUND_PREAMBLE/iterations:1 121 ms 112 ms 1 ROUND_FIRST_WIRE_COMMITMENTS 679 ms 657 ms 1 ROUND_SECOND_FIAT_SHAMIR_ETA 636 ms 561 ms 1 ROUND_THIRD_FIAT_SHAMIR_BETA_GAMMA 1255 ms 972 ms 1 ROUND_FOURTH_FIAT_SHAMIR_ALPHA_AND_COMMIT 1944 ms 1893 ms 1 ROUND_FIFTH_COMPUTE_QUOTIENT_EVALUTION 245 ms 187 ms 4 ROUND_SIXTH_BATCH_OPEN 569 ms 558 ms 1 ``` </details> <details><summary>Ultra Honk</summary> ``` Unable to determine clock rate from sysctl: hw.cpufrequency: No such file or directory This does not affect benchmark measurements, only the metadata output. 2023-10-26T19:42:43-04:00 Running ./bin/ultra_honk_bench Run on (10 X 24.1212 MHz CPU s) CPU Caches: L1 Data 64 KiB L1 Instruction 128 KiB L2 Unified 4096 KiB (x10) Load Average: 10.61, 17.83, 13.27 --------------------------------------------------------------------------------------- Benchmark Time CPU Iterations --------------------------------------------------------------------------------------- construct_proof_ultrahonk/sha256 510 ms 479 ms 2 construct_proof_ultrahonk/keccak 1833 ms 1749 ms 1 construct_proof_ultrahonk/ecdsa_verification 3396 ms 3193 ms 1 construct_proof_ultrahonk/merkle_membership 297 ms 279 ms 3 construct_proof_ultrahonk_power_of_2/13 121 ms 112 ms 6 construct_proof_ultrahonk_power_of_2/14 187 ms 177 ms 4 construct_proof_ultrahonk_power_of_2/15 320 ms 294 ms 2 construct_proof_ultrahonk_power_of_2/16 547 ms 524 ms 1 construct_proof_ultrahonk_power_of_2/17 1029 ms 991 ms 1 construct_proof_ultrahonk_power_of_2/18 1936 ms 1875 ms 1 ``` </details> <details><summary>Ultra Plonk</summary> ``` Unable to determine clock rate from sysctl: hw.cpufrequency: No such file or directory This does not affect benchmark measurements, only the metadata output. 2023-10-26T19:43:02-04:00 Running ./bin/ultra_plonk_bench Run on (10 X 24.1209 MHz CPU s) CPU Caches: L1 Data 64 KiB L1 Instruction 128 KiB L2 Unified 4096 KiB (x10) Load Average: 9.93, 17.21, 13.15 ---------------------------------------------------------------------------------------- Benchmark Time CPU Iterations ---------------------------------------------------------------------------------------- construct_proof_ultraplonk/sha256 672 ms 628 ms 1 construct_proof_ultraplonk/keccak 2477 ms 2374 ms 1 construct_proof_ultraplonk/ecdsa_verification 5086 ms 4582 ms 1 construct_proof_ultraplonk/merkle_membership 459 ms 346 ms 2 construct_proof_ultraplonk_power_of_2/13 184 ms 126 ms 5 construct_proof_ultraplonk_power_of_2/14 293 ms 202 ms 3 construct_proof_ultraplonk_power_of_2/15 463 ms 349 ms 2 construct_proof_ultraplonk_power_of_2/16 772 ms 601 ms 1 construct_proof_ultraplonk_power_of_2/17 1410 ms 1229 ms 1 construct_proof_ultraplonk_power_of_2/18 2468 ms 2394 ms 1 ``` </details> MAINFRAME STATS, ~128 cores <details><summary>Standard Plonk</summary> ``` 2023-10-26T23:39:37+00:00 Running ./bin/standard_plonk_bench Run on (128 X 2649.99 MHz CPU s) CPU Caches: L1 Data 32 KiB (x64) L1 Instruction 32 KiB (x64) L2 Unified 512 KiB (x64) L3 Unified 32768 KiB (x8) Load Average: 15.48, 13.93, 12.70 --------------------------------------------------------------------------------- Benchmark Time CPU Iterations --------------------------------------------------------------------------------- construct_proof_standard_power_of_2/13 128 ms 112 ms 6 construct_proof_standard_power_of_2/14 150 ms 131 ms 5 construct_proof_standard_power_of_2/15 182 ms 155 ms 4 construct_proof_standard_power_of_2/16 261 ms 203 ms 3 construct_proof_standard_power_of_2/17 410 ms 324 ms 2 construct_proof_standard_power_of_2/18 914 ms 710 ms 1 ``` </details> <details><summary>Ultra Honk Round Breakdown</summary> ``` 2023-10-26T23:39:46+00:00 Running ./bin/ultra_honk_rounds_bench Run on (128 X 2649.99 MHz CPU s) CPU Caches: L1 Data 32 KiB (x64) L1 Instruction 32 KiB (x64) L2 Unified 512 KiB (x64) L3 Unified 32768 KiB (x8) Load Average: 16.06, 14.13, 12.78 --------------------------------------------------------------------------------------- Benchmark Time CPU Iterations --------------------------------------------------------------------------------------- ROUND_PREAMBLE/iterations:1 29.8 ms 29.7 ms 1 ROUND_WIRE_COMMITMENTS/iterations:1 100 ms 53.4 ms 1 ROUND_SORTED_LIST_ACCUMULATOR/iterations:1 110 ms 87.6 ms 1 ROUND_GRAND_PRODUCT_COMPUTATION/iterations:1 190 ms 116 ms 1 ROUND_RELATION_CHECK 467 ms 300 ms 2 ROUND_ZEROMORPH 921 ms 807 ms 1 ``` </details> <details><summary>Ultra Plonk Round Breakdown</summary> ``` 2023-10-26T23:40:21+00:00 Running ./bin/ultra_plonk_rounds_bench Run on (128 X 2649.99 MHz CPU s) CPU Caches: L1 Data 32 KiB (x64) L1 Instruction 32 KiB (x64) L2 Unified 512 KiB (x64) L3 Unified 32768 KiB (x8) Load Average: 16.26, 14.76, 13.06 ------------------------------------------------------------------------------------ Benchmark Time CPU Iterations ------------------------------------------------------------------------------------ ROUND_PREAMBLE/iterations:1 238 ms 229 ms 1 ROUND_FIRST_WIRE_COMMITMENTS 375 ms 270 ms 2 ROUND_SECOND_FIAT_SHAMIR_ETA 282 ms 212 ms 3 ROUND_THIRD_FIAT_SHAMIR_BETA_GAMMA 675 ms 513 ms 1 ROUND_FOURTH_FIAT_SHAMIR_ALPHA_AND_COMMIT 775 ms 596 ms 1 ROUND_FIFTH_COMPUTE_QUOTIENT_EVALUTION 76.0 ms 56.9 ms 13 ROUND_SIXTH_BATCH_OPEN 357 ms 291 ms 2 ``` </details> <details><summary>Ultra Honk</summary> ``` 2023-10-26T23:43:17+00:00 Running ./bin/ultra_honk_bench Run on (128 X 3444.11 MHz CPU s) CPU Caches: L1 Data 32 KiB (x64) L1 Instruction 32 KiB (x64) L2 Unified 512 KiB (x64) L3 Unified 32768 KiB (x8) Load Average: 36.58, 26.06, 17.88 --------------------------------------------------------------------------------------- Benchmark Time CPU Iterations --------------------------------------------------------------------------------------- construct_proof_ultrahonk/sha256 344 ms 277 ms 3 construct_proof_ultrahonk/keccak 913 ms 706 ms 1 construct_proof_ultrahonk/ecdsa_verification 1645 ms 1270 ms 1 construct_proof_ultrahonk/merkle_membership 231 ms 191 ms 4 construct_proof_ultrahonk_power_of_2/13 149 ms 127 ms 5 construct_proof_ultrahonk_power_of_2/14 184 ms 152 ms 5 construct_proof_ultrahonk_power_of_2/15 243 ms 193 ms 4 construct_proof_ultrahonk_power_of_2/16 365 ms 278 ms 3 construct_proof_ultrahonk_power_of_2/17 545 ms 423 ms 2 construct_proof_ultrahonk_power_of_2/18 1053 ms 750 ms 1 ``` </details> <details><summary>Ultra Plonk</summary> ``` 2023-10-26T23:43:37+00:00 Running ./bin/ultra_plonk_bench Run on (128 X 2649.99 MHz CPU s) CPU Caches: L1 Data 32 KiB (x64) L1 Instruction 32 KiB (x64) L2 Unified 512 KiB (x64) L3 Unified 32768 KiB (x8) Load Average: 28.74, 24.91, 17.67 ---------------------------------------------------------------------------------------- Benchmark Time CPU Iterations ---------------------------------------------------------------------------------------- construct_proof_ultraplonk/sha256 473 ms 353 ms 2 construct_proof_ultraplonk/keccak 1379 ms 1044 ms 1 construct_proof_ultraplonk/ecdsa_verification 2363 ms 1785 ms 1 construct_proof_ultraplonk/merkle_membership 275 ms 240 ms 3 construct_proof_ultraplonk_power_of_2/13 196 ms 181 ms 4 construct_proof_ultraplonk_power_of_2/14 226 ms 203 ms 3 construct_proof_ultraplonk_power_of_2/15 287 ms 238 ms 3 construct_proof_ultraplonk_power_of_2/16 363 ms 300 ms 2 construct_proof_ultraplonk_power_of_2/17 656 ms 532 ms 1 construct_proof_ultraplonk_power_of_2/18 1326 ms 1019 ms 1 ``` --------- Co-authored-by: ludamad <adam@aztecprotocol.com> Co-authored-by: codygunton <codygunton@gmail.com>
- Loading branch information