Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move all buses and tables to LogUp-GKR #1505

Merged
merged 9 commits into from
Nov 4, 2024
Merged

Move all buses and tables to LogUp-GKR #1505

merged 9 commits into from
Nov 4, 2024

Conversation

plafer
Copy link
Contributor

@plafer plafer commented Sep 19, 2024 •
edited
Loading

Left to do:

  • fix bug in MidenLogUpGkrEval
  • Remove Op group table aux column & aux constraints
  • cleanup (i.e. address all TODO(plafer)

@plafer plafer force-pushed the plafer-offload-opgrouptable-to-gkr branch from 5e9f719 to bc89f1f Compare September 20, 2024 19:27
@plafer plafer added the no changelog This PR does not require an entry in the `CHANGELOG.md` file label Sep 20, 2024
@plafer
Copy link
Contributor Author

plafer commented Sep 20, 2024 •
edited by bobbinth
Loading

The preliminary results look pretty bad with the current branch more than doubling the proving time. The entire program on next is proved in 2.2s, while just proving GKR on this branch takes 3.2s. This can be attributed to:

  1. input layer max_degree going from 5 to 10
  2. Number of fractions going from 8 to 32
    • That is our input layer has 4x more fractions (or 8x more elements)

To fix this performance gaps, as previously discussed, we can introduce a is_halt column (should reduce degree by 3) to reduce the max_degree.

The radical increase in fractions is due to how to adapt the constraint equation for "standard multiset checks" (last equation in this section). Adding rows to the table becomes very expensive. That is,

$$ f_{g2} \cdot v_1 + f_{g4} \cdot \prod_{i=1}^3 v_i + f_{g8} \cdot (\prod_{i=1}^7 v_i) $$

when adapted to LogUp becomes

$$ \frac{f_{g2}}{v_1} + \frac{f_{g4}}{v_1} + \frac{f_{g4}}{v_2} + \frac{f_{g4}}{v_3} + \frac{f_{g8}}{v_1} + \frac{f_{g8}}{v_2} + \frac{f_{g8}}{v_3} + \frac{f_{g8}}{v_4} + \frac{f_{g8}}{v_5} + \frac{f_{g8}}{v_6} + \frac{f_{g8}}{v_7} $$

which itself adds 11 fractions.

next

$ MIDEN_LOG=info RAYON_NUM_THREADS=8 ./target/optimized/miden example blake3  -n 50

INFO     prove_program [ 2.25s | 3.52% / 100.00% ]
INFO     ┝━ execute_program [ 122ms | 5.43% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 262144 steps (8% padded) in 122 ms
INFO     ┝━ build_domain [ 2.42ms | 0.11% ] trace_length: 262144 | lde_domain_size: 2097152
INFO     ┝━ commit_to_main_trace_segment [ 826ms | 0.00% / 36.70% ]
INFO     │  ┝━ extend_execution_trace [ 574ms | 25.50% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 252ms | 11.20% ] tree_depth: 21
INFO     ┝━ commit_to_aux_trace_segment [ 208ms | 0.00% / 9.22% ]
INFO     │  ┝━ extend_execution_trace [ 118ms | 5.23% ] num_cols: 7 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 89.7ms | 3.99% ] tree_depth: 21
INFO     ┝━ evaluate_constraints [ 657ms | 29.20% ] ce_domain_size: 2097152
INFO     ┝━ commit_to_constraint_evaluations [ 215ms | 0.00% / 9.54% ]
INFO     │  ┝━ build_composition_poly_columns [ 33.0ms | 1.46% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 101ms | 4.50% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 80.5ms | 3.58% ] tree_depth: 21
INFO     ┝━ build_deep_composition_poly [ 89.4ms | 3.97% ]
INFO     ┝━ evaluate_deep_composition_poly [ 17.4ms | 0.77% ]
INFO     ┝━ compute_fri_layers [ 22.1ms | 0.98% ] num_layers: 4
INFO     ┝━ determine_query_positions [ 2.07ms | 0.09% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 10.5ms | 0.47% ]
--------------------------------
Executed program in 2251 ms

logup-gkr branch

INFO     prove_program [ 3.17s | 2.24% / 100.00% ]
INFO     ┝━ execute_program [ 122ms | 3.87% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 262144 steps (8% padded) in 122 ms
INFO     ┝━ build_domain [ 2.60ms | 0.08% ] trace_length: 262144 | lde_domain_size: 2097152
INFO     ┝━ commit_to_main_trace_segment [ 835ms | 0.00% / 26.35% ]
INFO     │  ┝━ extend_execution_trace [ 580ms | 18.30% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 255ms | 8.05% ] commitment_domain_size: 2097152
INFO     ┝━ prove_gkr [ 567ms | 0.00% / 17.89% ]
INFO     │  ┝━ evaluate_logup_gkr_circuit [ 50.7ms | 1.60% ]
INFO     │  ┝━ prove_intermediate_layers [ 105ms | 3.31% ]
INFO     │  ┝━ build_mls_from_main_trace_segment [ 24.8ms | 0.78% ]
INFO     │  ┕━ prove_input_layer [ 386ms | 12.19% ]
INFO     ┝━ build_logup_gkr_columns [ 83.8ms | 2.65% ]
INFO     ┝━ commit_to_aux_trace_segment [ 196ms | 0.00% / 6.20% ]
INFO     │  ┝━ extend_execution_trace [ 119ms | 3.74% ] num_cols: 8 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 77.8ms | 2.46% ] commitment_domain_size: 2097152
INFO     ┝━ evaluate_constraints [ 862ms | 27.21% ] ce_domain_size: 2097152
INFO     ┝━ commit_to_constraint_evaluations [ 215ms | 0.00% / 6.79% ]
INFO     │  ┝━ build_composition_poly_columns [ 34.5ms | 1.09% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 101ms | 3.20% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 79.0ms | 2.49% ] log_domain_size: 21
INFO     ┝━ build_deep_composition_poly [ 167ms | 5.29% ]
INFO     ┝━ evaluate_deep_composition_poly [ 17.2ms | 0.54% ]
INFO     ┝━ compute_fri_layers [ 23.1ms | 0.73% ] num_layers: 4
INFO     ┝━ determine_query_positions [ 285µs | 0.01% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 4.74ms | 0.15% ]
--------------------------------
Executed program in 3167 ms

This branch

$ MIDEN_LOG=info RAYON_NUM_THREADS=8 ./target/optimized/miden example blake3  -n 50

INFO     prove_program [ 5.91s | 1.12% / 100.00% ]
INFO     ┝━ execute_program [ 121ms | 2.04% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 262144 steps (8% padded) in 120 ms
INFO     ┝━ build_domain [ 2.53ms | 0.04% ] trace_length: 262144 | lde_domain_size: 2097152
INFO     ┝━ commit_to_main_trace_segment [ 854ms | 0.00% / 14.44% ]
INFO     │  ┝━ extend_execution_trace [ 577ms | 9.76% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 276ms | 4.67% ] commitment_domain_size: 2097152
INFO     ┝━ prove_gkr [ 3.20s | 0.00% / 54.14% ]
INFO     │  ┝━ evaluate_logup_gkr_circuit [ 329ms | 5.56% ]
INFO     │  ┝━ prove_intermediate_layers [ 427ms | 7.23% ]
INFO     │  ┝━ build_mls_from_main_trace_segment [ 54.4ms | 0.92% ]
INFO     │  ┕━ prove_input_layer [ 2.39s | 40.43% ]
INFO     ┝━ build_logup_gkr_columns [ 138ms | 2.33% ]
INFO     ┝━ commit_to_aux_trace_segment [ 208ms | 0.00% / 3.51% ]
INFO     │  ┝━ extend_execution_trace [ 123ms | 2.08% ] num_cols: 7 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 84.7ms | 1.43% ] commitment_domain_size: 2097152
INFO     ┝━ evaluate_constraints [ 896ms | 15.16% ] ce_domain_size: 2097152
INFO     ┝━ commit_to_constraint_evaluations [ 212ms | 0.00% / 3.58% ]
INFO     │  ┝━ build_composition_poly_columns [ 34.3ms | 0.58% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 100.0ms | 1.69% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 77.7ms | 1.31% ] log_domain_size: 21
INFO     ┝━ build_deep_composition_poly [ 167ms | 2.83% ]
INFO     ┝━ evaluate_deep_composition_poly [ 17.1ms | 0.29% ]
INFO     ┝━ compute_fri_layers [ 25.9ms | 0.44% ] num_layers: 4
INFO     ┝━ determine_query_positions [ 710µs | 0.01% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 4.50ms | 0.08% ]
--------------------------------
Executed program in 5913 ms

@Al-Kindi-0
Copy link
Collaborator

Al-Kindi-0 commented Sep 20, 2024 •
edited
Loading

logup-gkr branch

============================================================
Generated a program to compute 50-th iteration of BLAKE3 1-to-1 hash; expected result: [664088198, 4259451035, 218783966, 2579530912, 3535070201, 4210849069, 2460906358, 3194996903]
--------------------------------
INFO     prove_program [ 10.9s | 1.70% / 100.00% ]
INFO     ┝━ execute_program [ 167ms | 1.53% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 262144 steps (8% padded) in 166 ms
INFO     ┝━ build_domain [ 2.74ms | 0.03% ] trace_length: 262144 | lde_domain_size: 2097152
INFO     ┝━ commit_to_main_trace_segment [ 3.71s | 0.00% / 34.13% ]
INFO     │  ┝━ extend_execution_trace [ 3.22s | 29.65% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 487ms | 4.48% ] commitment_domain_size: 2097152
INFO     ┝━ prove_gkr [ 1.59s | 0.00% / 14.66% ]
INFO     │  ┝━ evaluate_logup_gkr_circuit [ 118ms | 1.09% ]
INFO     │  ┝━ prove_intermediate_layers [ 572ms | 5.26% ]
INFO     │  ┝━ build_mls_from_main_trace_segment [ 117ms | 1.07% ]
INFO     │  ┕━ prove_input_layer [ 787ms | 7.24% ]
INFO     ┝━ build_logup_gkr_columns [ 137ms | 1.26% ]
INFO     ┝━ commit_to_aux_trace_segment [ 825ms | 0.00% / 7.59% ]
INFO     │  ┝━ extend_execution_trace [ 701ms | 6.44% ] num_cols: 8 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 124ms | 1.14% ] commitment_domain_size: 2097152
INFO     ┝━ evaluate_constraints [ 2.93s | 26.95% ] ce_domain_size: 2097152
INFO     ┝━ commit_to_constraint_evaluations [ 897ms | 0.00% / 8.25% ]
INFO     │  ┝━ build_composition_poly_columns [ 114ms | 1.05% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 659ms | 6.06% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 124ms | 1.14% ] log_domain_size: 21
INFO     ┝━ build_deep_composition_poly [ 262ms | 2.41% ]
INFO     ┝━ evaluate_deep_composition_poly [ 90.9ms | 0.84% ]
INFO     ┝━ compute_fri_layers [ 50.9ms | 0.47% ] num_layers: 4
INFO     ┝━ determine_query_positions [ 477µs | 0.00% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 21.5ms | 0.20% ]
--------------------------------
Executed program in 10874 ms
Stack outputs: [664088198, 4259451035, 218783966, 2579530912, 3535070201, 4210849069, 2460906358, 3194996903]
Execution proof size: 104 KB
Execution proof security: 96 bits
--------------------------------
INFO     verify_program [ 1.43ms | 100.00% ]
Execution verified in 1 ms

this branch

============================================================
Generated a program to compute 50-th iteration of BLAKE3 1-to-1 hash; expected result: [664088198, 4259451035, 218783966, 2579530912, 3535070201, 4210849069, 2460906358, 3194996903]
--------------------------------
INFO     prove_program [ 17.0s | 0.88% / 100.00% ]
INFO     ┝━ execute_program [ 165ms | 0.97% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 262144 steps (8% padded) in 164 ms
INFO     ┝━ build_domain [ 2.90ms | 0.02% ] trace_length: 262144 | lde_domain_size: 2097152
INFO     ┝━ commit_to_main_trace_segment [ 3.56s | 0.00% / 20.87% ]
INFO     │  ┝━ extend_execution_trace [ 3.18s | 18.66% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 376ms | 2.21% ] commitment_domain_size: 2097152
INFO     ┝━ prove_gkr [ 7.61s | 0.00% / 44.65% ]
INFO     │  ┝━ evaluate_logup_gkr_circuit [ 860ms | 5.05% ]
INFO     │  ┝━ prove_intermediate_layers [ 2.24s | 13.16% ]
INFO     │  ┝━ build_mls_from_main_trace_segment [ 229ms | 1.34% ]
INFO     │  ┕━ prove_input_layer [ 4.27s | 25.09% ]
INFO     ┝━ build_logup_gkr_columns [ 230ms | 1.35% ]
INFO     ┝━ commit_to_aux_trace_segment [ 886ms | 0.00% / 5.20% ]
INFO     │  ┝━ extend_execution_trace [ 696ms | 4.09% ] num_cols: 7 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 190ms | 1.12% ] commitment_domain_size: 2097152
INFO     ┝━ evaluate_constraints [ 3.14s | 18.44% ] ce_domain_size: 2097152
INFO     ┝━ commit_to_constraint_evaluations [ 883ms | 0.00% / 5.18% ]
INFO     │  ┝━ build_composition_poly_columns [ 113ms | 0.66% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 654ms | 3.84% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 116ms | 0.68% ] log_domain_size: 21
INFO     ┝━ build_deep_composition_poly [ 253ms | 1.49% ]
INFO     ┝━ evaluate_deep_composition_poly [ 92.8ms | 0.54% ]
INFO     ┝━ compute_fri_layers [ 46.0ms | 0.27% ] num_layers: 4
INFO     ┝━ determine_query_positions [ 1.99ms | 0.01% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 21.0ms | 0.12% ]
--------------------------------
Executed program in 17034 ms
Stack outputs: [664088198, 4259451035, 218783966, 2579530912, 3535070201, 4210849069, 2460906358, 3194996903]
Execution proof size: 109 KB
Execution proof security: 96 bits
--------------------------------
INFO     verify_program [ 1.35ms | 100.00% ]
Execution verified in 1 ms

@Al-Kindi-0
Copy link
Collaborator

cargo build --profile release --features concurrent,executable,tracing-forest
MIDEN_LOG=info RAYON_NUM_THREADS=8 ./target/release/miden example -r blake3 -n 10
MIDEN_LOG=info RAYON_NUM_THREADS=8 ./target/release/miden example -r blake3 -n 20

This with aux segment size halved

============================================================
Generated a program to compute 10-th iteration of BLAKE3 1-to-1 hash; expected result: [149449452, 625480131, 2142084252, 3624842962, 3172897401, 3622047848, 4284683759, 1689491505]
--------------------------------
INFO     prove_program [ 21.4s | 0.11% / 100.00% ]
INFO     ┝━ execute_program [ 43.9ms | 0.20% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 65536 steps (26% padded) in 43 ms
INFO     ┝━ build_domain [ 1.61ms | 0.01% ] trace_length: 65536 | lde_domain_size: 524288
INFO     ┝━ commit_to_main_trace_segment [ 11.9s | 0.00% / 55.51% ]
INFO     │  ┝━ extend_execution_trace [ 729ms | 3.40% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 11.2s | 52.11% ] commitment_domain_size: 524288
INFO     ┝━ prove_gkr [ 1.86s | 0.00% / 8.68% ]
INFO     │  ┝━ evaluate_logup_gkr_circuit [ 216ms | 1.01% ]
INFO     │  ┝━ prove_intermediate_layers [ 537ms | 2.50% ]
INFO     │  ┝━ build_mls_from_main_trace_segment [ 51.1ms | 0.24% ]
INFO     │  ┕━ prove_input_layer [ 1.06s | 4.94% ]
INFO     ┝━ build_logup_gkr_columns [ 54.6ms | 0.25% ]
INFO     ┝━ commit_to_aux_trace_segment [ 2.27s | 0.00% / 10.60% ]
INFO     │  ┝━ extend_execution_trace [ 78.8ms | 0.37% ] num_cols: 3 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 2.20s | 10.24% ] commitment_domain_size: 524288
INFO     ┝━ evaluate_constraints [ 694ms | 3.24% ] ce_domain_size: 524288
INFO     ┝━ commit_to_constraint_evaluations [ 3.42s | 0.00% / 15.92% ]
INFO     │  ┝━ build_composition_poly_columns [ 26.4ms | 0.12% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 145ms | 0.68% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 3.24s | 15.12% ] log_domain_size: 19
INFO     ┝━ build_deep_composition_poly [ 65.7ms | 0.31% ]
INFO     ┝━ evaluate_deep_composition_poly [ 19.3ms | 0.09% ]
INFO     ┝━ compute_fri_layers [ 742ms | 3.46% ] num_layers: 7
INFO     ┝━ determine_query_positions [ 343ms | 1.60% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 2.88ms | 0.01% ]
--------------------------------
Executed program in 21446 ms
Stack outputs: [149449452, 625480131, 2142084252, 3624842962, 3172897401, 3622047848, 4284683759, 1689491505]
Execution proof size: 118 KB
Execution proof security: 96 bits
--------------------------------
INFO     verify_program [ 34.8ms | 100.00% ]
Execution verified in 34 ms


============================================================
Generated a program to compute 20-th iteration of BLAKE3 1-to-1 hash; expected result: [3129727324, 1676402145, 3126807310, 2534962533, 217698106, 123674944, 1574641409, 3953577339]
--------------------------------
INFO     prove_program [ 43.9s | 0.10% / 100.00% ]
INFO     ┝━ execute_program [ 72.0ms | 0.16% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 131072 steps (26% padded) in 72 ms
INFO     ┝━ build_domain [ 1.58ms | 0.00% ] trace_length: 131072 | lde_domain_size: 1048576
INFO     ┝━ commit_to_main_trace_segment [ 24.0s | 0.00% / 54.54% ]
INFO     │  ┝━ extend_execution_trace [ 1.48s | 3.37% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 22.5s | 51.17% ] commitment_domain_size: 1048576
INFO     ┝━ prove_gkr [ 3.73s | 0.00% / 8.50% ]
INFO     │  ┝━ evaluate_logup_gkr_circuit [ 438ms | 1.00% ]
INFO     │  ┝━ prove_intermediate_layers [ 1.08s | 2.46% ]
INFO     │  ┝━ build_mls_from_main_trace_segment [ 110ms | 0.25% ]
INFO     │  ┕━ prove_input_layer [ 2.10s | 4.79% ]
INFO     ┝━ build_logup_gkr_columns [ 116ms | 0.26% ]
INFO     ┝━ commit_to_aux_trace_segment [ 4.62s | 0.00% / 10.53% ]
INFO     │  ┝━ extend_execution_trace [ 164ms | 0.37% ] num_cols: 3 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 4.46s | 10.15% ] commitment_domain_size: 1048576
INFO     ┝━ evaluate_constraints [ 1.47s | 3.35% ] ce_domain_size: 1048576
INFO     ┝━ commit_to_constraint_evaluations [ 7.99s | 0.00% / 18.20% ]
INFO     │  ┝━ build_composition_poly_columns [ 57.0ms | 0.13% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 306ms | 0.70% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 7.63s | 17.37% ] log_domain_size: 20
INFO     ┝━ build_deep_composition_poly [ 129ms | 0.29% ]
INFO     ┝━ evaluate_deep_composition_poly [ 41.3ms | 0.09% ]
INFO     ┝━ compute_fri_layers [ 1.56s | 3.55% ] num_layers: 7
INFO     ┝━ determine_query_positions [ 178ms | 0.40% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 10.1ms | 0.02% ]
--------------------------------
Executed program in 43928 ms
Stack outputs: [3129727324, 1676402145, 3126807310, 2534962533, 217698106, 123674944, 1574641409, 3953577339]
Execution proof size: 129 KB
Execution proof security: 96 bits
--------------------------------
INFO     verify_program [ 38.3ms | 100.00% ]
Execution verified in 38 ms

Next

============================================================
Generated a program to compute 10-th iteration of BLAKE3 1-to-1 hash; expected result: [149449452, 625480131, 2142084252, 3624842962, 3172897401, 3622047848, 4284683759, 1689491505]
--------------------------------
INFO     prove_program [ 21.0s | 0.24% / 100.00% ]
INFO     ┝━ execute_program [ 37.1ms | 0.18% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 65536 steps (26% padded) in 37 ms
INFO     ┝━ build_domain [ 1.43ms | 0.01% ] trace_length: 65536 | lde_domain_size: 524288
INFO     ┝━ commit_to_main_trace_segment [ 11.9s | 0.00% / 56.57% ]
INFO     │  ┝━ extend_execution_trace [ 775ms | 3.69% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 11.1s | 52.88% ] tree_depth: 19
INFO     ┝━ commit_to_aux_trace_segment [ 3.59s | 0.00% / 17.13% ]
INFO     │  ┝━ extend_execution_trace [ 166ms | 0.79% ] num_cols: 7 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 3.42s | 16.33% ] tree_depth: 19
INFO     ┝━ evaluate_constraints [ 554ms | 2.64% ] ce_domain_size: 524288
INFO     ┝━ commit_to_constraint_evaluations [ 3.63s | 0.00% / 17.32% ]
INFO     │  ┝━ build_composition_poly_columns [ 28.8ms | 0.14% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 145ms | 0.69% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 3.46s | 16.49% ] tree_depth: 19
INFO     ┝━ build_deep_composition_poly [ 39.5ms | 0.19% ]
INFO     ┝━ evaluate_deep_composition_poly [ 21.2ms | 0.10% ]
INFO     ┝━ compute_fri_layers [ 785ms | 3.74% ] num_layers: 7
INFO     ┝━ determine_query_positions [ 391ms | 1.86% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 5.19ms | 0.02% ]
--------------------------------
Executed program in 20966 ms
Stack outputs: [149449452, 625480131, 2142084252, 3624842962, 3172897401, 3622047848, 4284683759, 1689491505]
Execution proof size: 102 KB
Execution proof security: 96 bits
--------------------------------
INFO     verify_program [ 31.2ms | 100.00% ]
Execution verified in 31 ms

============================================================
Generated a program to compute 20-th iteration of BLAKE3 1-to-1 hash; expected result: [3129727324, 1676402145, 3126807310, 2534962533, 217698106, 123674944, 1574641409, 3953577339]
--------------------------------
INFO     prove_program [ 41.9s | 0.23% / 100.00% ]
INFO     ┝━ execute_program [ 75.7ms | 0.18% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 131072 steps (26% padded) in 75 ms
INFO     ┝━ build_domain [ 2.34ms | 0.01% ] trace_length: 131072 | lde_domain_size: 1048576
INFO     ┝━ commit_to_main_trace_segment [ 23.9s | 0.00% / 57.09% ]
INFO     │  ┝━ extend_execution_trace [ 1.59s | 3.79% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 22.4s | 53.30% ] tree_depth: 20
INFO     ┝━ commit_to_aux_trace_segment [ 7.47s | 0.00% / 17.80% ]
INFO     │  ┝━ extend_execution_trace [ 350ms | 0.83% ] num_cols: 7 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 7.12s | 16.97% ] tree_depth: 20
INFO     ┝━ evaluate_constraints [ 1.11s | 2.65% ] ce_domain_size: 1048576
INFO     ┝━ commit_to_constraint_evaluations [ 7.35s | 0.00% / 17.52% ]
INFO     │  ┝━ build_composition_poly_columns [ 58.1ms | 0.14% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 321ms | 0.76% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 6.97s | 16.62% ] tree_depth: 20
INFO     ┝━ build_deep_composition_poly [ 69.3ms | 0.17% ]
INFO     ┝━ evaluate_deep_composition_poly [ 40.0ms | 0.10% ]
INFO     ┝━ compute_fri_layers [ 1.64s | 3.90% ] num_layers: 7
INFO     ┝━ determine_query_positions [ 142ms | 0.34% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 8.32ms | 0.02% ]
--------------------------------
Executed program in 41941 ms
Stack outputs: [3129727324, 1676402145, 3126807310, 2534962533, 217698106, 123674944, 1574641409, 3953577339]
Execution proof size: 111 KB
Execution proof security: 96 bits
--------------------------------
INFO     verify_program [ 31.8ms | 100.00% ]
Execution verified in 31 ms

@plafer plafer force-pushed the plafer-offload-opgrouptable-to-gkr branch 2 times, most recently from c87de25 to b7de6a8 Compare September 23, 2024 15:30
@plafer plafer changed the title Move Op group table to LogUp-GKR Move Op group table and block hash table to LogUp-GKR Sep 23, 2024
@plafer plafer force-pushed the plafer-offload-opgrouptable-to-gkr branch 2 times, most recently from a18f2b4 to b4a52a3 Compare September 24, 2024 18:42
@plafer plafer force-pushed the plafer-offload-opgrouptable-to-gkr branch from b4a52a3 to e7093e1 Compare September 25, 2024 19:40
@plafer
Copy link
Contributor Author

plafer commented Sep 27, 2024

Latest benchmarks (parallel version, 64 fractions containing all tables except for the chiplets bus and the stack overflow table)

TL;DR: the RPO-based benchmark is exactly at par with next!

  • this branch (all winterfell PRs merged)
    • blake3: 4088 ms
    • RPO: 13657 ms
  • next
    • blake3: 2251ms (+ ~300ms to account for auxiliary constraints we don't evaluate)
    • RPO: 13660 ms

This branch (all winterfell PRs merged)

$ MIDEN_LOG=info RAYON_NUM_THREADS=8 ./target/optimized/miden example blake3  -n 50
--------------------------------
INFO     prove_program [ 4.09s | 0.95% / 100.00% ]
INFO     ┝━ execute_program [ 126ms | 3.08% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 262144 steps (8% padded) in 126 ms
INFO     ┝━ build_domain [ 2.49ms | 0.06% ] trace_length: 262144 | lde_domain_size: 2097152
INFO     ┝━ commit_to_main_trace_segment [ 853ms | 0.00% / 20.86% ]
INFO     │  ┝━ extend_execution_trace [ 585ms | 14.30% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 268ms | 6.56% ] commitment_domain_size: 2097152
INFO     ┝━ prove_gkr [ 1.49s | 0.00% / 36.50% ]
INFO     │  ┝━ evaluate_logup_gkr_circuit [ 301ms | 7.36% ]
INFO     │  ┝━ prove_intermediate_layers [ 287ms | 7.01% ]
INFO     │  ┝━ build_mle_from_main_trace_segment [ 75.9ms | 1.86% ]
INFO     │  ┕━ prove_input_layer [ 829ms | 20.28% ]
INFO     ┝━ build_logup_gkr_columns [ 141ms | 3.45% ]
INFO     ┝━ commit_to_aux_trace_segment [ 109ms | 0.00% / 2.66% ]
INFO     │  ┝━ extend_execution_trace [ 50.4ms | 1.23% ] num_cols: 4 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 58.2ms | 1.42% ] commitment_domain_size: 2097152
INFO     ┝━ evaluate_constraints [ 905ms | 22.14% ] ce_domain_size: 2097152
INFO     ┝━ commit_to_constraint_evaluations [ 210ms | 0.00% / 5.15% ]
INFO     │  ┝━ build_composition_poly_columns [ 32.3ms | 0.79% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 101ms | 2.48% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 76.7ms | 1.88% ] log_domain_size: 21
INFO     ┝━ build_deep_composition_poly [ 167ms | 4.08% ]
INFO     ┝━ evaluate_deep_composition_poly [ 17.5ms | 0.43% ]
INFO     ┝━ compute_fri_layers [ 20.8ms | 0.51% ] num_layers: 4
INFO     ┝━ determine_query_positions [ 1.43ms | 0.04% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 4.43ms | 0.11% ]
--------------------------------
Executed program in 4088 ms

$ MIDEN_LOG=info RAYON_NUM_THREADS=8 ./target/optimized/miden example -r blake3  -n 20
--------------------------------
INFO     prove_program [ 13.7s | 0.18% / 100.00% ]
INFO     ┝━ execute_program [ 64.5ms | 0.47% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 131072 steps (26% padded) in 64 ms
INFO     ┝━ build_domain [ 1.47ms | 0.01% ] trace_length: 131072 | lde_domain_size: 1048576
INFO     ┝━ commit_to_main_trace_segment [ 7.79s | 0.00% / 57.03% ]
INFO     │  ┝━ extend_execution_trace [ 287ms | 2.10% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 7.50s | 54.93% ] commitment_domain_size: 1048576
INFO     ┝━ prove_gkr [ 785ms | 0.00% / 5.74% ]
INFO     │  ┝━ evaluate_logup_gkr_circuit [ 148ms | 1.09% ]
INFO     │  ┝━ prove_intermediate_layers [ 160ms | 1.17% ]
INFO     │  ┝━ build_mle_from_main_trace_segment [ 33.5ms | 0.25% ]
INFO     │  ┕━ prove_input_layer [ 443ms | 3.24% ]
INFO     ┝━ build_logup_gkr_columns [ 69.6ms | 0.51% ]
INFO     ┝━ commit_to_aux_trace_segment [ 1.52s | 0.00% / 11.15% ]
INFO     │  ┝━ extend_execution_trace [ 24.0ms | 0.18% ] num_cols: 4 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 1.50s | 10.97% ] commitment_domain_size: 1048576
INFO     ┝━ evaluate_constraints [ 435ms | 3.18% ] ce_domain_size: 1048576
INFO     ┝━ commit_to_constraint_evaluations [ 2.30s | 0.00% / 16.82% ]
INFO     │  ┝━ build_composition_poly_columns [ 15.5ms | 0.11% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 45.6ms | 0.33% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 2.24s | 16.37% ] log_domain_size: 20
INFO     ┝━ build_deep_composition_poly [ 86.2ms | 0.63% ]
INFO     ┝━ evaluate_deep_composition_poly [ 7.78ms | 0.06% ]
INFO     ┝━ compute_fri_layers [ 539ms | 3.95% ] num_layers: 7
INFO     ┝━ determine_query_positions [ 31.9ms | 0.23% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 3.40ms | 0.02% ]
--------------------------------
Executed program in 13657 ms

next

(only show the RPO-based benchmark; blake3 is unchanged and available in previous comments)

$ MIDEN_LOG=info RAYON_NUM_THREADS=8 ./target/optimized/miden example -r blake3  -n 20
============================================================
Generated a program to compute 20-th iteration of BLAKE3 1-to-1 hash; expected result: [3129727324, 1676402145, 3126807310, 2534962533, 217698106, 123674944, 1574641409, 3953577339]
--------------------------------
INFO     prove_program [ 13.7s | 0.32% / 100.00% ]
INFO     ┝━ execute_program [ 62.5ms | 0.46% ]
INFO     ┝━ ｉ [info]: Generated execution trace of 70 columns and 131072 steps (26% padded) in 62 ms
INFO     ┝━ build_domain [ 1.48ms | 0.01% ] trace_length: 131072 | lde_domain_size: 1048576
INFO     ┝━ commit_to_main_trace_segment [ 7.88s | 0.00% / 57.65% ]
INFO     │  ┝━ extend_execution_trace [ 281ms | 2.06% ] num_cols: 70 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 7.59s | 55.59% ] tree_depth: 20
INFO     ┝━ commit_to_aux_trace_segment [ 2.33s | 0.00% / 17.05% ]
INFO     │  ┝━ extend_execution_trace [ 64.2ms | 0.47% ] num_cols: 7 | blowup: 8
INFO     │  ┕━ compute_execution_trace_commitment [ 2.27s | 16.58% ] tree_depth: 20
INFO     ┝━ evaluate_constraints [ 336ms | 2.46% ] ce_domain_size: 1048576
INFO     ┝━ commit_to_constraint_evaluations [ 2.33s | 0.00% / 17.08% ]
INFO     │  ┝━ build_composition_poly_columns [ 16.3ms | 0.12% ] num_columns: 8
INFO     │  ┝━ evaluate_composition_poly_columns [ 55.7ms | 0.41% ]
INFO     │  ┕━ compute_constraint_evaluation_commitment [ 2.26s | 16.55% ] tree_depth: 20
INFO     ┝━ build_deep_composition_poly [ 51.6ms | 0.38% ]
INFO     ┝━ evaluate_deep_composition_poly [ 8.22ms | 0.06% ]
INFO     ┝━ compute_fri_layers [ 533ms | 3.90% ] num_layers: 7
INFO     ┝━ determine_query_positions [ 83.4ms | 0.61% ] grinding_factor: 16 | num_positions: 27
INFO     ┕━ build_proof_object [ 3.92ms | 0.03% ]
--------------------------------
Executed program in 13660 ms

@plafer plafer force-pushed the plafer-offload-opgrouptable-to-gkr branch from 8de48f9 to 77bb46d Compare September 30, 2024 14:58
@plafer plafer changed the title Move Op group table and block hash table to LogUp-GKR Move all buses and tables to LogUp-GKR Sep 30, 2024
@plafer plafer force-pushed the plafer-offload-opgrouptable-to-gkr branch from 77bb46d to eafc592 Compare October 2, 2024 20:18
@plafer plafer force-pushed the plafer-offload-opgrouptable-to-gkr branch from eafc592 to e21d358 Compare October 4, 2024 15:44
@bobbinth
Copy link
Contributor

bobbinth commented Nov 4, 2024

@plafer - should we merge this PR into the logup-gkr branch?

@plafer plafer marked this pull request as ready for review November 4, 2024 19:22
@plafer
Copy link
Contributor Author

plafer commented Nov 4, 2024

@bobbinth we might as well. The bus is not done yet, but in any case some of these PRs are already stale as they don't take into account the multiset check fixes that occurred in next over a couple of PRs. So the logup-gkr branch needs to be updated in any case.

TL;DR: let's merge this, and when/if we want to pick logup-gkr back up, we'll need some work to get it back up to date.

@plafer plafer merged commit 8aabc10 into logup-gkr Nov 4, 2024
9 checks passed
@plafer plafer deleted the plafer-offload-opgrouptable-to-gkr branch November 4, 2024 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
no changelog This PR does not require an entry in the `CHANGELOG.md` file
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@plafer @Al-Kindi-0 @bobbinth