refactor: share verifier rounds #4849

lucasxia01 · 2024-02-28T22:00:55Z

Addresses AztecProtocol/barretenberg#882, but doesn't fully close it since recursive verifiers aren't shared.

Creates OinkVerifier which executes all of the pre-sumcheck logic in the folding and ultra honk verifiers. This does not handle the recursive verifiers.

I also create OinkOutput which serves as the output state that is passed from OinkVerifier back to UltraVerifier or ProtogalaxyVerifier.

Please read contributing guidelines and remove this line.

AztecBot · 2024-02-28T22:34:44Z

Benchmark results

Metrics with a significant change:

note_successful_decrypting_time_in_ms (8): 240 (+26%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit 82f8cf5e and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	64 txs
l1_rollup_calldata_size_in_bytes	5,668	18,820	36,356
l1_rollup_calldata_gas	66,340	238,708	468,980
l1_rollup_execution_gas	647,438	929,066	1,305,160
l2_block_processing_time_in_ms	1,448 (+8%)	4,917 (+7%)	9,417 (+3%)
note_successful_decrypting_time_in_ms	⚠️ 240 (+26%)	566 (+15%)	989 (+1%)
note_trial_decrypting_time_in_ms	49.2 (-48%)	84.5 (-19%)	124 (+12%)
l2_block_building_time_in_ms	18,432 (+1%)	69,751 (+2%)	137,264 (+2%)
l2_block_rollup_simulation_time_in_ms	8,323 (+3%)	29,660 (+3%)	57,506 (+2%)
l2_block_public_tx_process_time_in_ms	10,081	40,025 (+2%)	79,652 (+1%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	5 blocks	10 blocks
node_history_sync_time_in_ms	13,881 (-4%)	26,512 (-5%)
note_history_successful_decrypting_time_in_ms	1,238 (+1%)	2,442 (-3%)
note_history_trial_decrypting_time_in_ms	86.0 (+24%)	156 (-1%)
node_database_size_in_bytes	19,103,824	35,672,144
pxe_database_size_in_bytes	29,859	59,414

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit	circuit_simulation_time_in_ms	circuit_input_size_in_bytes	circuit_output_size_in_bytes
private-kernel-init	283 (+1%)	44,366	28,244
private-kernel-ordering	215	52,868	14,326
base-parity	1,831 (+1%)	128	311
base-rollup	726 (+1%)	165,787	925
root-parity	1,612 (+2%)	1,244	311
root-rollup	63.2	4,487	789
private-kernel-inner	645	73,771	28,244
public-kernel-app-logic	445	35,260	28,215
public-kernel-tail	174 (+1%)	40,926	28,215
merge-rollup	8.34 (+1%)	2,696	925

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	512 leaves	1024 leaves	2048 leaves	4096 leaves	32 leaves
batch_insert_into_append_only_tree_16_depth_ms	10.1 (+1%)	16.1 (+1%)	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.8	31.6	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.585 (+1%)	0.496 (+1%)	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	47.5 (+1%)	72.5 (+1%)	232 (+2%)	444	893 (+2%)	1,730 (+3%)	N/A
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	96.0	159	543	1,055	2,079	4,127	N/A
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.487 (+1%)	0.447 (+1%)	0.421 (+1%)	0.416	0.422 (+2%)	0.414 (+3%)	N/A
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	54.4 (+1%)	107	337 (-3%)	659	1,330 (+2%)	2,625 (+3%)	N/A
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	104	207	691	1,363	2,707	5,395	N/A
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.482 (+1%)	0.481	0.458 (-4%)	0.454	0.457 (+1%)	0.455 (+3%)	N/A
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	61.6 (+1%)
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	109
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.538 (+1%)

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes
tx_size_in_bytes	22,012

Transaction processing duration by data writes.

Metric	0 new note hashes	1 new note hashes
tx_pxe_processing_time_ms	3,335 (+1%)	1,762 (+1%)

Metric	0 public data writes	1 public data writes
tx_sequencer_processing_time_ms	12.0 (+10%)	1,246

…sharing

lucasxia01 · 2024-03-14T17:05:08Z

barretenberg/cpp/src/barretenberg/protogalaxy/protogalaxy_verifier.cpp

-    inst->relation_parameters =
-        RelationParameters<FF>{ eta, beta, gamma, public_input_delta, lookup_grand_product_delta };
+    auto& key = inst->verification_key;
+    OinkVerifier<Flavor> oink_verifier{ key, transcript, domain_separator + '_' };


I don't really like this domain_separator hack, but it works

I had to add '_' to the domain separator so that normally we wouldn't add '_' to the manifest strings when domain_separator was empty (in the ultra_verifier case)

lucasxia01 · 2024-03-14T19:02:12Z

barretenberg/cpp/src/barretenberg/ultra_honk/oink_verifier.cpp

+    const auto pub_inputs_offset =
+        transcript->template receive_from_prover<uint32_t>(domain_separator + "pub_inputs_offset");
+
+    ASSERT(circuit_size == key->circuit_size);


this could also be turn into if (circuit_size != key->circuit_size) return false; instead of assert.

lucasxia01 · 2024-03-14T19:02:52Z

barretenberg/cpp/src/barretenberg/ultra_honk/ultra_verifier.cpp

@@ -31,7 +32,6 @@ UltraVerifier_<Flavor>::UltraVerifier_(UltraVerifier_&& other)
 template <typename Flavor> UltraVerifier_<Flavor>& UltraVerifier_<Flavor>::operator=(UltraVerifier_&& other)
 {
    key = other.key;
-    commitments.clear();


commitments wasn't being used...

lucasxia01 · 2024-03-14T19:03:49Z

barretenberg/cpp/src/barretenberg/ultra_honk/oink_verifier.hpp

+
+namespace bb {
+
+template <IsUltraFlavor Flavor> struct OinkOutput {


output state from Oink verifier

lucasxia01 · 2024-03-14T19:05:33Z

barretenberg/cpp/src/barretenberg/ultra_honk/oink_verifier.cpp

@@ -0,0 +1,140 @@
+#include "barretenberg/ultra_honk/oink_verifier.hpp"


this file is largely copied, just split up into different functions

…sharing

…/aztec-packages into lx/verifiers-round-sharing

lucasxia01 · 2024-03-18T19:19:04Z

barretenberg/cpp/src/barretenberg/ultra_honk/oink_verifier.cpp

+
+    ASSERT(circuit_size == key->circuit_size);
+    ASSERT(public_input_size == key->num_public_inputs);
+    ASSERT(pub_inputs_offset == key->pub_inputs_offset);


probably not needed as well. This should just be overwritten by transcript pub_inputs_offset?

codygunton

LGTM

codygunton · 2024-03-18T23:04:09Z

barretenberg/cpp/src/barretenberg/ultra_honk/ultra_verifier.cpp

-        return false;
+    // Copy the witness_commitments over to the VerifierCommitments
+    for (auto [wit_comm_1, wit_comm_2] : zip_view(commitments.get_witness(), witness_commitments.get_all())) {
+        wit_comm_1 = wit_comm_2;


Prob no reason not to std::move these but it's also not much data.

initial scoping for sharing verifier

0fed909

lucasxia01 marked this pull request as draft February 28, 2024 22:48

lucasxia01 self-assigned this Feb 28, 2024

lucasxia01 added 7 commits February 29, 2024 19:10

gcc fix

61a8fd3

Merge remote-tracking branch 'origin/master' into lx/verifiers-round-…

54dae20

…sharing

small updates

2282ab2

replaced protogalaxy with oink verifier

3a46aa6

renaming to OINK/moving files

d791c38

fix manifest

0b22942

Merge remote-tracking branch 'origin/master' into lx/verifiers-round-…

bdce18c

…sharing

lucasxia01 marked this pull request as ready for review March 14, 2024 17:04

lucasxia01 commented Mar 14, 2024

View reviewed changes

lucasxia01 added 2 commits March 14, 2024 19:00

split into rounds, moved into cpp, removed verified bool

8b11512

Merge branch 'master' into lx/verifiers-round-sharing

72d9fa9

lucasxia01 commented Mar 14, 2024

View reviewed changes

lucasxia01 added 3 commits March 18, 2024 15:57

Merge remote-tracking branch 'origin/master' into lx/verifiers-round-…

8813b32

…sharing

undo PI assertion to pass tests

c36b17f

Merge branch 'lx/verifiers-round-sharing' of github.com:AztecProtocol…

3bbf7a3

…/aztec-packages into lx/verifiers-round-sharing

lucasxia01 commented Mar 18, 2024

View reviewed changes

codygunton approved these changes Mar 18, 2024

View reviewed changes

codygunton enabled auto-merge (squash) March 18, 2024 23:04

codygunton merged commit 1139308 into master Mar 18, 2024
96 checks passed

codygunton deleted the lx/verifiers-round-sharing branch March 18, 2024 23:14

AztecBot mentioned this pull request Mar 18, 2024

chore(master): Release 0.30.0 #5296

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: share verifier rounds #4849

refactor: share verifier rounds #4849

lucasxia01 commented Feb 28, 2024 •

edited

Loading

AztecBot commented Feb 28, 2024 •

edited

Loading

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

lucasxia01 Mar 14, 2024

lucasxia01 Mar 14, 2024 •

edited

Loading

lucasxia01 Mar 14, 2024 •

edited

Loading

lucasxia01 Mar 14, 2024

lucasxia01 Mar 14, 2024

lucasxia01 Mar 14, 2024

lucasxia01 Mar 18, 2024

codygunton left a comment

codygunton Mar 18, 2024


		namespace bb {

		template <IsUltraFlavor Flavor> struct OinkOutput {

		@@ -0,0 +1,140 @@
		#include "barretenberg/ultra_honk/oink_verifier.hpp"

refactor: share verifier rounds #4849

refactor: share verifier rounds #4849

Conversation

lucasxia01 commented Feb 28, 2024 • edited Loading

AztecBot commented Feb 28, 2024 • edited Loading

Benchmark results

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

lucasxia01 Mar 14, 2024

Choose a reason for hiding this comment

lucasxia01 Mar 14, 2024 • edited Loading

Choose a reason for hiding this comment

lucasxia01 Mar 14, 2024 • edited Loading

Choose a reason for hiding this comment

lucasxia01 Mar 14, 2024

Choose a reason for hiding this comment

lucasxia01 Mar 14, 2024

Choose a reason for hiding this comment

lucasxia01 Mar 14, 2024

Choose a reason for hiding this comment

lucasxia01 Mar 18, 2024

Choose a reason for hiding this comment

codygunton left a comment

Choose a reason for hiding this comment

codygunton Mar 18, 2024

Choose a reason for hiding this comment

lucasxia01 commented Feb 28, 2024 •

edited

Loading

AztecBot commented Feb 28, 2024 •

edited

Loading

lucasxia01 Mar 14, 2024 •

edited

Loading

lucasxia01 Mar 14, 2024 •

edited

Loading