Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ECCVM correctly handles points at infinity and group operation edge cases #6388

Merged
merged 30 commits into from
May 27, 2024

Conversation

zac-williamson
Copy link
Contributor

@zac-williamson zac-williamson commented May 14, 2024

This PR adds handling for edge cases in the ECCVM to allow handling the point at infinity. In the ECCVM the point at infinity is encoded as (0,0), which does not lie on either of BN254 or Grumpkin.

From benchmarks below, additional ECCVM prover overhead: 1764 $\leadsto$ 1895 is 7.4%.

Before

Running ./client_ivc_bench
Run on (16 X 3591.91 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 36608 KiB (x1)
Load Average: 0.03, 0.38, 0.24
--------------------------------------------------------------------------------
Benchmark                      Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------
ClientIVCBench/Full/6      16069 ms        11155 ms            1 Arithmetic::accumulate=783.089k Arithmetic::accumulate(t)=1.49562G Auxiliary::accumulate=570.914k Auxiliary::accumulate(t)=3.77581G DatabusRead::accumulate=544 DatabusRead::accumulate(t)=1.3519M Decider::construct_proof=1 Decider::construct_proof(t)=494.789M DeltaRange::accumulate=446.369k DeltaRange::accumulate(t)=970.499M ECCVMProver(CircuitBuilder&)=1 ECCVMProver(CircuitBuilder&)(t)=176.304M ECCVMProver::construct_proof=1 ECCVMProver::construct_proof(t)=1.76396G Elliptic::accumulate=277.663k Elliptic::accumulate(t)=609.669M Goblin::merge=11 Goblin::merge(t)=152.308M Lookup::accumulate=2.62144M Lookup::accumulate(t)=11.7011G Permutation::accumulate=1.80147M Permutation::accumulate(t)=6.74356G PoseidonExt::accumulate=46.504k PoseidonExt::accumulate(t)=108.988M PoseidonInt::accumulate=181.69k PoseidonInt::accumulate(t)=320.388M ProtoGalaxyProver_::accumulator_update_round=10 ProtoGalaxyProver_::accumulator_update_round(t)=284.012M ProtoGalaxyProver_::combiner_quotient_round=10 ProtoGalaxyProver_::combiner_quotient_round(t)=2.82904G ProtoGalaxyProver_::perturbator_round=10 ProtoGalaxyProver_::perturbator_round(t)=775.352M ProtoGalaxyProver_::preparation_round=10 ProtoGalaxyProver_::preparation_round(t)=3.70662G ProtogalaxyProver::fold_instances=10 ProtogalaxyProver::fold_instances(t)=7.59505G ProverInstance(Circuit&)=11 ProverInstance(Circuit&)(t)=1.65694G TranslatorCircuitBuilder::constructor=1 TranslatorCircuitBuilder::constructor(t)=62.2611M TranslatorProver=1 TranslatorProver(t)=59.188M TranslatorProver::construct_proof=1 TranslatorProver::construct_proof(t)=842.306M batch_mul_with_endomorphism=30 batch_mul_with_endomorphism(t)=560.346M commit=459 commit(t)=4.06289G compute_combiner=10 compute_combiner(t)=2.82696G compute_perturbator=9 compute_perturbator(t)=774.979M compute_univariate=48 compute_univariate(t)=1.04316G construct_circuits=15 construct_circuits(t)=3.2485G
Benchmarking lock deleted.
client_ivc_bench.json                                                                                                                                       100% 4576   114.5KB/s   00:00    
function                                  ms     % sum
construct_circuits(t)                   3248    20.39%
ProverInstance(Circuit&)(t)             1657    10.40%
ProtogalaxyProver::fold_instances(t)    7595    47.68%
Decider::construct_proof(t)              495     3.11%
ECCVMProver(CircuitBuilder&)(t)          176     1.11%
ECCVMProver::construct_proof(t)         1764    11.07%
TranslatorProver::construct_proof(t)     842     5.29%
Goblin::merge(t)                         152     0.96%

Total time accounted for: 15930ms/16069ms = 99.14%

Major contributors:
function                                  ms    % sum
commit(t)                               4063   25.50%
compute_combiner(t)                     2827   17.75%
compute_perturbator(t)                   775    4.86%
compute_univariate(t)                   1043    6.55%

Breakdown of ProtogalaxyProver::fold_instances:
ProtoGalaxyProver_::preparation_round(t)           3707    48.80%
ProtoGalaxyProver_::perturbator_round(t)            775    10.21%
ProtoGalaxyProver_::combiner_quotient_round(t)     2829    37.25%
ProtoGalaxyProver_::accumulator_update_round(t)     284     3.74%

Relation contributions (times to be interpreted relatively):
function                        ms     % sum
Arithmetic::accumulate(t)     1496     5.81%
Permutation::accumulate(t)    6744    26.21%
Lookup::accumulate(t)        11701    45.48%
DeltaRange::accumulate(t)      970     3.77%
Elliptic::accumulate(t)        610     2.37%
Auxiliary::accumulate(t)      3776    14.68%
EccOp::accumulate(t)             0     0.00%
DatabusRead::accumulate(t)       1     0.01%
PoseidonExt::accumulate(t)     109     0.42%
PoseidonInt::accumulate(t)     320     1.25%

After

Running ./client_ivc_bench
Run on (16 X 3589.32 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1024 KiB (x8)
  L3 Unified 36608 KiB (x1)
Load Average: 0.08, 0.02, 0.01
--------------------------------------------------------------------------------
Benchmark                      Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------
ClientIVCBench/Full/6      16189 ms        11363 ms            1 Arithmetic::accumulate=783.089k Arithmetic::accumulate(t)=1.50572G Auxiliary::accumulate=570.914k Auxiliary::accumulate(t)=3.7492G DatabusRead::accumulate=544 DatabusRead::accumulate(t)=1.33712M Decider::construct_proof=1 Decider::construct_proof(t)=490.544M DeltaRange::accumulate=446.369k DeltaRange::accumulate(t)=963.901M ECCVMProver(CircuitBuilder&)=1 ECCVMProver(CircuitBuilder&)(t)=171.92M ECCVMProver::construct_proof=1 ECCVMProver::construct_proof(t)=1.89489G Elliptic::accumulate=277.663k Elliptic::accumulate(t)=632.104M Goblin::merge=11 Goblin::merge(t)=153.836M Lookup::accumulate=2.62144M Lookup::accumulate(t)=11.6883G Permutation::accumulate=1.80147M Permutation::accumulate(t)=6.77935G PoseidonExt::accumulate=46.504k PoseidonExt::accumulate(t)=105.641M PoseidonInt::accumulate=181.69k PoseidonInt::accumulate(t)=314.715M ProtoGalaxyProver_::accumulator_update_round=10 ProtoGalaxyProver_::accumulator_update_round(t)=284.645M ProtoGalaxyProver_::combiner_quotient_round=10 ProtoGalaxyProver_::combiner_quotient_round(t)=2.84548G ProtoGalaxyProver_::perturbator_round=10 ProtoGalaxyProver_::perturbator_round(t)=771.584M ProtoGalaxyProver_::preparation_round=10 ProtoGalaxyProver_::preparation_round(t)=3.67568G ProtogalaxyProver::fold_instances=10 ProtogalaxyProver::fold_instances(t)=7.57742G ProverInstance(Circuit&)=11 ProverInstance(Circuit&)(t)=1.68623G TranslatorCircuitBuilder::constructor=1 TranslatorCircuitBuilder::constructor(t)=62.0279M TranslatorProver=1 TranslatorProver(t)=55.6683M TranslatorProver::construct_proof=1 TranslatorProver::construct_proof(t)=824.76M batch_mul_with_endomorphism=30 batch_mul_with_endomorphism(t)=563.413M commit=470 commit(t)=4.05576G compute_combiner=10 compute_combiner(t)=2.84338G compute_perturbator=9 compute_perturbator(t)=771.214M compute_univariate=48 compute_univariate(t)=1.13311G construct_circuits=15 construct_circuits(t)=3.25952G
Benchmarking lock deleted.
client_ivc_bench.json                                                                                                                                       100% 4579   149.4KB/s   00:00    
function                                  ms     % sum
construct_circuits(t)                   3260    20.30%
ProverInstance(Circuit&)(t)             1686    10.50%
ProtogalaxyProver::fold_instances(t)    7577    47.18%
Decider::construct_proof(t)              491     3.05%
ECCVMProver(CircuitBuilder&)(t)          172     1.07%
ECCVMProver::construct_proof(t)         1895    11.80%
TranslatorProver::construct_proof(t)     825     5.14%
Goblin::merge(t)                         154     0.96%

Total time accounted for: 16059ms/16189ms = 99.20%

Major contributors:
function                                  ms    % sum
commit(t)                               4056   25.26%
compute_combiner(t)                     2843   17.71%
compute_perturbator(t)                   771    4.80%
compute_univariate(t)                   1133    7.06%

Breakdown of ProtogalaxyProver::fold_instances:
ProtoGalaxyProver_::preparation_round(t)           3676    48.51%
ProtoGalaxyProver_::perturbator_round(t)            772    10.18%
ProtoGalaxyProver_::combiner_quotient_round(t)     2845    37.55%
ProtoGalaxyProver_::accumulator_update_round(t)     285     3.76%

Relation contributions (times to be interpreted relatively):
function                        ms     % sum
Arithmetic::accumulate(t)     1506     5.85%
Permutation::accumulate(t)    6779    26.34%
Lookup::accumulate(t)        11688    45.41%
DeltaRange::accumulate(t)      964     3.74%
Elliptic::accumulate(t)        632     2.46%
Auxiliary::accumulate(t)      3749    14.57%
EccOp::accumulate(t)             0     0.00%
DatabusRead::accumulate(t)       1     0.01%
PoseidonExt::accumulate(t)     106     0.41%
PoseidonInt::accumulate(t)     315     1.22%

…inity, and have +/- methods that correctly handle points at infinity
… doubling edge cases

upgraded eccvm transcript to correctly handle an MSM composed entirely of points at infinity and/or zero scalars
moved eccvm boolean relations into low degree relation class, added missing boolean checks

removed unused transcript columns
@codygunton codygunton changed the title Updated the ECCVM to correctly handle points at infinity and group operation edge-cases fix: ECCVM correctly handles points at infinity and group operation edge-cases May 15, 2024
@codygunton codygunton changed the base branch from zw/biggroup-infinity to master May 15, 2024 13:14
@codygunton codygunton changed the base branch from master to zw/biggroup-infinity May 15, 2024 13:15
…s at infinity, and have +/- methods that correctly handle points at infinity"

This reverts commit 9f6b4ef.
@codygunton codygunton changed the base branch from zw/biggroup-infinity to master May 15, 2024 22:07
@fcarreiro fcarreiro removed their request for review May 16, 2024 12:54
@codygunton codygunton marked this pull request as draft May 16, 2024 13:02
`Relation ECCVMTranscriptRelation, subrelation index 24 failed at row 138`
@AztecBot
Copy link
Collaborator

AztecBot commented May 16, 2024

Benchmark results

Metrics with a significant change:

  • app_circuit_proving_time_in_ms (Token:transfer): 45,965 (-17%)
  • protocol_circuit_witness_generation_time_in_ms (private-kernel-inner): 4,796 (+17%)
  • protocol_circuit_proving_time_in_ms (private-kernel-tail): 36,040 (-17%)
Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric 1 threads 4 threads 16 threads 32 threads 64 threads
proof_construction_time_sha256 5,729 (+2%) 1,559 (+1%) 714 (+1%) 773 (+3%) 777

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 8 txs 32 txs 64 txs
l1_rollup_calldata_size_in_bytes 1,412 1,412 1,412
l1_rollup_calldata_gas 9,452 9,464 9,476
l1_rollup_execution_gas 616,093 616,105 616,117
l2_block_processing_time_in_ms 1,301 4,838 9,633
l2_block_building_time_in_ms 46,534 (-1%) 184,056 (-1%) 367,361 (-1%)
l2_block_rollup_simulation_time_in_ms 46,361 (-1%) 183,420 (-1%) 366,105 (-1%)
l2_block_public_tx_process_time_in_ms 24,829 103,445 211,064 (-1%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric 3 blocks 5 blocks
node_history_sync_time_in_ms 9,613 (+2%) 14,587 (+1%)
node_database_size_in_bytes 14,458,960 21,373,008
pxe_database_size_in_bytes 18,071 29,868

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit simulation_time_in_ms witness_generation_time_in_ms proving_time_in_ms input_size_in_bytes output_size_in_bytes proof_size_in_bytes num_public_inputs size_in_gates
private-kernel-init 163 (-1%) 3,723 (+4%) 22,296 (-12%) 20,630 64,614 89,536 2,731 1,048,576
private-kernel-inner 621 (-1%) ⚠️ 4,796 (+17%) 46,411 (-11%) 92,318 64,614 89,536 2,731 2,097,152
private-kernel-tail 542 (-1%) 2,925 (+5%) ⚠️ 36,040 (-17%) 90,237 77,498 10,656 266 2,097,152
base-parity 6.47 (+2%) 1,293 (+7%) 2,914 (+2%) 128 64.0 2,208 2.00 131,072
root-parity 49.5 (+1%) 48.5 (-24%) 44,447 (-12%) 27,084 64.0 2,720 18.0 2,097,152
base-rollup 800 (+1%) 2,308 (-5%) 74,982 (-13%) 119,610 756 3,648 47.0 4,194,304
root-rollup 114 (+1%) 65.1 (-15%) 19,842 (-13%) 25,297 620 3,456 41.0 1,048,576
public-kernel-app-logic 520 2,835 (-1%) 47,962 (-8%) 104,941 86,302 114,784 3,520 2,097,152
public-kernel-tail 1,181 (-1%) 23,526 (-4%) 154,210 (-15%) 395,386 7,522 10,656 266 8,388,608
private-kernel-reset-small 590 (-1%) 2,229 (+7%) 45,872 (-7%) 120,733 64,614 89,536 2,731 2,097,152
merge-rollup 28.6 N/A N/A 16,534 756 N/A N/A N/A
public-kernel-setup 625 N/A N/A 104,941 86,302 N/A N/A N/A
public-kernel-teardown 529 N/A N/A 104,941 86,302 N/A N/A N/A
private-kernel-tail-to-public N/A 8,482 (+3%) 84,828 (-10%) N/A N/A 114,784 3,520 4,194,304

Stats on running time collected for app circuits

Function input_size_in_bytes output_size_in_bytes witness_generation_time_in_ms proof_size_in_bytes proving_time_in_ms size_in_gates num_public_inputs
ContractClassRegisterer:register 1,344 9,944 464 N/A N/A N/A N/A
ContractInstanceDeployer:deploy 1,408 9,944 42.5 (+3%) N/A N/A N/A N/A
MultiCallEntrypoint:entrypoint 1,920 9,944 1,453 (+1%) N/A N/A N/A N/A
SchnorrAccount:constructor 1,312 9,944 1,026 (+2%) N/A N/A N/A N/A
SchnorrAccount:entrypoint 2,304 9,944 2,124 16,768 46,556 (-6%) 2,097,152 457
Token:privately_mint_private_note 1,280 9,944 1,166 (+4%) N/A N/A N/A N/A
Token:transfer 1,376 9,944 4,123 (+1%) 16,768 ⚠️ 45,965 (-17%) 2,097,152 457
Benchmarking:create_note 1,312 9,944 982 N/A N/A N/A N/A
FPC:fee_entrypoint_public 1,344 9,944 228 (-1%) N/A N/A N/A N/A
SchnorrAccount:spend_private_authwit 1,280 9,944 77.5 (-1%) N/A N/A N/A N/A
Token:unshield 1,376 9,944 3,261 (-2%) N/A N/A N/A N/A
FPC:fee_entrypoint_private 1,376 9,944 4,064 (-3%) N/A N/A N/A N/A

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric 1 leaves 16 leaves 64 leaves 128 leaves 512 leaves 1024 leaves 2048 leaves 4096 leaves 32 leaves
batch_insert_into_append_only_tree_16_depth_ms 10.5 (+1%) 17.0 (+1%) N/A N/A N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_count 16.7 31.8 N/A N/A N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_ms 0.609 (+1%) 0.523 (+1%) N/A N/A N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_32_depth_ms N/A N/A 49.2 (+2%) 76.8 247 (+1%) 480 (+1%) 933 (+1%) 1,846 N/A
batch_insert_into_append_only_tree_32_depth_hash_count N/A N/A 95.9 159 543 1,055 2,079 4,127 N/A
batch_insert_into_append_only_tree_32_depth_hash_ms N/A N/A 0.503 (+2%) 0.473 0.447 (+1%) 0.447 (+1%) 0.442 (+1%) 0.440 (+1%) N/A
batch_insert_into_indexed_tree_20_depth_ms N/A N/A 58.8 (+1%) 113 (+1%) 357 706 (+1%) 1,390 2,766 (+1%) N/A
batch_insert_into_indexed_tree_20_depth_hash_count N/A N/A 106 208 692 1,363 2,707 5,395 N/A
batch_insert_into_indexed_tree_20_depth_hash_ms N/A N/A 0.509 (+1%) 0.506 (+1%) 0.484 0.484 (+1%) 0.481 0.480 N/A
batch_insert_into_indexed_tree_40_depth_ms N/A N/A N/A N/A N/A N/A N/A N/A 62.8 (+1%)
batch_insert_into_indexed_tree_40_depth_hash_count N/A N/A N/A N/A N/A N/A N/A N/A 107
batch_insert_into_indexed_tree_40_depth_hash_ms N/A N/A N/A N/A N/A N/A N/A N/A 0.556 (+1%)

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric 0 registered classes 1 registered classes
tx_size_in_bytes 83,794 665,117

Transaction size based on fee payment method

| Metric | |
| - | |

@codygunton codygunton self-assigned this May 17, 2024
ludamad pushed a commit that referenced this pull request May 21, 2024
Tests that had been disabled were replaced by #6429. Two of those tests
now fail intermittently and will be fixed by the upcoming series of work
#6391 and #6388.
@codygunton codygunton marked this pull request as ready for review May 26, 2024 04:42
@codygunton codygunton changed the title fix: ECCVM correctly handles points at infinity and group operation edge-cases fix: ECCVM correctly handles points at infinity and group operation edge cases May 26, 2024
@codygunton codygunton enabled auto-merge (squash) May 26, 2024 13:03
@@ -119,7 +119,7 @@ TEST_F(ClientIVCTests, BasicFailure)
* @brief Prove and verify accumulation of an arbitrary set of circuits
*
*/
TEST_F(ClientIVCTests, DISABLED_BasicLarge)
TEST_F(ClientIVCTests, BasicLarge)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this PR allow us to reinstate these core tests.

@@ -37,17 +38,17 @@ class ECCVMFlavor {
using RelationSeparator = FF;
using MSM = bb::eccvm::MSM<CycleGroup>;

static constexpr size_t NUM_WIRES = 74;
static constexpr size_t NUM_WIRES = 85;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note 15% additional time spent on commitments.

@codygunton codygunton merged commit a022220 into master May 27, 2024
90 checks passed
@codygunton codygunton deleted the zw/eccvm-infinity branch May 27, 2024 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants