basic benchmark

[lispc]

Choose a circuit ( either spotTrade or Block including spotTrade ), get a table with following schema:

Both one core and multi core ( 4 core or 8 core ) ( or real time / user time ) results are useful :

time	groth16-zkutil	groth16-rapidsnark	plonk-plonkit
rescue hash	??	??	??
poseidon hash	??	??	??

plonkit should be upgraded to bellman beta branch ( If fully upgrading including solidity stuff costs much time, we can create a branch rather than push unfinished code to plonkit master )

The bench script code may be executed later so it is needed to be commited into the repo. ( We don't need a full auto script. Some manual operations are ok )

https://github.com/Fluidex/circuits/blob/master/tools/replace_hash.sh may be useful.

rapidsnark need a zkey, we may have to generate it.

[0xmountaintop]

Block(4, 1, 1) with poseidon_hash:

ubuntu@ip-172-31-11-58:~/repos/circuits/tools/benchmark/data/poseidon/block$ head *.time
==> plonkit.time <==
++ plonkit prove --srs_monomial_form /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20.key --circuit circuit.r1cs --witness witness.wtns --proof proof.bin

real    0m5.191s
user    1m21.696s
sys     0m3.429s

==> plonkit_lagrange.time <==
++ plonkit prove -m /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20.key -l /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20_lagrange.key -c circuit.r1cs -w witness.wtns -p proof.bin

real    0m5.074s
user    1m14.342s
sys     0m3.484s

==> rapidsnark.time <==
++ /home/ubuntu/repos/circuits/tools/benchmark/../../node_modules/rapidsnark/build/prover zkey.zkey witness.wtns proof.json public.json

real    0m1.316s
user    1m44.137s
sys     0m7.348s

==> zkutil.time <==
++ zkutil prove

real    0m1.197s
user    0m12.163s
sys     0m0.803s

[0xmountaintop]

Block(4, 1, 1) with rescue_hash:

ubuntu@ip-172-31-11-58:~/repos/circuits/tools/benchmark/data/rescue/block$ head *.time
==> plonkit.time <==
++ plonkit prove --srs_monomial_form /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20.key --circuit circuit.r1cs --witness witness.wtns --proof proof.bin

real    0m5.412s
user    1m23.291s
sys     0m3.742s

==> plonkit_lagrange.time <==
++ plonkit prove -m /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20.key -l /home/ubuntu/repos/circuits/tools/benchmark/keys/plonk/2pow20_lagrange.key -c circuit.r1cs -w witness.wtns -p proof.bin

real    0m5.270s
user    1m16.324s
sys     0m3.357s

==> rapidsnark.time <==
++ /home/ubuntu/repos/circuits/tools/benchmark/../../node_modules/rapidsnark/build/prover zkey.zkey witness.wtns proof.json public.json

real    0m1.288s
user    1m41.400s
sys     0m7.051s

==> zkutil.time <==
++ zkutil prove

real    0m1.805s
user    0m15.699s
sys     0m0.556s

[lispc]

better to post circuit here like 'Block(2,2,2,)'

[0xmountaintop]

I see.

[lispc]

==> ./data/rescue/massive/plonkit.time <==
real 2m34.194s
user 57m6.142s
sys 0m36.691s

==> ./data/rescue/massive/plonkit_lagrange.time <==
real 2m36.156s
user 53m46.198s
sys 0m30.630s

==> ./data/rescue/massive/zkutil.time <==
real 0m50.003s
user 10m31.481s
sys 0m8.330s

==> ./data/poseidon/massive/plonkit.time <==
real 2m36.773s
user 57m27.337s
sys 0m46.737s

==> ./data/poseidon/massive/plonkit_lagrange.time <==
real 2m39.482s
user 56m2.942s
sys 0m41.280s

==> ./data/poseidon/massive/zkutil.time <==
real 0m38.163s
user 9m37.281s
sys 0m6.639s
++ find ./ -name circuit.circom

• tail -n 3 ./data/rescue/massive/circuit.circom ./data/poseidon/massive/circuit.circom
  ==> ./data/rescue/massive/circuit.circom <==
  include "/home/ubuntu/repos/circuits/src/block.circom";
  component main = Block(10, 20, 20, 20);

[lispc]

rapidsnark zkey generation is too slow; zkutil only supports json witness, but nodejs seems cannot write huge json file (> 512MB ). We may need to cherry-pick wtns reading to zkutil

[lispc]

component main = Block(50, 20, 20, 20);

==> ./data/rescue/massive/plonkit.time <==
real 10m23.963s
user 222m25.871s
sys 3m27.515s

We will use this circuit for a while

[lispc]

Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz

component main = Block(100, 10, 10, 10); // it takes about 24h to process the circuit

real 18m50.111s
user 425m52.298s
sys 7m50.207s

component main = Block(100, 10, 10, 10);

real 0m14.529s
user 4m52.056s
sys 0m4.144s

[lispc]

Conclusion:

So the proving time is basically linear as expected.

100 tps needs 300 24core machine ( real core? ht core? )

Assume EC2 c5.12xlarge ( 48 vcpus $2.04 per Hour ). 100 tps costs 500-600$/h ? ( c5 instance has 3.5G freq ) , much cheaper than ETH GAS. Huawei cloud: C6s ( 48 vcpus 8.19CNY/h )

Latency about 15-20min~

[lispc]

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/plonkit_lagrange_gen.time <==
real 232m25.302s
user 7166m45.125s
sys 6m10.302s

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/plonkit.time <==
real 18m22.700s
user 421m24.006s
sys 5m55.358s

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/plonkit_lagrange.time <==
real 18m31.070s
user 371m51.932s
sys 10m21.829s

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/zkutil.time <==
real 1m51.334s
user 38m11.670s
sys 0m50.919s

==> /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.circom <==
include "/home/ubuntu/repos/circuits/src/block.circom";
component main = Block(100, 20, 20, 20);

[lispc]

[2021-05-15T05:08:07Z INFO  plonkit::plonk] transpile done, gates_count 42176699 hints size 11403099
[2021-05-15T05:11:29Z INFO  plonkit::plonk] circuit setup_polynomials.n 67108863 size 26

so 421766.99 gates per tx. 2**28 == 636 * 421766.99. more than 600 layer2 tx in one l2 block

[lispc]

Block(100, 20, 20, 20):

peak memory: 261GB

constructionPhase: 35:58.744
reduceConstants: 06:37.769
reduceConstraints: 12:27:16.233
classifySignals: 02:18.745
generateWitnessNames: 01:24.083
generateR1cs: 10:17.195
generateC: 29:40.610
generateSyms: .001
Command being timed: "node --trace-gc --trace-gc-ignore-scavenger --max-old-space-size=1400000 --initial-old-space-size=65500 --no-global-gc-scheduling --no-incremental-marking --max-semi-space-size=1024 --initial-heap-size=65500 /home/ubuntu/repos/circuits/node_modules/.bin/circom /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.circom -r /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.r1cs -c /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.cpp -s /home/ubuntu/repos/circuits/tools/benchmark/data/massive/circuit.sym -v"
User time (seconds): 49801.72
System time (seconds): 1605.20
Percent of CPU this job got: 102%
Elapsed (wall clock) time (h:mm:ss or m:ss): 13:59:48
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 274080964
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 1010020882
Voluntary context switches: 4558624
Involuntary context switches: 150365
Swaps: 0
File system inputs: 0
File system outputs: 21368608
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0

[lispc]

zksyn zk run verify-keys gen circuit-size

transfer needs 2 chunks, swap needs 5 chunks. So transfer needs 200k gates, swap needs 500k gates.

   Compiling key_generator v1.0.0 (/home/ubuntu/repos/zksync/core/bin/key_generator)
    Finished release [optimized] target(s) in 1m 50s
     Running `target/release/key_generator circuit-size`
Oct 09 08:52:33.004  INFO key_generator::zksync_key: Counting max zkSync circuit size for setup
Oct 09 08:52:38.107  INFO key_generator::zksync_key: setup_size_log2: 21, chunks: 10, gate_count: 2148475
Oct 09 08:52:55.589  INFO key_generator::zksync_key: setup_size_log2: 22, chunks: 32, gate_count: 4348332
Oct 09 08:53:22.293  INFO key_generator::zksync_key: setup_size_log2: 23, chunks: 72, gate_count: 8431960
Oct 09 08:54:36.849  INFO key_generator::zksync_key: setup_size_log2: 24, chunks: 156, gate_count: 16917362
Oct 09 08:56:45.615  INFO key_generator::zksync_key: setup_size_log2: 25, chunks: 322, gate_count: 33701320
Oct 09 09:01:02.216  INFO key_generator::zksync_key: setup_size_log2: 26, chunks: 654, gate_count: 67255497