refactor: optimize public call stack item hashing

[LHerskind]

Fixes #7092 for public, very similar to #7285.

Introduces a compressed version of the public_call_stack_item that is what we will be hashing instead of the full inputs with most values populated by zeros.

Notable savings is the SchnorrAccount entrypoint which is reduces all the way to 88K 👀

[LHerskind]

• refactor: showcase recursion case for transfer #7271
• refactor: optimize public call stack item hashing #7330 👈
• refactor: Optimize private call stack item hash for gate count #7285
• refactor: ARGS_HASH constants 64 -> 16 #7284
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @LHerskind and the rest of your teammates on Graphite

[github-actions]

Changes to circuit sizes

Generated at commit: 6fc00dff29e87ff93cefc7718c9f20385ae2eeaa, compared to commit: 586c0b019d0c2c462188f03c9c31d25a46e35eb6

🧾 Summary (100% most significant diffs)

Program	ACIR opcodes (+/-)	%	Circuit size (+/-)	%
public_kernel_setup	-645 ✅	-0.35%	-147,288 ✅	-10.04%
public_kernel_teardown	-1,642 ✅	-0.68%	-279,863 ✅	-16.82%
public_kernel_app_logic	-1,640 ✅	-0.68%	-279,861 ✅	-16.83%

---

Full diff report 👇

Program	ACIR opcodes (+/-)	%	Circuit size (+/-)	%
public_kernel_setup	182,710 (-645)	-0.35%	1,319,393 (-147,288)	-10.04%
public_kernel_teardown	241,237 (-1,642)	-0.68%	1,383,582 (-279,863)	-16.82%
public_kernel_app_logic	241,041 (-1,640)	-0.68%	1,383,344 (-279,861)	-16.83%

[AztecBot]

Benchmark results

Metrics with a significant change:

• app_circuit_witness_generation_time_in_ms (MultiCallEntrypoint:entrypoint): 651 (-31%)
• app_circuit_witness_generation_time_in_ms (GasToken:deploy): 553 (-20%)
• app_circuit_witness_generation_time_in_ms (SchnorrAccount:entrypoint): 820 (-35%)
• app_circuit_witness_generation_time_in_ms (FPC:fee_entrypoint_public): 97.1 (-59%)
• protocol_circuit_simulation_time_in_ms (public-kernel-setup): 443 (-18%)
• protocol_circuit_simulation_time_in_ms (public-kernel-app-logic): 424 (-16%)
• protocol_circuit_simulation_time_in_ms (public-kernel-teardown): 387 (-22%)
• avm_simulation_time_ms (Token:mint_public): 54.3 (-89%)
• avm_simulation_time_ms (Token:assert_minter_and_mint): 129 (+132%)
• avm_simulation_time_ms (Token:_increase_public_balance): 14.8 (-73%)
• avm_simulation_time_ms (FPC:pay_refund_with_shielded_rebate): 152 (+36%)
• app_circuit_proving_time_in_ms (SchnorrAccount:entrypoint): 4,722 (-83%)
• app_circuit_proving_time_in_ms (FPC:fee_entrypoint_public): 1,638 (-85%)
• app_circuit_size_in_gates (SchnorrAccount:entrypoint): 131,072 (-87%)
• app_circuit_size_in_gates (FPC:fee_entrypoint_public): 65,536 (-87%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric	1 threads	4 threads	16 threads	32 threads	64 threads
proof_construction_time_sha256_ms	5,719	1,545	708	732 (-3%)	779 (+1%)
proof_construction_time_sha256_30_ms	11,852 (+1%)	3,158	1,412	1,437	1,472
proof_construction_time_sha256_100_ms	43,772	11,763	5,440	5,396	5,541 (+3%)
proof_construction_time_poseidon_hash_ms	78.0	34.0	34.0	58.0 (-2%)	88.0 (+2%)
proof_construction_time_poseidon_hash_30_ms	1,515	415	200 (+1%)	229 (+4%)	264
proof_construction_time_poseidon_hash_100_ms	5,732	1,562	726 (+1%)	768 (-2%)	787

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	4 txs	8 txs	16 txs
l1_rollup_calldata_size_in_bytes	1,412	1,412	1,412
l1_rollup_calldata_gas	9,464	9,466	9,464
l1_rollup_execution_gas	611,203	611,356	611,505
l2_block_processing_time_in_ms	760	1,411	2,758 (+2%)
l2_block_building_time_in_ms	18,902 (-10%)	40,660 (-2%)	73,907 (-9%)
l2_block_rollup_simulation_time_in_ms	18,902 (-10%)	40,660 (-2%)	73,907 (-9%)
l2_block_public_tx_process_time_in_ms	16,125 (-10%)	37,679 (-2%)	70,960 (-9%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric	3 blocks	5 blocks
node_history_sync_time_in_ms	7,032 (+1%)	9,807 (-1%)
node_database_size_in_bytes	12,447,824	16,175,184
pxe_database_size_in_bytes	16,254	26,813

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit	simulation_time_in_ms	witness_generation_time_in_ms	proving_time_in_ms	input_size_in_bytes	output_size_in_bytes	proof_size_in_bytes	num_public_inputs	size_in_gates
private-kernel-init	110 (+1%)	402 (+2%)	14,635 (+12%)	19,990	55,022	74,432	2,259	524,288
private-kernel-inner	222	719 (-5%)	28,817 (+1%)	82,090	55,022	74,432	2,259	1,048,576
private-kernel-tail	1,052 (-1%)	2,582 (+3%)	55,839 (-8%)	62,345	62,057	14,912	399	2,097,152
base-parity	6.21 (+1%)	1,364 (-11%)	2,620 (-1%)	128	64.0	2,208	2.00	131,072
root-parity	49.2 (-1%)	66.5 (-2%)	41,062	27,100	64.0	2,720	18.0	2,097,152
base-rollup	6,226 (-5%)	4,962	91,744 (-3%)	170,330	728	3,648	47.0	4,194,304
root-rollup	110	78.9 (-6%)	24,099 (+1%)	25,253	620	3,456	41.0	1,048,576
public-kernel-setup	⚠️ 443 (-18%)	2,434 (+1%)	42,423 (+2%)	102,121	80,278	106,912	3,274	2,097,152
public-kernel-app-logic	⚠️ 424 (-16%)	3,383 (+1%)	43,189 (-1%)	102,121	80,278	106,912	3,274	2,097,152
public-kernel-tail	1,188 (+4%)	27,540 (+1%)	190,960	399,014	10,014	14,912	399	8,388,608
private-kernel-reset-small	291	1,344 (+9%)	28,169 (-10%)	79,209	55,022	74,432	2,259	1,048,576
public-kernel-teardown	⚠️ 387 (-22%)	3,333 (-3%)	42,967 (-7%)	102,121	80,278	106,912	3,274	2,097,152
merge-rollup	29.3	N/A	N/A	16,486	728	N/A	N/A	N/A
private-kernel-tail-to-public	N/A	8,576 (+1%)	57,775 (-4%)	N/A	N/A	106,912	3,274	2,097,152

Stats on running time collected for app circuits

Function	input_size_in_bytes	output_size_in_bytes	witness_generation_time_in_ms	proof_size_in_bytes	proving_time_in_ms	size_in_gates	num_public_inputs
ContractClassRegisterer:register	1,344	9,364	398 (+1%)	N/A	N/A	N/A	N/A
ContractInstanceDeployer:deploy	1,408	9,364	24.7 (+1%)	N/A	N/A	N/A	N/A
MultiCallEntrypoint:entrypoint	1,920	9,364	⚠️ 651 (-31%)	N/A	N/A	N/A	N/A
GasToken:deploy	1,376	9,364	⚠️ 553 (-20%)	N/A	N/A	N/A	N/A
SchnorrAccount:constructor	1,312	9,364	475	N/A	N/A	N/A	N/A
SchnorrAccount:entrypoint	2,304	9,364	⚠️ 820 (-35%)	16,000	⚠️ 4,722 (-83%)	⚠️ 131,072 (-87%)	433
Token:privately_mint_private_note	1,280	9,364	537 (-11%)	N/A	N/A	N/A	N/A
FPC:fee_entrypoint_public	1,344	9,364	⚠️ 97.1 (-59%)	16,000	⚠️ 1,638 (-85%)	⚠️ 65,536 (-87%)	433
Token:transfer	1,312	9,364	1,624 (-4%)	16,000	11,423 (-10%)	524,288	433
AuthRegistry:set_authorized (avm)	19,226	N/A	N/A	91,264	1,274 (-7%)	N/A	N/A
FPC:prepare_fee (avm)	26,668	N/A	N/A	91,328	2,816 (+1%)	N/A	N/A
Token:transfer_public (avm)	42,918	N/A	N/A	91,328	3,854 (-1%)	N/A	N/A
AuthRegistry:consume (avm)	33,104	N/A	N/A	91,264	2,703 (-3%)	N/A	N/A
FPC:pay_refund (avm)	36,833	N/A	N/A	91,296	24,152 (+3%)	N/A	N/A
Benchmarking:create_note	1,344	9,364	464	N/A	N/A	N/A	N/A
SchnorrAccount:verify_private_authwit	1,280	9,364	41.1	N/A	N/A	N/A	N/A
Token:unshield	1,376	9,364	1,308 (-5%)	N/A	N/A	N/A	N/A
FPC:fee_entrypoint_private	1,376	9,364	1,701 (-8%)	N/A	N/A	N/A	N/A

AVM Simulation

Time to simulate various public functions in the AVM.

Function	time_ms	bytecode_size_in_bytes
GasToken:_increase_public_balance	71.1	13,790
GasToken:set_portal	14.5 (-12%)	3,339
Token:constructor	90.6 (-1%)	23,692
FPC:constructor	64.1 (+6%)	13,592
GasToken:mint_public	53.2 (+6%)	10,158
Token:mint_public	⚠️ 54.3 (-89%)	19,034
Token:assert_minter_and_mint	⚠️ 129 (+132%)	12,925
AuthRegistry:set_authorized	27.7 (-11%)	7,812
FPC:prepare_fee	101 (-9%)	15,062
Token:transfer_public	34.3 (-14%)	31,218
FPC:pay_refund	131 (-3%)	25,260
Benchmarking:increment_balance	1,984 (-9%)	15,267
Token:_increase_public_balance	⚠️ 14.8 (-73%)	15,006
FPC:pay_refund_with_shielded_rebate	⚠️ 152 (+36%)	26,347

Public DB Access

Time to access various public DBs.

Function	time_ms
get-nullifier-index	0.155

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	256 leaves	512 leaves	1024 leaves
batch_insert_into_append_only_tree_16_depth_ms	10.4	16.9 (+1%)	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.8	31.7	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.602	0.519 (+1%)	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	48.3	77.2 (+1%)	132	245	479 (+2%)
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	95.9	159	287	543	1,055
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.493	0.474 (+1%)	0.453	0.444	0.446 (+1%)
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	60.0 (+1%)	112	184 (+1%)	354	704 (+1%)
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	109	207	355	691	1,363
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.506 (+1%)	0.502	0.487	0.478	0.481 (+1%)
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	73.0	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	133	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	0.519	N/A	N/A	N/A	N/A

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes	1 registered classes
tx_size_in_bytes	74,057	667,841

Transaction size based on fee payment method

| Metric | |
| - | |

[benesjan]

Looks good. Would just add a comment what is the compressed item and why we need it.

[benesjan]

Could you please add here a comment describing wtf is this and why we need it?

[LHerskind]

Will add 🫡

[sirasistant]

If my understanding is correct, I think this is exploitable. Imagine a public => public call. The AVM, when fullfilling the CALL opcode in the parent, asks the prover to provide a call stack item for the result of the call.

The prover, then spawns a child AVM, runs the function, and provides a call stack item.

Now the parent AVM exposes in its public inputs the hash of the nested call stack item, extracts the return values from the nested call stack item and writes the return values in the memory of the parent public function.

A malicious prover can, if the kernel does not check the return values when comparing the parent_function_avm_public_inputs.call_requests.hash vs the child_function_avm_public_inputs, provide to the parent AVM fake return values that will never be checked by the kernel to be equal to the ones actually returned by the child function. The parent function will think the child function returned something that wasn't returned

I think, the check request_hash <=> actual hash should check:

• Intent of the caller (target function, target selector, call context, arguments, etc)
• Any data exposed back to the caller that could change its behaviour (return values, revert code, gas left, etc)

In the case of private => public, since no data is exposed back to the caller, then only the intent needs to be checked, that is achieved in master, by this check:

    let item = if self.is_execution_request {
            self.as_execution_request()
        } else {
            self
        };

In the case of public => public, it's what I expressed above, both the intent and any data from the result that changes behavior of the caller

[LeilaWang]

Ooops. It's me that told Lasse that we didn't have to include the return_args hash. Because I think the public call request will be for the private functions only. It’s now being used between public calls because AVM can’t simulate the entire call request at once yet. But you are right it's an exploit at the moment and I guess we can include it now and remove it later!

[sirasistant]

It shouldn't be too complicated i think, It's just adding a few fields to the PublicCallStackItemCompressed struct and adding to it the logic of is_execution_request to it (maybe we could call it is_request_from_private?).

[LHerskind]

Will throw them back in, should be fairly straight forward with the struct in place.

So additions would be:

• returns_hash
• revert code
• gas left (all 4)

[sirasistant]

Thanks!