feat(avm): initial external calls

[Maddiaa0]

• Implements call and static call opcodes without gas accounting

Note: there is alot of code duplication in these implementations

closes: #4127

[Maddiaa0]

Current dependencies on/for this PR:

• master
  • PR feat(avm): field arithmetic #4142
    • PR feat(avm): link up storage #4150
      • PR refactor(avm): remove the state manager in favour of journal #4195
        • PR feat(avm): initial external calls #4194 👈
          • PR chore: remove stubbed docs #4196
            • PR feat: accrued substate instructions #4197
              • PR feat(avm): environment getters #4203
                • PR feat(avm): revert instruction #4206
                  • PR refactor(avm): make interpreter a function not a class #4272

This stack of pull requests is managed by Graphite.

[AztecBot]

Benchmark results

Metrics with a significant change:

• batch_insert_into_append_only_tree_16_depth_ms (8): 27.1 (+89%)
• batch_insert_into_append_only_tree_16_depth_hash_ms (8): 0.832 (+43%)
• batch_insert_into_append_only_tree_32_depth_ms (512): 364 (+43%)
• batch_insert_into_append_only_tree_32_depth_hash_ms (512): 0.630 (+43%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit 18581265 and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	128 txs
l1_rollup_calldata_size_in_bytes	45,412	179,524	716,068
l1_rollup_calldata_gas	221,644	866,752	3,448,564
l1_rollup_execution_gas	314,063	984,424	3,668,153
l2_block_processing_time_in_ms	1,514 (+8%)	5,349 (+1%)	20,854
note_successful_decrypting_time_in_ms	239	819 (+3%)	3,121 (-1%)
note_trial_decrypting_time_in_ms	48.0 (+3%)	47.7 (-62%)	142 (-4%)
l2_block_building_time_in_ms	16,129	64,086	256,806
l2_block_rollup_simulation_time_in_ms	11,746	46,740	187,614
l2_block_public_tx_process_time_in_ms	4,348	17,257	68,900

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	5 blocks	10 blocks
node_history_sync_time_in_ms	16,013	30,329
note_history_successful_decrypting_time_in_ms	1,957	3,805 (-1%)
note_history_trial_decrypting_time_in_ms	153 (+13%)	399 (+61%)
node_database_size_in_bytes	3,583,762	3,609,393
pxe_database_size_in_bytes	29,923	59,478

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit	circuit_simulation_time_in_ms	circuit_input_size_in_bytes	circuit_output_size_in_bytes
private-kernel-init	213 (-1%)	44,681	25,561
private-kernel-ordering	166 (-1%)	43,753	14,809
base-rollup	1,199 (-1%)	128,338	881
root-rollup	79.0 (-2%)	4,088	677
private-kernel-inner	292 (-1%)	71,208	25,561
public-kernel-private-input	206 (-1%)	31,863	25,561
public-kernel-non-first-iteration	203 (-1%)	31,905	25,561
merge-rollup	7.64 (-1%)	2,608	881

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	2 leaves	8 leaves	16 leaves	32 leaves	128 leaves	64 leaves	512 leaves	1024 leaves	2048 leaves	8192 leaves
batch_insert_into_append_only_tree_16_depth_ms	12.2 (+1%)	17.9 (-4%)	⚠️ 27.1 (+89%)	19.6 (-3%)	27.2 (+5%)	66.9	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.9	17.5	23.0	31.6	47.0	143	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.625 (+1%)	0.628 (-4%)	⚠️ 0.832 (+43%)	0.552	0.500 (+3%)	0.444	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	N/A	N/A	N/A	83.8 (-5%)	51.9	⚠️ 364 (+43%)	478	920 (+1%)	3,582
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	N/A	N/A	N/A	159	96.0	543	1,055	2,079	8,223
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	0.483 (-3%)	0.498	⚠️ 0.630 (+43%)	0.437	0.430	0.428
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	N/A	N/A	N/A	119 (+2%)	62.3	368 (-1%)	774 (+1%)	1,455 (+2%)	5,647
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	N/A	N/A	N/A	197	104	691	1,363	2,707	10,771
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	0.509 (+2%)	0.503	0.477	0.488	0.475	0.474
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	N/A	67.4	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	N/A	94.1	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	N/A	0.596	N/A	N/A	N/A	N/A	N/A	N/A	N/A

Miscellaneous

Transaction sizes based on how many contracts are deployed in the tx.

Metric	0 deployed contracts	1 deployed contracts
tx_size_in_bytes	15,455	32,222

Transaction processing duration by data writes.

Metric	0 new commitments	1 new commitments
tx_pxe_processing_time_ms	375 (-1%)	1,086 (-1%)

Metric	1 public data writes
tx_sequencer_processing_time_ms	536 (-1%)

[fcarreiro]

IIUC this does not conform to the execute in the Instruction interface. In particular the interpreter run() is not awaiting on execute.

[Maddiaa0]

Ah yes correct, I will need to make everything async, will update ( I forgot to do this )

[fcarreiro]

I think the concepts of ExecutionEnvironment, Context, State, are getting a bit unclear to me. For example, a "context" in this line is actually a state:

aztec-packages/yarn-project/acir-simulator/src/avm/interpreter/interpreter.test.ts

Line 31 in 4027633

const context = new AvmMachineState(calldata, executionEnvironment);

[fcarreiro]

On the same line, it's getting less clear what the "entry point" would be for a client. First I expected it to be the interpreter, but it needs some pre-processing so no.

[Maddiaa0]

Ah the line above hangs from a pr where everything had different names, but were changed to line up with the yellow paper's definitions - I can rename the offending line.

The entry point would be instantiating a context, from the kernels function inputs, which will execute an interpreter.

Let me know if this model blows and we can get back to pen and paper

[fcarreiro]

In principle it feels strange that an instruction ends up transitively using an Interpreter, and that an interpreter uses Instructions. I understand that conceptually you "switch context" and "start a new execution" but this kind of breaks the elegance of having instructions be basically just a lambda.

I'm a bit surprised that this works, but we could leave it like this until we find a better way (which likely involve a more complex design).

[Maddiaa0]

I agree here, it feels a little recursive. But in my mind a nested instruction call is somewhat recursive, in this case the downstream call needs a completely fresh context to work within

[fcarreiro]

Encoding like this will not work mid/long term. I'd expect an encode_bytecode that takes Instruction[] to be more maintainable (but then Instruction needs a different design).

[Maddiaa0]

I agree, i was following a pattern laid out in another pr. I will alter this in future work and not this exact pr ( as it will be an annoyance later too )

[fcarreiro]

Better .haveLength(1)?
In general using more specific matchers gives better error messages.

[Maddiaa0]

I gave this ago, but the Map type does not have a property with .length on it as both keys(), values() and entries() return iterators, sadly we cant use it in this case; although i agree it looks much nicer