Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(avm executor): kernel outputs & execution hints in executor #6769

Merged
merged 19 commits into from
Jun 3, 2024
Merged

feat(avm executor): kernel outputs & execution hints in executor #6769

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
1ba1882
feat: add test that state getting opcodes are executed correctly
Maddiaa0 May 28, 2024
f955016
sweep
Maddiaa0 May 28, 2024
6da5d10
chore: add emit log execution tests
Maddiaa0 May 29, 2024
c813a44
temp
Maddiaa0 May 29, 2024
f59cb2e
fix: merge
Maddiaa0 May 30, 2024
33c9c01
fix: align offsets with real values
Maddiaa0 May 31, 2024
494f966
feat: mvp execution hints
Maddiaa0 May 31, 2024
fe02298
feat: sload and sstore execution
Maddiaa0 May 31, 2024
69a0ce0
fix: update sstore side effect counter
Maddiaa0 May 31, 2024
18fef35
temp
Maddiaa0 Jun 2, 2024
81d1ff5
feat: kernel exists opcodes with hints
Maddiaa0 Jun 2, 2024
e9c3acd
feat: update execution hints to map side effect counter to value
Maddiaa0 Jun 3, 2024
aca948c
chore: rebase fix, update constants
Maddiaa0 Jun 3, 2024
c4816e7
fix: generalise sload
Maddiaa0 Jun 3, 2024
390ec2d
chore: cleanup kernel test
Maddiaa0 Jun 3, 2024
c529842
chore: revert bb main.cpp changes
Maddiaa0 Jun 3, 2024
dd72cf8
Merge branch 'master' into md/kernel-outputs-executor
Maddiaa0 Jun 3, 2024
c258dd4
Merge branch 'master' into md/kernel-outputs-executor
Maddiaa0 Jun 3, 2024
36060f1
fix: gcc build error - no int cast
Maddiaa0 Jun 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 19 additions & 12 deletions barretenberg/cpp/pil/avm/avm_kernel.pil
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,30 +25,37 @@ namespace avm_kernel(256);
pol commit side_effect_counter;

// FIXED INDEXES
// Exists checks
pol START_NOTE_HASH_EXISTS_WRITE_OFFSET = 0;
pol START_EMIT_NOTE_HASH_WRITE_OFFSET = 4;
pol START_NULLIFIER_EXISTS_OFFSET = 8;
pol START_EMIT_NULLIFIER_WRITE_OFFSET = 12;
pol START_L1_TO_L2_MSG_EXISTS_WRITE_OFFSET = 16;
pol START_EMIT_UNENCRYPTED_LOG_WRITE_OFFSET = 20;
pol START_EMIT_L2_TO_l1_MSG = 24;
pol START_NULLIFIER_EXISTS_OFFSET = 32; // START_NOTE_HASH_EXISTS_WRITE_OFFSET + MAX_NOTE_HASH_READ_REQUESTS_PER_CALL TODO: exists and non exists
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjusted these values to line up with the true offsets when using the real PER_CALL limits, previously they were just small ranges as an MVP

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you now autogen these from TS?

pol START_L1_TO_L2_MSG_EXISTS_WRITE_OFFSET = 96; // START_NULLIFIER_EXISTS_OFFET + (MAX_NULLIFIER_READ_REQUESTS_PER_CALL + MAX_NULLIFIER_NON_EXISTENT_READ_REQUESTS_PER_CALL)

// Public storage requests
pol START_SLOAD_WRITE_OFFSET = 112; // START_L1_TO_L2_MSG_EXISTS_WRITE_OFFSET + MAX_L1_TO_L2_MSG_READ_REQUESTS_PER_CALL
pol START_SSTORE_WRITE_OFFSET = 144; // START_SSTORE_WRITE_OFFSET + MAX_PUBLIC_DATA_UPDATE_REQUESTS_PER_CALL

// Emit data
pol START_EMIT_NOTE_HASH_WRITE_OFFSET = 160; // START_SLOAD_WRITE_OFFSET + MAX_PUBLIC_DATA_READS_PER_CALL
pol START_EMIT_NULLIFIER_WRITE_OFFSET = 176; // START_EMIT_NOTE_HASH_WRITE_OFFSET + MAX_NEW_NOTE_HASHES_PER_CALL
pol START_EMIT_L2_TO_l1_MSG = 192; // START_EMIT_NULLIFIER_WRITE_OFFSET + MAX_NEW_NULLIFIERS_PER_CALL
pol START_EMIT_UNENCRYPTED_LOG_WRITE_OFFSET = 194; // START_EMIT_L2_TO_L1_MSG + MAX_NEW_L2_TO_L1_MSGS_PER_CALL
Comment on lines +28 to +41
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these hand-set or code-generated?

Copy link
Member Author

@Maddiaa0 Maddiaa0 May 31, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now hand set, but they mirror values in cpp, so you just hover over the value in the LSP and copy it over, our pil does not compile time evaluate constants yet


pol START_SLOAD_WRITE_OFFSET = 28;
pol START_SSTORE_WRITE_OFFSET = 32;

// TODO(https://github.com/AztecProtocol/aztec-packages/issues/6465): Must constrain write_offset counters to be less than side effect MAX
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just reordered

// Current write offsets for each opcode
pol commit note_hash_exist_write_offset;
pol commit emit_note_hash_write_offset;
pol commit nullifier_exists_write_offset;
pol commit emit_nullifier_write_offset;
pol commit l1_to_l2_msg_exists_write_offset;
pol commit emit_unencrypted_log_write_offset;
pol commit emit_l2_to_l1_msg_write_offset;

pol commit sload_write_offset;
pol commit sstore_write_offset;

pol commit emit_note_hash_write_offset;
pol commit emit_nullifier_write_offset;
pol commit emit_unencrypted_log_write_offset;
pol commit emit_l2_to_l1_msg_write_offset;


pol NOT_LAST = (1 - avm_main.last);

// Constraints to increase the offsets when the opcodes are found
Expand Down
16 changes: 8 additions & 8 deletions barretenberg/cpp/src/barretenberg/relations/generated/avm/avm_main.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1342,7 +1342,7 @@ template <typename FF_> class avm_mainImpl {
Avm_DECLARE_VIEWS(114);

auto tmp = (avm_main_sel_op_emit_note_hash *
(avm_kernel_kernel_out_offset - (avm_kernel_emit_note_hash_write_offset + FF(4))));
(avm_kernel_kernel_out_offset - (avm_kernel_emit_note_hash_write_offset + FF(160))));
tmp *= scaling_factor;
std::get<114>(evals) += tmp;
}
Expand All @@ -1359,7 +1359,7 @@ template <typename FF_> class avm_mainImpl {
Avm_DECLARE_VIEWS(116);

auto tmp = (avm_main_sel_op_nullifier_exists *
(avm_kernel_kernel_out_offset - (avm_kernel_nullifier_exists_write_offset + FF(8))));
(avm_kernel_kernel_out_offset - (avm_kernel_nullifier_exists_write_offset + FF(32))));
tmp *= scaling_factor;
std::get<116>(evals) += tmp;
}
Expand All @@ -1376,7 +1376,7 @@ template <typename FF_> class avm_mainImpl {
Avm_DECLARE_VIEWS(118);

auto tmp = (avm_main_sel_op_emit_nullifier *
(avm_kernel_kernel_out_offset - (avm_kernel_emit_nullifier_write_offset + FF(12))));
(avm_kernel_kernel_out_offset - (avm_kernel_emit_nullifier_write_offset + FF(176))));
tmp *= scaling_factor;
std::get<118>(evals) += tmp;
}
Expand All @@ -1393,7 +1393,7 @@ template <typename FF_> class avm_mainImpl {
Avm_DECLARE_VIEWS(120);

auto tmp = (avm_main_sel_op_l1_to_l2_msg_exists *
(avm_kernel_kernel_out_offset - (avm_kernel_l1_to_l2_msg_exists_write_offset + FF(16))));
(avm_kernel_kernel_out_offset - (avm_kernel_l1_to_l2_msg_exists_write_offset + FF(96))));
tmp *= scaling_factor;
std::get<120>(evals) += tmp;
}
Expand All @@ -1410,7 +1410,7 @@ template <typename FF_> class avm_mainImpl {
Avm_DECLARE_VIEWS(122);

auto tmp = (avm_main_sel_op_emit_unencrypted_log *
(avm_kernel_kernel_out_offset - (avm_kernel_emit_unencrypted_log_write_offset + FF(20))));
(avm_kernel_kernel_out_offset - (avm_kernel_emit_unencrypted_log_write_offset + FF(194))));
tmp *= scaling_factor;
std::get<122>(evals) += tmp;
}
Expand All @@ -1427,7 +1427,7 @@ template <typename FF_> class avm_mainImpl {
Avm_DECLARE_VIEWS(124);

auto tmp = (avm_main_sel_op_emit_l2_to_l1_msg *
(avm_kernel_kernel_out_offset - (avm_kernel_emit_l2_to_l1_msg_write_offset + FF(24))));
(avm_kernel_kernel_out_offset - (avm_kernel_emit_l2_to_l1_msg_write_offset + FF(192))));
tmp *= scaling_factor;
std::get<124>(evals) += tmp;
}
Expand All @@ -1444,7 +1444,7 @@ template <typename FF_> class avm_mainImpl {
Avm_DECLARE_VIEWS(126);

auto tmp =
(avm_main_sel_op_sload * (avm_kernel_kernel_out_offset - (avm_kernel_sload_write_offset + FF(28))));
(avm_main_sel_op_sload * (avm_kernel_kernel_out_offset - (avm_kernel_sload_write_offset + FF(112))));
tmp *= scaling_factor;
std::get<126>(evals) += tmp;
}
Expand All @@ -1461,7 +1461,7 @@ template <typename FF_> class avm_mainImpl {
Avm_DECLARE_VIEWS(128);

auto tmp =
(avm_main_sel_op_sstore * (avm_kernel_kernel_out_offset - (avm_kernel_sstore_write_offset + FF(32))));
(avm_main_sel_op_sstore * (avm_kernel_kernel_out_offset - (avm_kernel_sstore_write_offset + FF(144))));
tmp *= scaling_factor;
std::get<128>(evals) += tmp;
}
Expand Down
8 changes: 8 additions & 0 deletions barretenberg/cpp/src/barretenberg/vm/avm_trace/avm_common.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,17 @@

#include <array>
#include <cstdint>
#include <unordered_map>

namespace bb::avm_trace {

using Flavor = bb::AvmFlavor;
using FF = Flavor::FF;

} // namespace bb::avm_trace

namespace bb::avm_trace {

// There are 4 public input columns, 1 for context inputs, and 3 for emitting side effects
using VmPublicInputs = std::tuple<std::array<FF, KERNEL_INPUTS_LENGTH>, // Input: Kernel context inputs
std::array<FF, KERNEL_OUTPUTS_LENGTH>, // Output: Kernel outputs data
Expand All @@ -35,4 +40,7 @@ static const size_t NUM_MEM_SPACES = 256;
static const uint8_t INTERNAL_CALL_SPACE_ID = 255;
static const uint32_t MAX_SIZE_INTERNAL_STACK = 1 << 16;

// Side effect counter -> value
using ExecutionHints = std::unordered_map<uint32_t, FF>;

} // namespace bb::avm_trace
11 changes: 11 additions & 0 deletions barretenberg/cpp/src/barretenberg/vm/avm_trace/avm_deserialization.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ namespace {
const std::vector<OperandType> three_operand_format = {
OperandType::INDIRECT, OperandType::TAG, OperandType::UINT32, OperandType::UINT32, OperandType::UINT32,
};
const std::vector<OperandType> kernel_input_operand_format = { OperandType::INDIRECT, OperandType::UINT32 };

const std::vector<OperandType> getter_format = {
OperandType::INDIRECT,
Expand Down Expand Up @@ -53,6 +54,16 @@ const std::unordered_map<OpCode, std::vector<OperandType>> OPCODE_WIRE_FORMAT =
{ OpCode::FEEPERL2GAS, getter_format },
{ OpCode::FEEPERDAGAS, getter_format },
{ OpCode::TRANSACTIONFEE, getter_format },

// TODO: ordering inline with spec
{ OpCode::EMITNOTEHASH, getter_format }, // TODO: new format for these
{ OpCode::EMITNULLIFIER, getter_format }, // TODO: new format for these
{ OpCode::EMITUNENCRYPTEDLOG, getter_format },
{ OpCode::SLOAD, { OperandType::INDIRECT, OperandType::UINT32, OperandType::UINT32 } },
{ OpCode::SSTORE, { OperandType::INDIRECT, OperandType::UINT32, OperandType::UINT32 } },
{ OpCode::NOTEHASHEXISTS, { OperandType::INDIRECT, OperandType::UINT32, OperandType::UINT32 } },
{ OpCode::NULLIFIEREXISTS, { OperandType::INDIRECT, OperandType::UINT32, OperandType::UINT32 } },
{ OpCode::L1TOL2MSGEXISTS, { OperandType::INDIRECT, OperandType::UINT32, OperandType::UINT32 } },
// CONTRACTCALLDEPTH, -- not in simulator
// Execution Environment - Globals
{ OpCode::CHAINID, getter_format },
Expand Down
78 changes: 58 additions & 20 deletions barretenberg/cpp/src/barretenberg/vm/avm_trace/avm_execution.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,16 @@ std::vector<FF> Execution::getDefaultPublicInputs()
* @return The verifier key and zk proof of the execution.
*/
std::tuple<AvmFlavor::VerificationKey, HonkProof> Execution::prove(std::vector<uint8_t> const& bytecode,
std::vector<FF> const& calldata)
std::vector<FF> const& calldata,
std::vector<FF> const& public_inputs_vec,
ExecutionHints const& execution_hints)
{
// TODO: temp
info("logging to silence warning for now: ", public_inputs_vec.size());

auto instructions = Deserialization::parse(bytecode);
std::vector<FF> returndata{};
auto trace = gen_trace(instructions, returndata, calldata, getDefaultPublicInputs());
auto trace = gen_trace(instructions, returndata, calldata, getDefaultPublicInputs(), execution_hints);
auto circuit_builder = bb::AvmCircuitBuilder();
circuit_builder.set_trace(std::move(trace));

Expand All @@ -63,6 +68,23 @@ std::tuple<AvmFlavor::VerificationKey, HonkProof> Execution::prove(std::vector<u
return std::make_tuple(*verifier.key, proof);
}

/**
* @brief Generate the execution trace pertaining to the supplied instructions.
*
* @param instructions A vector of the instructions to be executed.
* @param calldata expressed as a vector of finite field elements.
* @param public_inputs expressed as a vector of finite field elements.
* @return The trace as a vector of Row.
*/
std::vector<Row> Execution::gen_trace(std::vector<Instruction> const& instructions,
std::vector<FF> const& calldata,
std::vector<FF> const& public_inputs,
ExecutionHints const& execution_hints)
{
std::vector<FF> returndata{};
return gen_trace(instructions, returndata, calldata, public_inputs, execution_hints);
}

/**
* @brief Convert Public Inputs
*
Expand Down Expand Up @@ -135,28 +157,13 @@ bool Execution::verify(AvmFlavor::VerificationKey vk, HonkProof const& proof)
*
* @param instructions A vector of the instructions to be executed.
* @param calldata expressed as a vector of finite field elements.
* @param public_inputs expressed as a vector of finite field elements.
* @return The trace as a vector of Row.
*/
std::vector<Row> Execution::gen_trace(std::vector<Instruction> const& instructions,
std::vector<FF> const& calldata,
std::vector<FF> const& public_inputs)
{
std::vector<FF> returndata{};
return gen_trace(instructions, returndata, calldata, public_inputs);
}

/**
* @brief Generate the execution trace pertaining to the supplied instructions.
*
* @param instructions A vector of the instructions to be executed.
* @param calldata expressed as a vector of finite field elements.
* @return The trace as a vector of Row.
*/
std::vector<Row> Execution::gen_trace(std::vector<Instruction> const& instructions, std::vector<FF> const& calldata)
std::vector<FF> const& public_inputs_vec)
{
std::vector<FF> returndata{};
std::vector<FF> public_inputs_vec = {};
return gen_trace(instructions, returndata, calldata, public_inputs_vec);
}

Expand All @@ -171,13 +178,14 @@ std::vector<Row> Execution::gen_trace(std::vector<Instruction> const& instructio
std::vector<Row> Execution::gen_trace(std::vector<Instruction> const& instructions,
std::vector<FF>& returndata,
std::vector<FF> const& calldata,
std::vector<FF> const& public_inputs_vec)
std::vector<FF> const& public_inputs_vec,
ExecutionHints const& execution_hints)

{
// TODO(https://github.com/AztecProtocol/aztec-packages/issues/6718): construction of the public input columns
// should be done in the kernel - this is stubbed and underconstrained
VmPublicInputs public_inputs = convert_public_inputs(public_inputs_vec);
AvmTraceBuilder trace_builder(public_inputs);
AvmTraceBuilder trace_builder(public_inputs, execution_hints);

// Copied version of pc maintained in trace builder. The value of pc is evolving based
// on opcode logic and therefore is not maintained here. However, the next opcode in the execution
Expand Down Expand Up @@ -338,6 +346,36 @@ std::vector<Row> Execution::gen_trace(std::vector<Instruction> const& instructio
case OpCode::TIMESTAMP:
trace_builder.op_timestamp(std::get<uint32_t>(inst.operands.at(1)));
break;
case OpCode::NOTEHASHEXISTS:
trace_builder.op_note_hash_exists(std::get<uint32_t>(inst.operands.at(1)),
std::get<uint32_t>(inst.operands.at(2)));
break;
case OpCode::EMITNOTEHASH:
trace_builder.op_emit_note_hash(std::get<uint32_t>(inst.operands.at(1)));
break;
case OpCode::NULLIFIEREXISTS:
trace_builder.op_nullifier_exists(std::get<uint32_t>(inst.operands.at(1)),
std::get<uint32_t>(inst.operands.at(2)));
break;
case OpCode::EMITNULLIFIER:
trace_builder.op_emit_nullifier(std::get<uint32_t>(inst.operands.at(1)));
break;
case OpCode::SLOAD:
trace_builder.op_sload(std::get<uint32_t>(inst.operands.at(1)), std::get<uint32_t>(inst.operands.at(2)));
break;
case OpCode::SSTORE:
trace_builder.op_sstore(std::get<uint32_t>(inst.operands.at(1)), std::get<uint32_t>(inst.operands.at(2)));
break;
case OpCode::L1TOL2MSGEXISTS:
trace_builder.op_l1_to_l2_msg_exists(std::get<uint32_t>(inst.operands.at(1)),
std::get<uint32_t>(inst.operands.at(2)));
break;
case OpCode::EMITUNENCRYPTEDLOG:
trace_builder.op_emit_unencrypted_log(std::get<uint32_t>(inst.operands.at(1)));
break;
case OpCode::SENDL2TOL1MSG:
trace_builder.op_emit_l2_to_l1_msg(std::get<uint32_t>(inst.operands.at(1)));
break;
// Machine State - Internal Control Flow
case OpCode::JUMP:
trace_builder.jump(std::get<uint32_t>(inst.operands.at(0)));
Expand Down
14 changes: 10 additions & 4 deletions barretenberg/cpp/src/barretenberg/vm/avm_trace/avm_execution.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,20 @@ class Execution {
static std::vector<Row> gen_trace(std::vector<Instruction> const& instructions,
std::vector<FF>& returndata,
std::vector<FF> const& calldata,
std::vector<FF> const& public_inputs);
std::vector<FF> const& public_inputs,
ExecutionHints const& execution_hints = {});
static std::vector<Row> gen_trace(std::vector<Instruction> const& instructions,
std::vector<FF> const& calldata = {});
std::vector<FF> const& calldata = {},
std::vector<FF> const& public_inputs = {});
static std::vector<Row> gen_trace(std::vector<Instruction> const& instructions,
std::vector<FF> const& calldata,
std::vector<FF> const& public_inputs);
std::vector<FF> const& public_inputs,
ExecutionHints const& execution_hints);

static std::tuple<AvmFlavor::VerificationKey, bb::HonkProof> prove(std::vector<uint8_t> const& bytecode,
std::vector<FF> const& calldata = {});
std::vector<FF> const& calldata = {},
std::vector<FF> const& public_inputs_vec = {},
ExecutionHints const& execution_hints = {});
static bool verify(AvmFlavor::VerificationKey vk, HonkProof const& proof);
};

Expand Down
Loading
Loading