Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Integrated native ACVM #4903

Merged
merged 67 commits into from
Mar 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
e141d30
Initial commit
PhilWindle Mar 3, 2024
73fef6b
Fixes
PhilWindle Mar 3, 2024
fd8714c
Merge branch 'master' into pw/acvm-sim
PhilWindle Mar 4, 2024
1eef3b3
Updated rebuild patterns
PhilWindle Mar 4, 2024
82a8bfb
Fixes
PhilWindle Mar 4, 2024
31de2e6
Cleanup
PhilWindle Mar 4, 2024
5d17ada
Fixes
PhilWindle Mar 4, 2024
b70afd6
Cleanup
PhilWindle Mar 4, 2024
bbc4f10
Comments
PhilWindle Mar 4, 2024
fd7969d
Merge branch 'master' into pw/acvm-sim
PhilWindle Mar 4, 2024
d231e4e
Formatting
PhilWindle Mar 4, 2024
32719b8
Merge branch 'pw/acvm-sim' of github.com:AztecProtocol/aztec-packages…
PhilWindle Mar 4, 2024
33ec2d4
Fixes
PhilWindle Mar 4, 2024
88cada1
Lock file
PhilWindle Mar 4, 2024
c1f13bc
Test
PhilWindle Mar 4, 2024
2502ec8
Attempt fix
PhilWindle Mar 4, 2024
10f99d1
Fixes
PhilWindle Mar 4, 2024
9720e26
Removed nargo
PhilWindle Mar 4, 2024
3aa5141
Comment
PhilWindle Mar 4, 2024
0b42c30
Fix
PhilWindle Mar 4, 2024
60b9c11
Formatting
PhilWindle Mar 4, 2024
c7b50e8
Fix
PhilWindle Mar 4, 2024
3fad6f1
Review changes
PhilWindle Mar 4, 2024
f894f85
Review changes
PhilWindle Mar 5, 2024
217137d
Review changes
PhilWindle Mar 5, 2024
4f29002
Fixes
PhilWindle Mar 5, 2024
4f087e0
Created new 'fs' package in noir
PhilWindle Mar 5, 2024
fa621c7
More refactoring
PhilWindle Mar 5, 2024
1d3bb36
Revert "More refactoring"
PhilWindle Mar 5, 2024
8edc752
Revert "Created new 'fs' package in noir"
PhilWindle Mar 5, 2024
9293da2
Formatting
PhilWindle Mar 5, 2024
83bfc4d
Fix
PhilWindle Mar 5, 2024
cf85be9
Formatting
PhilWindle Mar 5, 2024
a5ba188
Dockerfile debugging
PhilWindle Mar 5, 2024
99df044
First attempt at docker fix
PhilWindle Mar 5, 2024
77f6c1a
Merge branch 'master' into pw/acvm-sim
PhilWindle Mar 5, 2024
f6c0ce6
Fix
PhilWindle Mar 5, 2024
64a9d87
Format revert
PhilWindle Mar 5, 2024
3e00858
Format fix
PhilWindle Mar 5, 2024
f05e4ae
Fix
PhilWindle Mar 5, 2024
662f6cc
Merge branch 'pw/acvm-sim' of github.com:AztecProtocol/aztec-packages…
PhilWindle Mar 5, 2024
b2b34c6
Removed unrequired option
PhilWindle Mar 5, 2024
eb91042
Merge branch 'master' into pw/acvm-sim
PhilWindle Mar 5, 2024
4cbcfb4
Docker file updates
PhilWindle Mar 6, 2024
f067eb9
Fix
PhilWindle Mar 6, 2024
fca003a
Fix
PhilWindle Mar 6, 2024
745a359
Merge branch 'master' into pw/acvm-sim
PhilWindle Mar 6, 2024
6bf506f
Fix build manifest
PhilWindle Mar 6, 2024
d36f8bc
Try resource anchor
PhilWindle Mar 6, 2024
1985c62
Another attempt
PhilWindle Mar 6, 2024
bb2686e
Try again
PhilWindle Mar 6, 2024
9234d28
And again
PhilWindle Mar 6, 2024
9e0cce3
Try again
PhilWindle Mar 6, 2024
4a47fbd
Try again
PhilWindle Mar 6, 2024
3486cb0
Try again
PhilWindle Mar 6, 2024
c7d2fcb
Merge branch 'master' into pw/acvm-sim
PhilWindle Mar 6, 2024
b426f52
Upgrade e2e test machines
PhilWindle Mar 6, 2024
532e065
Upgrade e2e machines
PhilWindle Mar 6, 2024
d0f61f4
Force re-run
PhilWindle Mar 6, 2024
eef61ae
Use WASM for small circuits
PhilWindle Mar 6, 2024
2857617
Revert machine upgrade
PhilWindle Mar 6, 2024
a3ead17
Capture average circuit simulation times
PhilWindle Mar 6, 2024
8391b8e
Formatting
PhilWindle Mar 6, 2024
3ea825f
Merge branch 'master' into pw/acvm-sim
PhilWindle Mar 6, 2024
392b770
Build fix
PhilWindle Mar 6, 2024
8c710ed
Merge branch 'pw/acvm-sim' of github.com:AztecProtocol/aztec-packages…
PhilWindle Mar 6, 2024
9201823
Formatting
PhilWindle Mar 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 49 additions & 132 deletions .circleci/config.yml

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion build_manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# dependencies: An array of other projects that this project depends on.
# runDependencies: Additional projects that are needed to run a container/compose file. Ensures they're pulled first.

# Builds noir for x86_64 and arm64, creating a runnable container just with nargo.
# Builds noir for x86_64 and arm64, creating a runnable container just with nargo + acvm.
noir:
buildDir: noir
dockerfile: Dockerfile.native
Expand Down Expand Up @@ -169,6 +169,7 @@ yarn-project:
- noir-packages
- l1-contracts
- noir-projects
- noir
multiarch: host

# A runnable container, sets entrypoint to be the aztec infrastructure entrypoint.
Expand Down Expand Up @@ -215,6 +216,7 @@ end-to-end:
- noir-packages
- l1-contracts
- noir-projects
- noir
runDependencies:
- aztec

Expand Down
1 change: 1 addition & 0 deletions noir/.rebuild_patterns_native
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@
^noir/noir-repo/tooling/nargo_toml
^noir/noir-repo/tooling/nargo_fmt
^noir/noir-repo/tooling/noirc_abi
^noir/noir-repo/tooling/acvm_cli
1 change: 1 addition & 0 deletions noir/Dockerfile.native
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,5 @@ FROM ubuntu:focal
# Install git as nargo needs it to clone.
RUN apt-get update && apt-get install -y git tini && rm -rf /var/lib/apt/lists/* && apt-get clean
COPY --from=0 /usr/src/noir/noir-repo/target/release/nargo /usr/src/noir/noir-repo/target/release/nargo
COPY --from=0 /usr/src/noir/noir-repo/target/release/acvm /usr/src/noir/noir-repo/target/release/acvm
ENTRYPOINT ["/usr/bin/tini", "--", "/usr/src/noir/noir-repo/target/release/nargo"]
20 changes: 20 additions & 0 deletions noir/noir-repo/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion noir/noir-repo/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ members = [
"tooling/nargo_toml",
"tooling/noirc_abi",
"tooling/noirc_abi_wasm",
"tooling/acvm_cli",
# ACVM
"acvm-repo/acir_field",
"acvm-repo/acir",
Expand All @@ -36,7 +37,7 @@ members = [
"acvm-repo/blackbox_solver",
"acvm-repo/bn254_blackbox_solver",
]
default-members = ["tooling/nargo_cli"]
default-members = ["tooling/nargo_cli", "tooling/acvm_cli"]
resolver = "2"

[workspace.package]
Expand Down Expand Up @@ -78,6 +79,7 @@ noir_lsp = { path = "tooling/lsp" }
noir_debugger = { path = "tooling/debugger" }
noirc_abi = { path = "tooling/noirc_abi" }
bb_abstraction_leaks = { path = "tooling/bb_abstraction_leaks" }
acvm_cli = { path = "tooling/acvm_cli" }

# LSP
async-lsp = { version = "0.1.0", default-features = false }
Expand Down
2 changes: 1 addition & 1 deletion noir/noir-repo/docs/scripts/codegen_nargo_reference.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,4 @@ sidebar_position: 0
---
" > $NARGO_REFERENCE

cargo run -F codegen-docs -- info >> $NARGO_REFERENCE
cargo run --bin nargo -F codegen-docs -- info >> $NARGO_REFERENCE
38 changes: 38 additions & 0 deletions noir/noir-repo/tooling/acvm_cli/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[package]
name = "acvm_cli"
description = "The entrypoint for executing the ACVM"
# x-release-please-start-version
version = "0.40.0"
# x-release-please-end
authors.workspace = true
edition.workspace = true
license.workspace = true
rust-version.workspace = true
repository.workspace = true

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

# Rename binary from `acvm_cli` to `acvm`
[[bin]]
name = "acvm"
path = "src/main.rs"

[dependencies]
thiserror.workspace = true
toml.workspace = true
color-eyre = "0.6.2"
clap.workspace = true
acvm.workspace = true
nargo.workspace = true
const_format.workspace = true
bn254_blackbox_solver.workspace = true
acir.workspace = true

# Logs
tracing-subscriber.workspace = true
tracing-appender = "0.2.3"

[dev-dependencies]
rand = "0.8.5"
proptest = "1.2.0"
paste = "1.0.14"
79 changes: 79 additions & 0 deletions noir/noir-repo/tooling/acvm_cli/src/cli/execute_cmd.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
use std::io::{self, Write};

use acir::circuit::Circuit;
use acir::native_types::WitnessMap;
use bn254_blackbox_solver::Bn254BlackBoxSolver;
use clap::Args;

use crate::cli::fs::inputs::{read_bytecode_from_file, read_inputs_from_file};
use crate::cli::fs::witness::save_witness_to_dir;
use crate::errors::CliError;
use nargo::ops::{execute_circuit, DefaultForeignCallExecutor};

use super::fs::witness::create_output_witness_string;

/// Executes a circuit to calculate its return value
#[derive(Debug, Clone, Args)]
pub(crate) struct ExecuteCommand {
/// Write the execution witness to named file
#[clap(long, short)]
output_witness: Option<String>,

/// The name of the toml file which contains the input witness map
#[clap(long, short)]
input_witness: String,

/// The name of the binary file containing circuit bytecode
#[clap(long, short)]
bytecode: String,

/// The working directory
#[clap(long, short)]
working_directory: String,

/// Set to print output witness to stdout
#[clap(long, short, action)]
print: bool,
}

fn run_command(args: ExecuteCommand) -> Result<String, CliError> {
let bytecode = read_bytecode_from_file(&args.working_directory, &args.bytecode)?;
let circuit_inputs = read_inputs_from_file(&args.working_directory, &args.input_witness)?;
let output_witness = execute_program_from_witness(&circuit_inputs, &bytecode, None)?;
let output_witness_string = create_output_witness_string(&output_witness)?;
if args.output_witness.is_some() {
save_witness_to_dir(
&output_witness_string,
&args.working_directory,
&args.output_witness.unwrap(),
)?;
}
Ok(output_witness_string)
}

pub(crate) fn run(args: ExecuteCommand) -> Result<String, CliError> {
let print = args.print;
let output_witness_string = run_command(args)?;
if print {
io::stdout().write_all(output_witness_string.as_bytes()).unwrap();
}
Ok(output_witness_string)
}

pub(crate) fn execute_program_from_witness(
inputs_map: &WitnessMap,
bytecode: &Vec<u8>,
foreign_call_resolver_url: Option<&str>,
) -> Result<WitnessMap, CliError> {
let blackbox_solver = Bn254BlackBoxSolver::new();
let circuit: Circuit = Circuit::deserialize_circuit(&bytecode)
.map_err(|_| CliError::CircuitDeserializationError())?;
let result = execute_circuit(
&circuit,
inputs_map.clone(),
&blackbox_solver,
&mut DefaultForeignCallExecutor::new(true, foreign_call_resolver_url),
)
.map_err(|e| CliError::CircuitExecutionError(e));
result
}
54 changes: 54 additions & 0 deletions noir/noir-repo/tooling/acvm_cli/src/cli/fs/inputs.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
use acir::{
native_types::{Witness, WitnessMap},
FieldElement,
};
use toml::Table;

use crate::errors::{CliError, FilesystemError};
use std::{fs::read, path::Path};

/// Returns the circuit's parameters parsed from a toml file at the given location
pub(crate) fn read_inputs_from_file<P: AsRef<Path>>(
working_directory: P,
file_name: &String,
) -> Result<WitnessMap, CliError> {
let file_path = working_directory.as_ref().join(file_name);
if !file_path.exists() {
return Err(CliError::FilesystemError(FilesystemError::MissingTomlFile(
file_name.to_owned(),
file_path,
)));
}

let input_string = std::fs::read_to_string(file_path)
.map_err(|_| FilesystemError::InvalidTomlFile(file_name.clone()))?;
let input_map = input_string
.parse::<Table>()
.map_err(|_| FilesystemError::InvalidTomlFile(file_name.clone()))?;
let mut witnesses: WitnessMap = WitnessMap::new();
for (key, value) in input_map.into_iter() {
let index =
Witness(key.trim().parse().map_err(|_| CliError::WitnessIndexError(key.clone()))?);
if !value.is_str() {
return Err(CliError::WitnessValueError(key.clone()));
}
let field = FieldElement::from_hex(value.as_str().unwrap()).unwrap();
witnesses.insert(index, field);
}

Ok(witnesses)
}

/// Returns the circuit's bytecode read from the file at the given location
pub(crate) fn read_bytecode_from_file<P: AsRef<Path>>(
working_directory: P,
file_name: &String,
) -> Result<Vec<u8>, FilesystemError> {
let file_path = working_directory.as_ref().join(file_name);
if !file_path.exists() {
return Err(FilesystemError::MissingBytecodeFile(file_name.to_owned(), file_path));
}
let bytecode: Vec<u8> =
read(file_path).map_err(|_| FilesystemError::InvalidBytecodeFile(file_name.clone()))?;
Ok(bytecode)
}
2 changes: 2 additions & 0 deletions noir/noir-repo/tooling/acvm_cli/src/cli/fs/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
pub(super) mod inputs;
pub(super) mod witness;
36 changes: 36 additions & 0 deletions noir/noir-repo/tooling/acvm_cli/src/cli/fs/witness.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
use std::{
collections::BTreeMap,
fs::File,
io::Write,
path::{Path, PathBuf},
};

use acvm::acir::native_types::WitnessMap;

use crate::errors::{CliError, FilesystemError};

/// Saves the provided output witnesses to a toml file created at the given location
pub(crate) fn save_witness_to_dir<P: AsRef<Path>>(
output_witness: &String,
witness_dir: P,
file_name: &String,
) -> Result<PathBuf, FilesystemError> {
let witness_path = witness_dir.as_ref().join(file_name);

let mut file = File::create(&witness_path)
.map_err(|_| FilesystemError::OutputWitnessCreationFailed(file_name.clone()))?;
write!(file, "{}", output_witness)
.map_err(|_| FilesystemError::OutputWitnessWriteFailed(file_name.clone()))?;

Ok(witness_path)
}

/// Creates a toml representation of the provided witness map
pub(crate) fn create_output_witness_string(witnesses: &WitnessMap) -> Result<String, CliError> {
let mut witness_map: BTreeMap<String, String> = BTreeMap::new();
for (key, value) in witnesses.clone().into_iter() {
witness_map.insert(key.0.to_string(), format!("0x{}", value.to_hex()));
}

toml::to_string(&witness_map).map_err(|_| CliError::OutputWitnessSerializationFailed())
}
41 changes: 41 additions & 0 deletions noir/noir-repo/tooling/acvm_cli/src/cli/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
use clap::{Parser, Subcommand};
use color_eyre::eyre;
use const_format::formatcp;

mod execute_cmd;
mod fs;

const ACVM_VERSION: &str = env!("CARGO_PKG_VERSION");

static VERSION_STRING: &str = formatcp!("version = {}\n", ACVM_VERSION,);

#[derive(Parser, Debug)]
#[command(name="acvm", author, version=VERSION_STRING, about, long_about = None)]
struct ACVMCli {
#[command(subcommand)]
command: ACVMCommand,
}

#[non_exhaustive]
#[derive(Subcommand, Clone, Debug)]
enum ACVMCommand {
Execute(execute_cmd::ExecuteCommand),
}

#[cfg(not(feature = "codegen-docs"))]
pub(crate) fn start_cli() -> eyre::Result<()> {
let ACVMCli { command } = ACVMCli::parse();

match command {
ACVMCommand::Execute(args) => execute_cmd::run(args),
}?;

Ok(())
}

#[cfg(feature = "codegen-docs")]
pub(crate) fn start_cli() -> eyre::Result<()> {
let markdown: String = clap_markdown::help_markdown::<NargoCli>();
println!("{markdown}");
Ok(())
}
Loading
Loading