Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add noirc_abi_wasm crate for ABI encoding in JS #1945

Merged
merged 49 commits into from
Sep 5, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
e611498
chore: temp commit
TomAFrench Jul 17, 2023
90be749
chore: update package.json
TomAFrench Jul 18, 2023
901314d
fix: package name and paths for build.sh
kobyhallx Jul 18, 2023
e2775a4
feat: add another package to flake
kobyhallx Jul 18, 2023
38b7569
fix: shell env to 1.66.1
kobyhallx Jul 18, 2023
c468e17
chore: refactor scripts
TomAFrench Jul 18, 2023
8d615e4
chore: add test framework for new abi package
TomAFrench Jul 18, 2023
04ac605
chore: remove unused dependencies
TomAFrench Jul 19, 2023
0bc9944
chore: run abi encoding tests in CI
TomAFrench Jul 19, 2023
cbcca58
chore: fix typo in new workflow
TomAFrench Jul 19, 2023
b1b218f
chore: allow running `yarn install` in a non-root directory
TomAFrench Jul 19, 2023
6f4b098
chore: standardise triggers for workflow
TomAFrench Jul 19, 2023
616530f
chore: fix action yaml
TomAFrench Jul 19, 2023
91f46c2
chore: fix inputs
TomAFrench Jul 19, 2023
6063d64
chore: move cd to same step as yarn install
TomAFrench Jul 19, 2023
d7579c6
feat: add devcontainer config
kobyhallx Jul 19, 2023
4432308
Merge branch 'abi-wasm' of github.com:noir-lang/noir into abi-wasm
kobyhallx Jul 19, 2023
0ccf055
Merge branch 'master' into abi-wasm
TomAFrench Aug 1, 2023
285ee1a
Merge branch 'master' into abi-wasm
TomAFrench Aug 10, 2023
6df8af1
chore: pull across build process from `noir_wasm`
TomAFrench Aug 10, 2023
197c961
chore: test on firefox
TomAFrench Aug 10, 2023
b90b84f
chore: fix typo in README
TomAFrench Aug 10, 2023
075fde8
Merge branch 'master' into abi-wasm
TomAFrench Aug 15, 2023
4182a68
chore: update `noirc_abi_wasm` build scripts
TomAFrench Aug 15, 2023
f261c73
chore: fix merge
TomAFrench Aug 15, 2023
f3b28c0
chore: Cargo.toml tweaks
TomAFrench Aug 15, 2023
75d2137
chore: Fix typo (#2315)
kevaundray Aug 15, 2023
48b20fa
chore: Make `wasm` tests pull from `result` directory (#2319)
TomAFrench Aug 15, 2023
2684b56
chore: mirror `wasm` build scripts
TomAFrench Aug 15, 2023
9ed9fd8
chore: remove `.devcontainer.json`
TomAFrench Aug 15, 2023
311dc99
chore: fix build command
TomAFrench Aug 15, 2023
1ca0ce2
Merge branch 'master' into abi-wasm
TomAFrench Aug 15, 2023
c1d8623
chore: update package name in build script
TomAFrench Aug 15, 2023
d8b8c93
chore: revert change to wasm build script
TomAFrench Aug 15, 2023
e0edbd6
chore: fix package name
TomAFrench Aug 15, 2023
c970b4f
Merge branch 'master' into abi-wasm
TomAFrench Aug 17, 2023
a71e5a7
chore: revert changes to flake.nix
TomAFrench Aug 17, 2023
ec855ff
chore: fix caching of `noirc_abi_wasm`
TomAFrench Aug 17, 2023
1bfab41
chore(build): add wasm-opt
kobyhallx Sep 1, 2023
778af65
wip: getting abiEncoder to compile
kobyhallx Sep 4, 2023
41720c6
Merge branch 'master' into abi-wasm
kobyhallx Sep 5, 2023
0f87c5c
chore: adjust package paths
kobyhallx Sep 5, 2023
dc8324a
Merge branch 'master' into abi-wasm
kobyhallx Sep 5, 2023
bc21e14
fix: cache issue
kobyhallx Sep 5, 2023
43cc363
chore: bump nix pkgs version
kobyhallx Sep 5, 2023
b005033
Update crates/noirc_abi_wasm/test/browser/abi_encode.test.ts
kobyhallx Sep 5, 2023
05e86ef
chore: resolving nits
kobyhallx Sep 5, 2023
3eaea24
Merge branch 'abi-wasm' of github.com:noir-lang/noir into abi-wasm
kobyhallx Sep 5, 2023
1182be1
chore: retain artifact for 10d instead of 3d
kobyhallx Sep 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ members = [
"crates/fm",
"crates/arena",
"crates/noirc_abi",
"crates/noirc_abi_wasm",
"crates/iter-extended",
"crates/wasm",
]
Expand Down
Empty file.
34 changes: 34 additions & 0 deletions crates/noirc_abi_wasm/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
[package]
name = "noirc_abi_wasm"
version.workspace = true
authors.workspace = true
edition.workspace = true

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html


[lib]
crate-type = ["cdylib"]

[dependencies]
acvm.workspace = true
noirc_abi.workspace = true
iter-extended.workspace = true
wasm-bindgen.workspace = true
serde.workspace = true

js-sys = "0.3.62"
log = "0.4.17"
wasm-logger = "0.2.0"
console_error_panic_hook = "0.1.7"
gloo-utils = { version = "0.1", features = ["serde"] }

# This is an unused dependency, we are adding it
# so that we can enable the js feature in getrandom.
getrandom = { version = "*", features = ["js"] }
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved

[build-dependencies]
build-data = "0.1.3"

[dev-dependencies]
wasm-bindgen-test = "0.3.36"
17 changes: 17 additions & 0 deletions crates/noirc_abi_wasm/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Noir Lang ABi JavaScript Package

This JavaScript package enables users to ABI encode inputs to a Noir program, i.e. generating an initial witness.

## Building from source

Outside of the [noir repo](https://github.com/noir-lang/noir), this package can be built using the command below:

```bash
nix build -L github:noir-lang/noir/master#abi_wasm
```

If you are within the noir repo and would like to build local changes, you can use:

```bash
nix build -L #abi_wasm
```
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
14 changes: 14 additions & 0 deletions crates/noirc_abi_wasm/build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
const GIT_COMMIT: &&str = &"GIT_COMMIT";

fn main() {
// Only use build_data if the environment variable isn't set
// The environment variable is always set when working via Nix
if std::env::var(GIT_COMMIT).is_err() {
build_data::set_GIT_COMMIT();
build_data::set_GIT_DIRTY();
build_data::no_debug_rebuilds();
}

build_data::set_SOURCE_TIMESTAMP();
build_data::no_debug_rebuilds();
}
22 changes: 22 additions & 0 deletions crates/noirc_abi_wasm/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/env bash

function require_command {
if ! command -v "$1" >/dev/null 2>&1; then
echo "Error: $1 is required but not installed." >&2
exit 1
fi
}

require_command toml2json
require_command jq
require_command cargo
require_command wasm-bindgen
require_command wasm-opt

export pname=$(toml2json < Cargo.toml | jq -r .package.name)

./preBuild.sh
cargo build --lib --release --package noirc_abi_wasm --target wasm32-unknown-unknown
./postBuild.sh
./installPhase.sh

9 changes: 9 additions & 0 deletions crates/noirc_abi_wasm/installPhase.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/usr/bin/env bash

mkdir -p $out
cp ./crates/wasm/README.md $out/
cp ./crates/wasm/package.json $out/
cp -r ./pkg/* $out/

echo "## Tracking" >> $out/README.md
echo "Built from [noir-lang/noir@$GIT_COMMIT](https://github.com/noir-lang/noir/tree/$GIT_COMMIT)" >> $out/README.md
23 changes: 23 additions & 0 deletions crates/noirc_abi_wasm/package.json
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"name": "@noir-lang/noir_wasm",
"collaborators": [
"The Noir Team <team@noir-lang.org>"
],
"version": "0.8.0",
"files": [
"nodejs",
"web",
"package.json"
],
"main": "./nodejs/noir_wasm.js",
"types": "./web/noir_wasm.d.ts",
"module": "./web/noir_wasm.js",
"sideEffects": false,
"peerDependencies": {
"@noir-lang/noir-source-resolver": "1.1.2"
},
"repository": {
"type": "git",
"url": "https://github.com/noir-lang/noir_wasm.git"
}
}
11 changes: 11 additions & 0 deletions crates/noirc_abi_wasm/postBuild.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env bash

# TODO: Handle the wasm target being built in release mode
WASM_BINARY=./target/wasm32-unknown-unknown/release/${pname}.wasm
NODE_WASM=./pkg/nodejs/${pname}_bg.wasm
BROWSER_WASM=./pkg/web/${pname}_bg.wasm

wasm-bindgen $WASM_BINARY --out-dir ./pkg/nodejs --typescript --target nodejs
wasm-bindgen $WASM_BINARY --out-dir ./pkg/web --typescript --target web
wasm-opt $NODE_WASM -o $NODE_WASM -O
wasm-opt $BROWSER_WASM -o $BROWSER_WASM -O
6 changes: 6 additions & 0 deletions crates/noirc_abi_wasm/preBuild.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash

# Clear out the existing build artifacts as these aren't automatically removed by wasm-pack.
if [ -d ./pkg/ ]; then
rm -rf ./pkg/
fi
112 changes: 112 additions & 0 deletions crates/noirc_abi_wasm/src/js_witness_map.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
use acvm::{
acir::native_types::{Witness, WitnessMap},
FieldElement,
};
use js_sys::{JsString, Map};
use wasm_bindgen::prelude::{wasm_bindgen, JsValue};

#[wasm_bindgen(typescript_custom_section)]
const WITNESS_MAP: &'static str = r#"
// Map from witness index to hex string value of witness.
export type WitnessMap = Map<number, string>;
"#;

// WitnessMap
#[wasm_bindgen]
extern "C" {
#[wasm_bindgen(extends = Map, js_name = "WitnessMap", typescript_type = "WitnessMap")]
#[derive(Clone, Debug, PartialEq, Eq)]
pub type JsWitnessMap;

#[wasm_bindgen(constructor, js_class = "Map")]
pub fn new() -> JsWitnessMap;

}

impl Default for JsWitnessMap {
fn default() -> Self {
Self::new()
}
}

impl From<WitnessMap> for JsWitnessMap {
fn from(witness_map: WitnessMap) -> Self {
let js_map = JsWitnessMap::new();
for (key, value) in witness_map {
js_map.set(
&js_sys::Number::from(key.witness_index()),
&field_element_to_js_string(&value),
);
}
js_map
}
}

impl From<JsWitnessMap> for WitnessMap {
fn from(js_map: JsWitnessMap) -> Self {
let mut witness_map = WitnessMap::new();
js_map.for_each(&mut |value, key| {
let witness_index = Witness(key.as_f64().unwrap() as u32);
let witness_value = js_value_to_field_element(value).unwrap();
witness_map.insert(witness_index, witness_value);
});
witness_map
}
}

pub(crate) fn js_value_to_field_element(js_value: JsValue) -> Result<FieldElement, JsString> {
let hex_str = js_value.as_string().ok_or("failed to parse field element from non-string")?;

FieldElement::from_hex(&hex_str)
.ok_or_else(|| format!("Invalid hex string: '{}'", hex_str).into())
}

pub(crate) fn field_element_to_js_string(field_element: &FieldElement) -> JsString {
// This currently maps `0x30644e72e131a029b85045b68181585d2833e84879b9709143e1f593f0000000`
// to the bigint `-1n`. This fails when converting back to a `FieldElement`.
// js_sys::BigInt::from_str(&value.to_hex()).unwrap()

format!("0x{}", field_element.to_hex()).into()
}

#[cfg(test)]
mod test {
use wasm_bindgen_test::wasm_bindgen_test as test;

use std::collections::BTreeMap;

use acvm::{
acir::native_types::{Witness, WitnessMap},
FieldElement,
};
use wasm_bindgen::JsValue;

use crate::JsWitnessMap;

#[test]
fn test_witness_map_to_js() {
let witness_map = BTreeMap::from([
(Witness(1), FieldElement::one()),
(Witness(2), FieldElement::zero()),
(Witness(3), -FieldElement::one()),
]);
let witness_map = WitnessMap::from(witness_map);

let js_map = JsWitnessMap::from(witness_map);

assert_eq!(
js_map.get(&JsValue::from(1)),
JsValue::from_str("0x0000000000000000000000000000000000000000000000000000000000000001")
);
assert_eq!(
js_map.get(&JsValue::from(2)),
JsValue::from_str("0x0000000000000000000000000000000000000000000000000000000000000000")
);
assert_eq!(
js_map.get(&JsValue::from(3)),
// Equal to 21888242871839275222246405745257275088548364400416034343698204186575808495616,
// which is field modulus - 1: https://docs.rs/ark-bn254/latest/ark_bn254/
JsValue::from_str("0x30644e72e131a029b85045b68181585d2833e84879b9709143e1f593f0000000")
);
}
}
Loading