Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add dependency resolver for noir_wasm and implement FileManager for consistency with native interface #3891

Merged
merged 53 commits into from
Jan 11, 2024

Conversation

Thunkar
Copy link
Contributor

@Thunkar Thunkar commented Dec 20, 2023

Description

Merging the work done here:

AztecProtocol/aztec-packages#3696
#3781
#3760

Plus some extras to make the API nicer.

Problem

Closes(?) #3695

Summary

Makes noir_wasm easier to work with, including dependency resolution and bundling. This package can be used from both node and the browser with identical API leveraging a virtual filesystem.

Uses webpack for bundling, which is done in two steps:

  1. rust -> wasm (cjs/esm)
  2. TS + wasm (cjs/esm) -> universal package for web

Tests have been migrated to mocha and playwright.

Additional Context

I really want to test it here before merging, but it's in a state in which it can be reviewed before we commit to an API.
Done: signorecello/noir-playground#32

Even though the initial memFS-backed FileManager developed by @alexghr is still here, it is not used for the web version due to import problems. The way it works now, webpack uses memfs directly it to alias the node fs module (which seems to be its intended use case) and allows us to use the nodejs fs API everywhere.

Documentation

Documentation is required for usage, but should basically be:

// Node.js

import { compile, createFileManager } from '@noir-lang/noir_wasm'; // Rename!!

const fm = createFileManager(myProjectPath);
const myCompiledCode = await compile(fm);
// Browser

import { compile, createFileManager } from '@noir-lang/noir_wasm'; // Rename!!

const fm = createFileManager('/');
for (const path of files) {
  await fm.writeFile(path, await getFileAsStream(path));
}
const myCompiledCode = await compile(fm);

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [Exceptional Case] Documentation to be submitted in a separate PR.

PR Checklist

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

@Thunkar Thunkar changed the title (feat): noir_wasm revamped: deps, cjs, esm, bundled, nice API feat: noir_wasm revamped: deps, cjs, esm, bundled, nice API Dec 20, 2023
@Thunkar Thunkar changed the title feat: noir_wasm revamped: deps, cjs, esm, bundled, nice API feat: noir_wasm revamped: deps, cjs, esm, bundled, nicer API Dec 20, 2023
compiler/wasm/package.json Outdated Show resolved Hide resolved
compiler/wasm/package.json Show resolved Hide resolved
compiler/wasm/src/types/abi.ts Outdated Show resolved Hide resolved
Copy link

socket-security bot commented Dec 21, 2023

@Thunkar Thunkar marked this pull request as ready for review December 21, 2023 17:00
Copy link
Member

@TomAFrench TomAFrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small nits. Will do a final review on the actual core and approve later today/tomorrow.

.github/workflows/test-js-packages.yml Outdated Show resolved Hide resolved
compiler/integration-tests/scripts/codegen-verifiers.sh Outdated Show resolved Hide resolved
compiler/wasm/build-fixtures.sh Outdated Show resolved Hide resolved
compiler/wasm/package.json Show resolved Hide resolved
* master:
  chore(ci): disable commit comments on docs publishing (#3998)
  feat: simplify chains of casts to be all in terms of the original `ValueId` (#3984)
  chore: quiet down aztec-bot! (#4006)
  chore: force public functions for trait implementation (#3986)
  feat: docker testing flow (#3895)
  feat: decompose `Instruction::Constrain` into multiple more basic constraints (#3892)
  feat: remove truncation from brillig casts (#3997)
  chore: add test case showing reexports (#4001)
  feat: remove truncations which can be seen to be noops using type information (#3953)
  chore: remove homemade keccakf1600 implementation (#3996)
@TomAFrench TomAFrench changed the title feat: noir_wasm revamped: deps, cjs, esm, bundled, nicer API feat: Add dependency resolver for noir_wasm and implement FileManager for consistency with native interface Jan 11, 2024
Copy link
Member

@TomAFrench TomAFrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Thunkar Thunkar added this pull request to the merge queue Jan 11, 2024
Merged via the queue into master with commit c29c7d7 Jan 11, 2024
32 checks passed
@Thunkar Thunkar deleted the gj/noir_wasm branch January 11, 2024 12:40
TomAFrench added a commit that referenced this pull request Jan 11, 2024
* master:
  feat: Add dependency resolver for `noir_wasm` and implement `FileManager` for consistency with native interface (#3891)
TomAFrench added a commit that referenced this pull request Jan 11, 2024
* master:
  feat: Add dependency resolver for `noir_wasm` and implement `FileManager` for consistency with native interface (#3891)
  chore(docs): devcontainer docs (#4007)
@TomAFrench TomAFrench mentioned this pull request Jan 11, 2024
5 tasks
TomAFrench added a commit that referenced this pull request Jan 12, 2024
* master: (285 commits)
  feat: Add `assert_max_bit_size` method to `Field` (#4016)
  fix: fixup exports from `noir_wasm` (#4022)
  chore(ci): fix docker CI (#4014)
  feat(lsp): goto trait method declaration (#3991)
  chore: add action to check for dead links (#3992)
  chore: improve cspell setup (#3985)
  feat: Add dependency resolver for `noir_wasm` and implement `FileManager` for consistency with native interface (#3891)
  chore(docs): devcontainer docs (#4007)
  chore(ci): disable commit comments on docs publishing (#3998)
  feat: simplify chains of casts to be all in terms of the original `ValueId` (#3984)
  chore: quiet down aztec-bot! (#4006)
  chore: force public functions for trait implementation (#3986)
  feat: docker testing flow (#3895)
  feat: decompose `Instruction::Constrain` into multiple more basic constraints (#3892)
  feat: remove truncation from brillig casts (#3997)
  chore: add test case showing reexports (#4001)
  feat: remove truncations which can be seen to be noops using type information (#3953)
  chore: remove homemade keccakf1600 implementation (#3996)
  feat!: Breaking changes from aztec-packages (#3955)
  chore(docs): small grammar fix on the landing page (#3989)
  ...
github-merge-queue bot pushed a commit that referenced this pull request Jan 22, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>0.23.0</summary>

## [0.23.0](v0.22.0...v0.23.0)
(2024-01-22)


### ⚠ BREAKING CHANGES

* Ban nested slices
([#4018](#4018))
* Breaking changes from aztec-packages
([#3955](#3955))
* Rename Arithmetic opcode to AssertZero
([#3840](#3840))
* remove circuit methods from noir_wasm
([#3869](#3869))

### Features

* Add `assert_max_bit_size` method to `Field`
([#4016](#4016))
([bc9a44f](bc9a44f))
* Add `noir-compiler` checks to `aztec_macros`
([#4031](#4031))
([420a5c7](420a5c7))
* Add a `--force` flag to force a full recompile
([#4054](#4054))
([27a8e68](27a8e68))
* Add dependency resolver for `noir_wasm` and implement `FileManager`
for consistency with native interface
([#3891](#3891))
([c29c7d7](c29c7d7))
* Add foreign call support to `noir_codegen` functions
([#3933](#3933))
([e5e52a8](e5e52a8))
* Add MVP `nargo export` command
([#3870](#3870))
([fbb51ed](fbb51ed))
* Add support for codegenning multiple functions which use the same
structs in their interface
([#3868](#3868))
([1dcfcc5](1dcfcc5))
* Added efficient field comparisons for bn254
([#4042](#4042))
([1f9cad0](1f9cad0))
* Assert maximum bit size when creating a U128 from an integer
([#4024](#4024))
([8f9c7e4](8f9c7e4))
* Avoid unnecessary range checks by inspecting instructions for casts
([#4039](#4039))
([378c18e](378c18e))
* Breaking changes from aztec-packages
([#3955](#3955))
([5be049e](5be049e))
* Bubble up `Instruction::Constrain`s to be applied as early as
possible. ([#4065](#4065))
([66f5cdd](66f5cdd))
* Cached LSP parsing
([#4083](#4083))
([b4f724e](b4f724e))
* Comparison for signed integers
([#3873](#3873))
([bcbd49b](bcbd49b))
* Decompose `Instruction::Cast` to have an explicit truncation
instruction ([#3946](#3946))
([35f18ef](35f18ef))
* Decompose `Instruction::Constrain` into multiple more basic
constraints ([#3892](#3892))
([51cf9d3](51cf9d3))
* Docker testing flow
([#3895](#3895))
([179c90d](179c90d))
* Extract parsing to its own pass and do it in parallel
([#4063](#4063))
([569cbbc](569cbbc))
* Implement `Eq` trait on curve points
([#3944](#3944))
([abf751a](abf751a))
* Implement DAP protocol in Nargo
([#3627](#3627))
([13834d4](13834d4))
* Implement generic traits
([#4000](#4000))
([916fd15](916fd15))
* Implement Operator Overloading
([#3931](#3931))
([4b16090](4b16090))
* **lsp:** Cache definitions for goto requests
([#3930](#3930))
([4a2140f](4a2140f))
* **lsp:** Goto global
([#4043](#4043))
([15237b3](15237b3))
* **lsp:** Goto struct member inside Impl method
([#3918](#3918))
([99c2c5a](99c2c5a))
* **lsp:** Goto trait from trait impl
([#3956](#3956))
([eb566e2](eb566e2))
* **lsp:** Goto trait method declaration
([#3991](#3991))
([eb79166](eb79166))
* **lsp:** Goto type alias
([#4061](#4061))
([dc83385](dc83385))
* **lsp:** Goto type definition
([#4029](#4029))
([8bb4ddf](8bb4ddf))
* **lsp:** Re-add code lens feature with improved performance
([#3829](#3829))
([8f5cd6c](8f5cd6c))
* Optimize array ops for arrays of structs
([#4027](#4027))
([c9ec0d8](c9ec0d8))
* Optimize logic gate ACIR-gen
([#3897](#3897))
([926460a](926460a))
* Prefer `AcirContext`-native methods for performing logic operations
([#3898](#3898))
([0ec39b8](0ec39b8))
* Remove range constraints from witnesses which are constrained to be
constants ([#3928](#3928))
([afe9c7a](afe9c7a))
* Remove truncation from brillig casts
([#3997](#3997))
([857ff97](857ff97))
* Remove truncations which can be seen to be noops using type
information ([#3953](#3953))
([cc3c2c2](cc3c2c2))
* Remove unnecessary predicate from `Lt` instruction
([#3922](#3922))
([a63433f](a63433f))
* Simplify chains of casts to be all in terms of the original `ValueId`
([#3984](#3984))
([2384d3e](2384d3e))
* Simplify multiplications by `0` or `1` in ACIR gen
([#3924](#3924))
([e58844d](e58844d))
* Support for u128
([#3913](#3913))
([b4911dc](b4911dc))
* Support printing more types
([#4071](#4071))
([f5c4632](f5c4632))
* Sync `aztec-packages`
([#4011](#4011))
([fee2452](fee2452))
* Sync commits from `aztec-packages`
([#4068](#4068))
([7a8f3a3](7a8f3a3))
* Use singleton `WasmBlackBoxFunctionSolver` in `noir_js`
([#3966](#3966))
([10b28de](10b28de))


### Bug Fixes

* Acir gen doesn't panic on unsupported BB function
([#3866](#3866))
([34fd978](34fd978))
* Allow abi encoding arrays of structs from JS
([#3867](#3867))
([9b713f8](9b713f8))
* Allow abi encoding tuples from JS
([#3894](#3894))
([f7fa181](f7fa181))
* Allow ast when macro errors
([#4005](#4005))
([efccec3](efccec3))
* Allow lsp to run inside of a docker container
([#3876](#3876))
([2529977](2529977))
* Bit-shifts for signed integers
([#3890](#3890))
([6ddd98a](6ddd98a))
* Checks for cyclic dependencies
([#3699](#3699))
([642011a](642011a))
* **debugger:** Crash when stepping through locations spanning multiple
lines ([#3920](#3920))
([223e860](223e860))
* Don't fail if no tests and the user didn't provide a pattern
([#3864](#3864))
([decbd0f](decbd0f))
* Fix advisory issue in cargo-deny
([#4077](#4077))
([19baea0](19baea0))
* Fixing dark mode background on the CTA button
([#3882](#3882))
([57eae42](57eae42))
* Fixup exports from `noir_wasm`
([#4022](#4022))
([358cdd2](358cdd2))
* Handle multiple imports in the same file
([#3903](#3903))
([219423e](219423e))
* Hoist constraints on inputs to top of program
([#4076](#4076))
([447aa34](447aa34))
* Implement missing codegen for `BlackBoxFunc::EcdsaSecp256r1` in
brillig ([#3943](#3943))
([2c5eceb](2c5eceb))
* Improve `nargo test` output
([#3973](#3973))
([3ab5ff4](3ab5ff4))
* Make `constant_to_radix` emit a slice instead of an array
([#4049](#4049))
([5cdb1d0](5cdb1d0))
* Operator overloading & static trait method references resolving to
generic impls ([#3967](#3967))
([f1de8fa](f1de8fa))
* Preserve brillig entrypoint functions without arguments
([#3951](#3951))
([1111465](1111465))
* Prevent `Instruction::Constrain`s for non-primitive types
([#3916](#3916))
([467948f](467948f))
* Remove panic for adding an invalid crate name in wasm compiler
([#3977](#3977))
([7a1baa5](7a1baa5))
* Return error rather instead of panicking on invalid circuit
([#3976](#3976))
([67201bf](67201bf))
* Search all levels of struct nesting before codegenning primitive types
([#3970](#3970))
([13ae014](13ae014))
* Update generics docs to mention we have traits now
([#3980](#3980))
([c2acdf1](c2acdf1))


### Miscellaneous Chores

* Ban nested slices
([#4018](#4018))
([f8a1fb7](f8a1fb7))
* Remove circuit methods from noir_wasm
([#3869](#3869))
([12d884e](12d884e))
* Rename Arithmetic opcode to AssertZero
([#3840](#3840))
([836f171](836f171))
</details>

<details><summary>0.39.0</summary>

## [0.39.0](v0.38.0...v0.39.0)
(2024-01-22)


### ⚠ BREAKING CHANGES

* Breaking changes from aztec-packages
([#3955](#3955))
* Rename Arithmetic opcode to AssertZero
([#3840](#3840))
* Remove unused methods on ACIR opcodes
([#3841](#3841))
* Remove partial backend feature
([#3805](#3805))

### Features

* Aztec-packages
([#3754](#3754))
([c043265](c043265))
* Breaking changes from aztec-packages
([#3955](#3955))
([5be049e](5be049e))
* Remove range constraints from witnesses which are constrained to be
constants ([#3928](#3928))
([afe9c7a](afe9c7a))
* Speed up transformation of debug messages
([#3815](#3815))
([2a8af1e](2a8af1e))
* Sync `aztec-packages`
([#4011](#4011))
([fee2452](fee2452))
* Sync commits from `aztec-packages`
([#4068](#4068))
([7a8f3a3](7a8f3a3))


### Bug Fixes

* Deserialize odd length hex literals
([#3747](#3747))
([4000fb2](4000fb2))
* Return error rather instead of panicking on invalid circuit
([#3976](#3976))
([67201bf](67201bf))


### Miscellaneous Chores

* Remove partial backend feature
([#3805](#3805))
([0383100](0383100))
* Remove unused methods on ACIR opcodes
([#3841](#3841))
([9e5d0e8](9e5d0e8))
* Rename Arithmetic opcode to AssertZero
([#3840](#3840))
([836f171](836f171))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: TomAFrench <tom@tomfren.ch>
@Savio-Sou Savio-Sou mentioned this pull request Jan 23, 2024
github-merge-queue bot pushed a commit that referenced this pull request Jan 25, 2024
# Description

## Problem\*

Resolves #4157

## Summary\*

#3891 didn't update release flow
so it's still trying to build `noir_wasm` using nix. This PR updates to
use the new build process.

## Additional Context



## Documentation\*

Check one:
- [x] No documentation needed.
- [ ] Documentation included in this PR.
- [ ] **[Exceptional Case]** Documentation to be submitted in a separate
PR.

# PR Checklist\*

- [x] I have tested the changes locally.
- [x] I have formatted the changes with [Prettier](https://prettier.io/)
and/or `cargo fmt` on default settings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants