Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: turn on elaborator #7451

Merged
merged 17 commits into from
Jul 16, 2024
Merged

chore: turn on elaborator #7451

merged 17 commits into from
Jul 16, 2024

Conversation

TomAFrench
Copy link
Member

This PR turns off usage of the legacy name resolution/typechecking code to instead use the new elaborator.

I'm seeing panics when performing keccak hashes however.

@nventuro
Copy link
Contributor

nventuro commented Jul 11, 2024

We get some build errors in a couple contracts, this seems to be related to the macros that deal with enqueuing calls.

@TomAFrench TomAFrench changed the title chore: turn off elaborator chore: turn on elaborator Jul 12, 2024
@Thunkar
Copy link
Contributor

Thunkar commented Jul 12, 2024

I think I found out the issue: it's not the enqueue calls, but the way we're injecting the calculated function signatures after the HIR pass.

Since we don't have the typedchecked AST when generating the contract interfaces, we leave a placeholder to be replaced during the HIR pass (example taken from crowdfunding_contract.

 pub fn _check_deadline(self) -> dep::aztec::context::PublicStaticVoidCallInterface<15,()> {
            let mut args_acc: [Field] = &[];

            let selector = dep::aztec::protocol_types::abis::function_selector::FunctionSelector::from_signature("SELECTOR_PLACEHOLDER");
            dep::aztec::context::PublicStaticVoidCallInterface {
                target_contract: self.target_contract,
                selector,
                name: "_check_deadline",
                args: args_acc,
                original: | inputs: dep::aztec::context::inputs::PublicContextInputs | -> () {
                    _check_deadline(inputs)
                },
                is_static: true,
                gas_opts: dep::aztec::context::gas::GasOpts::default()
            }
        }

In this case, "SELECTOR_PLACEHOLDER" would be replaced by "_check_deadline()". The compiler is crashing here because "SELECTOR_PLACEHOLDER" is 20 characters long and "_check_deadline()" is 17, which is generating an incorrect range check in the SSA:

Screenshot 2024-07-12 at 16 45 14

I've changed the placeholder to be an empty string and it seems to work, but would love @jfecher's input on this.

@AztecBot
Copy link
Collaborator

AztecBot commented Jul 12, 2024

Docs Preview

Hey there! 👋 You can check your preview at https://66967a58fb4c60ad2ffc6551--aztec-docs-dev.netlify.app

@Thunkar
Copy link
Contributor

Thunkar commented Jul 12, 2024

While at it, added a better error message for noir-lang/noir#5384

@TomAFrench I don't think it's worth it investing in a better solution, since that generic is going away first and then the whole macros folder (hopefully soon) after.

@jfecher
Copy link
Contributor

jfecher commented Jul 12, 2024

I've changed the placeholder to be an empty string and it seems to work, but would love @jfecher's input on this.

The panic you're getting looks to be a result of simplifying a constant call to Keccak256. When simplifying that call in the compiler it looks like we have no checks at all that the second argument (message_size) is actually in bounds of the first argument (the array). This is surely good code.

For now, maybe check your calls to keccak256?

The compiler is crashing here because "SELECTOR_PLACEHOLDER" is 20 characters long and "_check_deadline()" is 17

I'm assuming this was just speculation? Since the panic location in SSA in the code doesn't deal with names at all.

No idea how this would be triggered or affected by the SELECTOR_PLACEHOLDER / _check_deadline check though.

Also, surprise code review:

original: | inputs: dep::aztec::context::inputs::PublicContextInputs | -> () {
                    _check_deadline(inputs)
                },

This should be able to be simplified to:

original: _check_deadline,

@jfecher
Copy link
Contributor

jfecher commented Jul 12, 2024

Another thing to note is that in the elaborator the aztec_macros call on the typed AST occurs after type checking where as in the legacy code it occurred before type checking. I'm hoping this won't be an issue but am assuming it will be.

@Thunkar
Copy link
Contributor

Thunkar commented Jul 12, 2024

I clearly didn't do a good job of explaining the issue and the fix 😅

Those keccak256 calls are done when generating function selectors from signatures (which are strings). We are essentially "tricking" the compiler, giving it a particular string and then replacing its value with another one after the HIR pass, which seems to confuse it (the bytes.len() call is somehow not "updated" with the new string length) which permeates all the way to the SSA generating invalid range checks. This seems to be in line with what you said:

Another thing to note is that in the elaborator the aztec_macros call on the typed AST occurs after type checking where as in the legacy code it occurred before type checking. I'm hoping this won't be an issue but am assuming it will be.

Could it be possible that the as_bytes string builtin gets executed before we perform the swap and therefore the length of the string passed to keccak256 is not updated?

Might be completely off base though.

This should be able to be simplified to:

original: _check_deadline,

Unfortunately this is a very simple case of autogenerated code with no args in public. More complex ones required the verbose version 😢

@jfecher
Copy link
Contributor

jfecher commented Jul 12, 2024

Could it be possible that the as_bytes string builtin gets executed before we perform the swap and therefore the length of the string passed to keccak256 is not updated?

I don't think so. The macros are executed before monomorphization which is before SSA where any calls to keccak256 are possibly optimized. It's possible the change isn't being picked up for some reason.

Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
@Thunkar
Copy link
Contributor

Thunkar commented Jul 15, 2024

Sorry @TomAFrench to be hijacking your PR, but we really need this ASAP so we don't fall behind. Currently investigating some hash mismatches in the e2e tests, really hoping for CI issue, but doesn't look like it.

@TomAFrench
Copy link
Member Author

Hey no worries, this isn't hijacking at all. I opened this just to track what errors were being emitted so they can be fixed so I'm expecting other people to jump in.

@Thunkar
Copy link
Contributor

Thunkar commented Jul 16, 2024

After looking into the issue, it seems like one of the interner APIs we use in the macros (context.def_interner.update_expression) was generating problems during monomorphization and the wrong version of the FunctionSelector::from_signature function was being selected. I have no idea why and it looks very non trivial to fix...so I didn't, and looked for a workaround instead (we're gonna kill this code soon™️ anyways)

Since updating expressions doesn't seem to generate any problems in cases where we don't have to deal with generics at all, I decided to use the same approach as in note_interface.nr and generate the selector "offline". This is actually better in terms of constraints and really the way we will hopefully do it when we migrate to metaprogramming, since those values are known at compile time and doesn't make sense to recompute and constrain them.

@@ -73,14 +73,14 @@ describe('e2e_blacklist_token_contract access control', () => {
const newRole = new Role().withAdmin().withAdmin();
await expect(
t.asset.withWallet(t.other).methods.update_roles(AztecAddress.random(), newRole.toNoirStruct()).prove(),
).rejects.toThrow("Assertion failed: caller is not admin 'caller_roles.is_admin'");
).rejects.toThrow(/Assertion failed: caller is not admin .*/);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes are due to this bug: noir-lang/noir#5523

It's the only difference in behavior (besides macro fuckery) we've found

Copy link
Contributor

@sklppy88 sklppy88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice !

@nventuro nventuro merged commit 0599500 into master Jul 16, 2024
92 checks passed
@nventuro nventuro deleted the tf/use-elaborator branch July 16, 2024 15:46
rahul-kothari pushed a commit that referenced this pull request Jul 16, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.46.7</summary>

##
[0.46.7](aztec-package-v0.46.6...aztec-package-v0.46.7)
(2024-07-16)


### Features

* Devnet updates
([#7421](#7421))
([103f099](103f099))


### Bug Fixes

* Cli l1-chain-id option
([#7490](#7490))
([307bc57](307bc57))


### Miscellaneous

* Turn on elaborator
([#7451](#7451))
([0599500](0599500))
</details>

<details><summary>barretenberg.js: 0.46.7</summary>

##
[0.46.7](barretenberg.js-v0.46.6...barretenberg.js-v0.46.7)
(2024-07-16)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.46.7</summary>

##
[0.46.7](aztec-packages-v0.46.6...aztec-packages-v0.46.7)
(2024-07-16)


### Features

* Add unconstrained context to txe
([#7448](#7448))
([699fb79](699fb79))
* Add unconstrained getters to sharedmutable
([#7429](#7429))
([c0ff566](c0ff566))
* Devnet updates
([#7421](#7421))
([103f099](103f099))
* Point::fromXandSign(...)
([#7455](#7455))
([225c6f6](225c6f6))


### Bug Fixes

* **avm:** Update generated verifier
([#7492](#7492))
([f1216a7](f1216a7))
* Cli l1-chain-id option
([#7490](#7490))
([307bc57](307bc57))
* Don't pass secrets to earthly-ci 'publish docs' command
([#7481](#7481))
([a3f6feb](a3f6feb))
* Fix msg_sender direct call exploit
([#7404](#7404))
([1dcae45](1dcae45))
* Missing NoteSelector from JSON RPC proxies
([#7493](#7493))
([b209fad](b209fad))
* **pxe:** Best effort noir call stack generation
([#7336](#7336))
([0c7459b](0c7459b))
* Validate gas used
([#7459](#7459))
([6dc7598](6dc7598))


### Miscellaneous

* **avm:** More stats and codegen cleanup
([#7475](#7475))
([1a6c7f2](1a6c7f2))
* Checking compute_encrypted_note_log against TS impl
([#7491](#7491))
([1e8a597](1e8a597))
* Included subrelation witness degrees in the relations relevant to
zk-sumcheck
([#7479](#7479))
([457a115](457a115))
* Replace relative paths to noir-protocol-circuits
([71960d4](71960d4))
* Turn on elaborator
([#7451](#7451))
([0599500](0599500))
</details>

<details><summary>barretenberg: 0.46.7</summary>

##
[0.46.7](barretenberg-v0.46.6...barretenberg-v0.46.7)
(2024-07-16)


### Features

* Point::fromXandSign(...)
([#7455](#7455))
([225c6f6](225c6f6))


### Bug Fixes

* **avm:** Update generated verifier
([#7492](#7492))
([f1216a7](f1216a7))


### Miscellaneous

* **avm:** More stats and codegen cleanup
([#7475](#7475))
([1a6c7f2](1a6c7f2))
* Included subrelation witness degrees in the relations relevant to
zk-sumcheck
([#7479](#7479))
([457a115](457a115))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
AztecBot added a commit to AztecProtocol/barretenberg that referenced this pull request Jul 17, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.46.7</summary>

##
[0.46.7](AztecProtocol/aztec-packages@aztec-package-v0.46.6...aztec-package-v0.46.7)
(2024-07-16)


### Features

* Devnet updates
([#7421](AztecProtocol/aztec-packages#7421))
([103f099](AztecProtocol/aztec-packages@103f099))


### Bug Fixes

* Cli l1-chain-id option
([#7490](AztecProtocol/aztec-packages#7490))
([307bc57](AztecProtocol/aztec-packages@307bc57))


### Miscellaneous

* Turn on elaborator
([#7451](AztecProtocol/aztec-packages#7451))
([0599500](AztecProtocol/aztec-packages@0599500))
</details>

<details><summary>barretenberg.js: 0.46.7</summary>

##
[0.46.7](AztecProtocol/aztec-packages@barretenberg.js-v0.46.6...barretenberg.js-v0.46.7)
(2024-07-16)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.46.7</summary>

##
[0.46.7](AztecProtocol/aztec-packages@aztec-packages-v0.46.6...aztec-packages-v0.46.7)
(2024-07-16)


### Features

* Add unconstrained context to txe
([#7448](AztecProtocol/aztec-packages#7448))
([699fb79](AztecProtocol/aztec-packages@699fb79))
* Add unconstrained getters to sharedmutable
([#7429](AztecProtocol/aztec-packages#7429))
([c0ff566](AztecProtocol/aztec-packages@c0ff566))
* Devnet updates
([#7421](AztecProtocol/aztec-packages#7421))
([103f099](AztecProtocol/aztec-packages@103f099))
* Point::fromXandSign(...)
([#7455](AztecProtocol/aztec-packages#7455))
([225c6f6](AztecProtocol/aztec-packages@225c6f6))


### Bug Fixes

* **avm:** Update generated verifier
([#7492](AztecProtocol/aztec-packages#7492))
([f1216a7](AztecProtocol/aztec-packages@f1216a7))
* Cli l1-chain-id option
([#7490](AztecProtocol/aztec-packages#7490))
([307bc57](AztecProtocol/aztec-packages@307bc57))
* Don't pass secrets to earthly-ci 'publish docs' command
([#7481](AztecProtocol/aztec-packages#7481))
([a3f6feb](AztecProtocol/aztec-packages@a3f6feb))
* Fix msg_sender direct call exploit
([#7404](AztecProtocol/aztec-packages#7404))
([1dcae45](AztecProtocol/aztec-packages@1dcae45))
* Missing NoteSelector from JSON RPC proxies
([#7493](AztecProtocol/aztec-packages#7493))
([b209fad](AztecProtocol/aztec-packages@b209fad))
* **pxe:** Best effort noir call stack generation
([#7336](AztecProtocol/aztec-packages#7336))
([0c7459b](AztecProtocol/aztec-packages@0c7459b))
* Validate gas used
([#7459](AztecProtocol/aztec-packages#7459))
([6dc7598](AztecProtocol/aztec-packages@6dc7598))


### Miscellaneous

* **avm:** More stats and codegen cleanup
([#7475](AztecProtocol/aztec-packages#7475))
([1a6c7f2](AztecProtocol/aztec-packages@1a6c7f2))
* Checking compute_encrypted_note_log against TS impl
([#7491](AztecProtocol/aztec-packages#7491))
([1e8a597](AztecProtocol/aztec-packages@1e8a597))
* Included subrelation witness degrees in the relations relevant to
zk-sumcheck
([#7479](AztecProtocol/aztec-packages#7479))
([457a115](AztecProtocol/aztec-packages@457a115))
* Replace relative paths to noir-protocol-circuits
([71960d4](AztecProtocol/aztec-packages@71960d4))
* Turn on elaborator
([#7451](AztecProtocol/aztec-packages#7451))
([0599500](AztecProtocol/aztec-packages@0599500))
</details>

<details><summary>barretenberg: 0.46.7</summary>

##
[0.46.7](AztecProtocol/aztec-packages@barretenberg-v0.46.6...barretenberg-v0.46.7)
(2024-07-16)


### Features

* Point::fromXandSign(...)
([#7455](AztecProtocol/aztec-packages#7455))
([225c6f6](AztecProtocol/aztec-packages@225c6f6))


### Bug Fixes

* **avm:** Update generated verifier
([#7492](AztecProtocol/aztec-packages#7492))
([f1216a7](AztecProtocol/aztec-packages@f1216a7))


### Miscellaneous

* **avm:** More stats and codegen cleanup
([#7475](AztecProtocol/aztec-packages#7475))
([1a6c7f2](AztecProtocol/aztec-packages@1a6c7f2))
* Included subrelation witness degrees in the relations relevant to
zk-sumcheck
([#7479](AztecProtocol/aztec-packages#7479))
([457a115](AztecProtocol/aztec-packages@457a115))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
github-merge-queue bot pushed a commit to noir-lang/noir that referenced this pull request Jul 30, 2024
Automated pull of Noir development from
[aztec-packages](https://github.com/AztecProtocol/aztec-packages).
BEGIN_COMMIT_OVERRIDE
feat: Sync from noir
(AztecProtocol/aztec-packages#7583)
feat: Sync from noir
(AztecProtocol/aztec-packages#7577)
feat: Make Brillig do integer arithmetic operations using u128 instead
of Bigint (AztecProtocol/aztec-packages#7518)
feat: Avoid heap allocs when going to/from field
(AztecProtocol/aztec-packages#7547)
fix: Revert "feat: Sync from noir
(AztecProtocol/aztec-packages#7512)"
(AztecProtocol/aztec-packages#7558)
feat: Sync from noir
(AztecProtocol/aztec-packages#7512)
feat: Sync from noir
(AztecProtocol/aztec-packages#7454)
chore: turn on elaborator
(AztecProtocol/aztec-packages#7451)
fix: Add trailing extra arguments for backend in gates_flamegraph
(AztecProtocol/aztec-packages#7472)
feat: Sync from noir
(AztecProtocol/aztec-packages#7444)
feat: Sync from noir
(AztecProtocol/aztec-packages#7432)
feat: Integrate new proving systems in e2e
(AztecProtocol/aztec-packages#6971)
feat: typing return values of embedded_curve_ops
(AztecProtocol/aztec-packages#7413)
END_COMMIT_OVERRIDE

---------

Co-authored-by: Maxim Vezenov <mvezenov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants