Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: proof surgery class #8236

Merged
merged 8 commits into from
Aug 28, 2024
Merged

feat: proof surgery class #8236

merged 8 commits into from
Aug 28, 2024

Conversation

ledwards2225
Copy link
Contributor

@ledwards2225 ledwards2225 commented Aug 27, 2024

Adds a ProofSurgeon class that manages all proof surgery, e.g. splitting public inputs out of proof for acir and reconstructing again for bberg. Simplifies things quite a bit in the process.

@ledwards2225 ledwards2225 self-assigned this Aug 27, 2024
// the proof and public_inputs we subtract and add the corresponding amount from the respective sizes.
size_t size_of_proof_with_no_pub_inputs = input.proof.size() - bb::AGGREGATION_OBJECT_SIZE;
size_t total_num_public_inputs = input.public_inputs.size() + bb::AGGREGATION_OBJECT_SIZE;
create_dummy_vkey_and_proof(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lucas you were right that the moving of the agg object from proof to pub inputs was needed in a sense but it was just because this method expects the "raw" sizes of each. Rather than actually move it back and forth I just provide this method the adjusted sizes. BTW I think the long term solution for this dummy proof is just to read in some data from a file or something. This is too brittle

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well we could just pass the unadjusted values to the create_dummy_vkey_and_proof function as well and handle it there

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually did it that way first but found this to be slightly clearer since its external to the method where we know about the structure of the proof/pub inputs. Either way its kind of bad but its better than it was. I'll hopefully have time to clean all of this up more soon

* @return RecursionWitnessData
*/
static RecursionWitnessData populate_recursion_witness_data(bb::SlabVector<bb::fr>& witness,
std::vector<bb::fr>& proof_witnesses,
Copy link
Contributor Author

@ledwards2225 ledwards2225 Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pretty substantial simplification of the old logic, driven in part by the fact that there's no reason (as far as I can tell) to place the data into witness in any special way, as long as the component indices are correct. So I just basically concatenate the key, proof and pub inputs into with witness without worrying about reinserting the pub inputs in the proof etc. This is also probably closer to how the witness looks coming from an actual noir program.

@ledwards2225 ledwards2225 marked this pull request as ready for review August 28, 2024 17:56
// Fourth key field is the whether the proof contains an aggregation object.
builder.assert_equal(builder.add_variable(1), key_fields[4].witness_index);
builder.assert_equal(builder.add_variable(1), key_fields[4].witness_index); // WORKTODO: 4 = bug?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also note this weird index that looks like a bug to me. If not then its definitely worth a comment explaining why

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm yeah, it should be 3...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why we wouldn't get some sort of failure if we assert_equal the 4th index with both 1 and 0 variables...?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, a little concerning. My vague guess was that the 4th-index value was always non-zero and was somehow just getting converted to "true" somewhere along the way. not sure

Copy link
Contributor

@lucasxia01 lucasxia01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good in general

// Fourth key field is the whether the proof contains an aggregation object.
builder.assert_equal(builder.add_variable(1), key_fields[4].witness_index);
builder.assert_equal(builder.add_variable(1), key_fields[4].witness_index); // WORKTODO: 4 = bug?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm yeah, it should be 3...

// Fourth key field is the whether the proof contains an aggregation object.
builder.assert_equal(builder.add_variable(1), key_fields[4].witness_index);
builder.assert_equal(builder.add_variable(1), key_fields[4].witness_index); // WORKTODO: 4 = bug?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why we wouldn't get some sort of failure if we assert_equal the 4th index with both 1 and 0 variables...?

// the proof and public_inputs we subtract and add the corresponding amount from the respective sizes.
size_t size_of_proof_with_no_pub_inputs = input.proof.size() - bb::AGGREGATION_OBJECT_SIZE;
size_t total_num_public_inputs = input.public_inputs.size() + bb::AGGREGATION_OBJECT_SIZE;
create_dummy_vkey_and_proof(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well we could just pass the unadjusted values to the create_dummy_vkey_and_proof function as well and handle it there

@ledwards2225 ledwards2225 merged commit 10d7edd into master Aug 28, 2024
37 checks passed
@ledwards2225 ledwards2225 deleted the lde/proof_surgeon branch August 28, 2024 21:13
TomAFrench pushed a commit that referenced this pull request Aug 29, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.51.1</summary>

##
[0.51.1](aztec-package-v0.51.0...aztec-package-v0.51.1)
(2024-08-29)


### Features

* Add status check to prover agent
([#8248](#8248))
([7b3006a](7b3006a))
* Faster L1 deployment
([#8234](#8234))
([51d6699](51d6699))
* Spartan token transfer
([#8163](#8163))
([38f0157](38f0157))
</details>

<details><summary>barretenberg.js: 0.51.1</summary>

##
[0.51.1](barretenberg.js-v0.51.0...barretenberg.js-v0.51.1)
(2024-08-29)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.51.1</summary>

##
[0.51.1](aztec-packages-v0.51.0...aztec-packages-v0.51.1)
(2024-08-29)


### Features

* Add CLI command for gathering proving metrics
([#8221](#8221))
([5929a42](5929a42))
* Add status check to prover agent
([#8248](#8248))
([7b3006a](7b3006a))
* **avm:** 1-slot sload/sstore (nr, ts)
([#8264](#8264))
([bdd9b06](bdd9b06))
* **avm:** Range check gadget
([#7967](#7967))
([0dd954e](0dd954e))
* **docs:** Add partial notes doc
([#8192](#8192))
([4299bbd](4299bbd))
* Faster L1 deployment
([#8234](#8234))
([51d6699](51d6699))
* Initial validator set
([#8133](#8133))
([6d31ad2](6d31ad2))
* L1-publisher cleanup
([#8148](#8148))
([6ae2535](6ae2535))
* Proof surgery class
([#8236](#8236))
([10d7edd](10d7edd))
* Request specific transactions through the p2p layer
([#8185](#8185))
([54e1cc7](54e1cc7))
* Slot duration flexibility
([#8122](#8122))
([708e4e5](708e4e5))
* Spartan token transfer
([#8163](#8163))
([38f0157](38f0157))


### Bug Fixes

* Attempt to fix nightly test
([#8222](#8222))
([477eec5](477eec5))
* **avm-simulator:** Await avm bytecode check
([#8268](#8268))
([4410eb3](4410eb3))
* **bb-prover:** Create structure for AVM vk
([#8233](#8233))
([55b6ba2](55b6ba2))
* **bb:** Mac build
([#8255](#8255))
([ac54f5c](ac54f5c))
* **ci:** Spot-runner-action was not built
([#8274](#8274))
([c1509c1](c1509c1))
* **ci:** Try fix brotli edge-case
([#8256](#8256))
([e03ea0b](e03ea0b))
* Docker containers healthchecks
([#8228](#8228))
([19edbbb](19edbbb))
* **docs:** Update entrypoint details on accounts page
([#8184](#8184))
([8453ec7](8453ec7))
* Export brillig names in contract functions
([#8212](#8212))
([4745741](4745741))
* Fixes for the nightly test run against Sepolia
([#8229](#8229))
([cfc65c6](cfc65c6))
* Handle constant output for sha256
([#8251](#8251))
([0653ba5](0653ba5))
* Log public vm errors as warn in prover-agent
([#8247](#8247))
([9f4ea9f](9f4ea9f))
* Remove devnet ARM builds for now
([#8202](#8202))
([81ef715](81ef715))
* Remove fundFpc step from bootstrap
([#8245](#8245))
([a742531](a742531))
* Ts codegen
([#8267](#8267))
([cb58800](cb58800))


### Miscellaneous

* Add check to just release images to devnet-deploys
([#8242](#8242))
([aa6791d](aa6791d))
* Add partial note support for value note
([#8141](#8141))
([daa57cc](daa57cc))
* Always run `build-check` step in `publish-bb.yml`
([#8240](#8240))
([5e9749f](5e9749f))
* **avm:** Replace range and cmp with gadgets
([#8164](#8164))
([cc12558](cc12558))
* Basic network matrix
([#8257](#8257))
([2a76b1a](2a76b1a)),
closes
[#8001](#8001)
* **bb:** Use std::span in pippenger for scalars
([#8269](#8269))
([2323cd5](2323cd5))
* Configure interval mining for anvil
([#8211](#8211))
([eba57b4](eba57b4))
* Create external-ci-approved.yml
([#8235](#8235))
([24b059b](24b059b))
* Disallow prune in devnet + add onlyOwners
([#8134](#8134))
([c736f96](c736f96))
* Fix various warnings in noir code
([#8258](#8258))
([1c6b478](1c6b478))
* Less noisy AVM failures in proving
([#8227](#8227))
([03bcd62](03bcd62))
* Open an issue if publishing bb fails
([#8223](#8223))
([2d7a775](2d7a775))
* Reinstate l1-contracts package
([#8250](#8250))
([263a912](263a912))
* Remove unused generic parameters
([#8249](#8249))
([00ed045](00ed045))
* Replace relative paths to noir-protocol-circuits
([1783c80](1783c80))
* Replace relative paths to noir-protocol-circuits
([ffe1f35](ffe1f35))
* Report prover metrics
([#8155](#8155))
([dc7bcdf](dc7bcdf)),
closes
[#7675](#7675)
* Rework balances map
([#8127](#8127))
([1cac3dd](1cac3dd)),
closes
[#8104](#8104)
* Run CI after merges to provernet
([#8244](#8244))
([97e5e25](97e5e25))


### Documentation

* Minor fixes
([#8273](#8273))
([2b8af9e](2b8af9e))
</details>

<details><summary>barretenberg: 0.51.1</summary>

##
[0.51.1](barretenberg-v0.51.0...barretenberg-v0.51.1)
(2024-08-29)


### Features

* **avm:** 1-slot sload/sstore (nr, ts)
([#8264](#8264))
([bdd9b06](bdd9b06))
* **avm:** Range check gadget
([#7967](#7967))
([0dd954e](0dd954e))
* Proof surgery class
([#8236](#8236))
([10d7edd](10d7edd))


### Bug Fixes

* **bb-prover:** Create structure for AVM vk
([#8233](#8233))
([55b6ba2](55b6ba2))
* **bb:** Mac build
([#8255](#8255))
([ac54f5c](ac54f5c))
* Handle constant output for sha256
([#8251](#8251))
([0653ba5](0653ba5))


### Miscellaneous

* **avm:** Replace range and cmp with gadgets
([#8164](#8164))
([cc12558](cc12558))
* **bb:** Use std::span in pippenger for scalars
([#8269](#8269))
([2323cd5](2323cd5))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
AztecBot added a commit to AztecProtocol/barretenberg that referenced this pull request Aug 30, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.51.1</summary>

##
[0.51.1](AztecProtocol/aztec-packages@aztec-package-v0.51.0...aztec-package-v0.51.1)
(2024-08-29)


### Features

* Add status check to prover agent
([#8248](AztecProtocol/aztec-packages#8248))
([7b3006a](AztecProtocol/aztec-packages@7b3006a))
* Faster L1 deployment
([#8234](AztecProtocol/aztec-packages#8234))
([51d6699](AztecProtocol/aztec-packages@51d6699))
* Spartan token transfer
([#8163](AztecProtocol/aztec-packages#8163))
([38f0157](AztecProtocol/aztec-packages@38f0157))
</details>

<details><summary>barretenberg.js: 0.51.1</summary>

##
[0.51.1](AztecProtocol/aztec-packages@barretenberg.js-v0.51.0...barretenberg.js-v0.51.1)
(2024-08-29)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.51.1</summary>

##
[0.51.1](AztecProtocol/aztec-packages@aztec-packages-v0.51.0...aztec-packages-v0.51.1)
(2024-08-29)


### Features

* Add CLI command for gathering proving metrics
([#8221](AztecProtocol/aztec-packages#8221))
([5929a42](AztecProtocol/aztec-packages@5929a42))
* Add status check to prover agent
([#8248](AztecProtocol/aztec-packages#8248))
([7b3006a](AztecProtocol/aztec-packages@7b3006a))
* **avm:** 1-slot sload/sstore (nr, ts)
([#8264](AztecProtocol/aztec-packages#8264))
([bdd9b06](AztecProtocol/aztec-packages@bdd9b06))
* **avm:** Range check gadget
([#7967](AztecProtocol/aztec-packages#7967))
([0dd954e](AztecProtocol/aztec-packages@0dd954e))
* **docs:** Add partial notes doc
([#8192](AztecProtocol/aztec-packages#8192))
([4299bbd](AztecProtocol/aztec-packages@4299bbd))
* Faster L1 deployment
([#8234](AztecProtocol/aztec-packages#8234))
([51d6699](AztecProtocol/aztec-packages@51d6699))
* Initial validator set
([#8133](AztecProtocol/aztec-packages#8133))
([6d31ad2](AztecProtocol/aztec-packages@6d31ad2))
* L1-publisher cleanup
([#8148](AztecProtocol/aztec-packages#8148))
([6ae2535](AztecProtocol/aztec-packages@6ae2535))
* Proof surgery class
([#8236](AztecProtocol/aztec-packages#8236))
([10d7edd](AztecProtocol/aztec-packages@10d7edd))
* Request specific transactions through the p2p layer
([#8185](AztecProtocol/aztec-packages#8185))
([54e1cc7](AztecProtocol/aztec-packages@54e1cc7))
* Slot duration flexibility
([#8122](AztecProtocol/aztec-packages#8122))
([708e4e5](AztecProtocol/aztec-packages@708e4e5))
* Spartan token transfer
([#8163](AztecProtocol/aztec-packages#8163))
([38f0157](AztecProtocol/aztec-packages@38f0157))


### Bug Fixes

* Attempt to fix nightly test
([#8222](AztecProtocol/aztec-packages#8222))
([477eec5](AztecProtocol/aztec-packages@477eec5))
* **avm-simulator:** Await avm bytecode check
([#8268](AztecProtocol/aztec-packages#8268))
([4410eb3](AztecProtocol/aztec-packages@4410eb3))
* **bb-prover:** Create structure for AVM vk
([#8233](AztecProtocol/aztec-packages#8233))
([55b6ba2](AztecProtocol/aztec-packages@55b6ba2))
* **bb:** Mac build
([#8255](AztecProtocol/aztec-packages#8255))
([ac54f5c](AztecProtocol/aztec-packages@ac54f5c))
* **ci:** Spot-runner-action was not built
([#8274](AztecProtocol/aztec-packages#8274))
([c1509c1](AztecProtocol/aztec-packages@c1509c1))
* **ci:** Try fix brotli edge-case
([#8256](AztecProtocol/aztec-packages#8256))
([e03ea0b](AztecProtocol/aztec-packages@e03ea0b))
* Docker containers healthchecks
([#8228](AztecProtocol/aztec-packages#8228))
([19edbbb](AztecProtocol/aztec-packages@19edbbb))
* **docs:** Update entrypoint details on accounts page
([#8184](AztecProtocol/aztec-packages#8184))
([8453ec7](AztecProtocol/aztec-packages@8453ec7))
* Export brillig names in contract functions
([#8212](AztecProtocol/aztec-packages#8212))
([4745741](AztecProtocol/aztec-packages@4745741))
* Fixes for the nightly test run against Sepolia
([#8229](AztecProtocol/aztec-packages#8229))
([cfc65c6](AztecProtocol/aztec-packages@cfc65c6))
* Handle constant output for sha256
([#8251](AztecProtocol/aztec-packages#8251))
([0653ba5](AztecProtocol/aztec-packages@0653ba5))
* Log public vm errors as warn in prover-agent
([#8247](AztecProtocol/aztec-packages#8247))
([9f4ea9f](AztecProtocol/aztec-packages@9f4ea9f))
* Remove devnet ARM builds for now
([#8202](AztecProtocol/aztec-packages#8202))
([81ef715](AztecProtocol/aztec-packages@81ef715))
* Remove fundFpc step from bootstrap
([#8245](AztecProtocol/aztec-packages#8245))
([a742531](AztecProtocol/aztec-packages@a742531))
* Ts codegen
([#8267](AztecProtocol/aztec-packages#8267))
([cb58800](AztecProtocol/aztec-packages@cb58800))


### Miscellaneous

* Add check to just release images to devnet-deploys
([#8242](AztecProtocol/aztec-packages#8242))
([aa6791d](AztecProtocol/aztec-packages@aa6791d))
* Add partial note support for value note
([#8141](AztecProtocol/aztec-packages#8141))
([daa57cc](AztecProtocol/aztec-packages@daa57cc))
* Always run `build-check` step in `publish-bb.yml`
([#8240](AztecProtocol/aztec-packages#8240))
([5e9749f](AztecProtocol/aztec-packages@5e9749f))
* **avm:** Replace range and cmp with gadgets
([#8164](AztecProtocol/aztec-packages#8164))
([cc12558](AztecProtocol/aztec-packages@cc12558))
* Basic network matrix
([#8257](AztecProtocol/aztec-packages#8257))
([2a76b1a](AztecProtocol/aztec-packages@2a76b1a)),
closes
[#8001](AztecProtocol/aztec-packages#8001)
* **bb:** Use std::span in pippenger for scalars
([#8269](AztecProtocol/aztec-packages#8269))
([2323cd5](AztecProtocol/aztec-packages@2323cd5))
* Configure interval mining for anvil
([#8211](AztecProtocol/aztec-packages#8211))
([eba57b4](AztecProtocol/aztec-packages@eba57b4))
* Create external-ci-approved.yml
([#8235](AztecProtocol/aztec-packages#8235))
([24b059b](AztecProtocol/aztec-packages@24b059b))
* Disallow prune in devnet + add onlyOwners
([#8134](AztecProtocol/aztec-packages#8134))
([c736f96](AztecProtocol/aztec-packages@c736f96))
* Fix various warnings in noir code
([#8258](AztecProtocol/aztec-packages#8258))
([1c6b478](AztecProtocol/aztec-packages@1c6b478))
* Less noisy AVM failures in proving
([#8227](AztecProtocol/aztec-packages#8227))
([03bcd62](AztecProtocol/aztec-packages@03bcd62))
* Open an issue if publishing bb fails
([#8223](AztecProtocol/aztec-packages#8223))
([2d7a775](AztecProtocol/aztec-packages@2d7a775))
* Reinstate l1-contracts package
([#8250](AztecProtocol/aztec-packages#8250))
([263a912](AztecProtocol/aztec-packages@263a912))
* Remove unused generic parameters
([#8249](AztecProtocol/aztec-packages#8249))
([00ed045](AztecProtocol/aztec-packages@00ed045))
* Replace relative paths to noir-protocol-circuits
([1783c80](AztecProtocol/aztec-packages@1783c80))
* Replace relative paths to noir-protocol-circuits
([ffe1f35](AztecProtocol/aztec-packages@ffe1f35))
* Report prover metrics
([#8155](AztecProtocol/aztec-packages#8155))
([dc7bcdf](AztecProtocol/aztec-packages@dc7bcdf)),
closes
[#7675](AztecProtocol/aztec-packages#7675)
* Rework balances map
([#8127](AztecProtocol/aztec-packages#8127))
([1cac3dd](AztecProtocol/aztec-packages@1cac3dd)),
closes
[#8104](AztecProtocol/aztec-packages#8104)
* Run CI after merges to provernet
([#8244](AztecProtocol/aztec-packages#8244))
([97e5e25](AztecProtocol/aztec-packages@97e5e25))


### Documentation

* Minor fixes
([#8273](AztecProtocol/aztec-packages#8273))
([2b8af9e](AztecProtocol/aztec-packages@2b8af9e))
</details>

<details><summary>barretenberg: 0.51.1</summary>

##
[0.51.1](AztecProtocol/aztec-packages@barretenberg-v0.51.0...barretenberg-v0.51.1)
(2024-08-29)


### Features

* **avm:** 1-slot sload/sstore (nr, ts)
([#8264](AztecProtocol/aztec-packages#8264))
([bdd9b06](AztecProtocol/aztec-packages@bdd9b06))
* **avm:** Range check gadget
([#7967](AztecProtocol/aztec-packages#7967))
([0dd954e](AztecProtocol/aztec-packages@0dd954e))
* Proof surgery class
([#8236](AztecProtocol/aztec-packages#8236))
([10d7edd](AztecProtocol/aztec-packages@10d7edd))


### Bug Fixes

* **bb-prover:** Create structure for AVM vk
([#8233](AztecProtocol/aztec-packages#8233))
([55b6ba2](AztecProtocol/aztec-packages@55b6ba2))
* **bb:** Mac build
([#8255](AztecProtocol/aztec-packages#8255))
([ac54f5c](AztecProtocol/aztec-packages@ac54f5c))
* Handle constant output for sha256
([#8251](AztecProtocol/aztec-packages#8251))
([0653ba5](AztecProtocol/aztec-packages@0653ba5))


### Miscellaneous

* **avm:** Replace range and cmp with gadgets
([#8164](AztecProtocol/aztec-packages#8164))
([cc12558](AztecProtocol/aztec-packages@cc12558))
* **bb:** Use std::span in pippenger for scalars
([#8269](AztecProtocol/aztec-packages#8269))
([2323cd5](AztecProtocol/aztec-packages@2323cd5))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants