Finish documenting / testing Zeromorph #142

huitseeker · 2023-11-27T16:20:35Z

Tests & documents Zeromorph.

Most tests inspired / adapted from the equivalent ones in the Aztec PR. Our 3d-party notices amended accordingly.

Note

Lurk isn't expected to compile on the Zeromorph branch.

Closes #35

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

- Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector, - correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`. - Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes.

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction. - correct an off-by-one error in the y_k terms

- Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs` - Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs`

adr1anh · 2023-11-28T08:04:06Z

src/provider/non_hiding_zeromorph.rs

-      UVKZGPoly::new(q_hat)
-    };
+    // Compute the batched, lifted-degree quotient `\hat{q}`
+    // q_hat = \sum_{i=0}^{num_vars-1} y^i * q_i(x)


Missing a x^{D-di}, where D = 2^{num_vars} and di = deg(q_i)

* fix: remove a TODO using RefCast * fix: remove unsightly clone * doc: comment intermediate ZM steps * refactor: Refactor `open` function in zeromorph provider Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code. * test: test batched_lifted_degree_quotient Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials. * refactor: Refine computations in non_hiding_zeromorph.rs - Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector, - correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`. - Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes. * test: test partially evaluated quotient \zeta_x - Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction. - correct an off-by-one error in the y_k terms * test: test partially evaluated quotient - Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs` - Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs` * fix: refine q_hat comment

* Schoolbook adapted KZG PoC Remove toxic waste from setup parameters Refactorings ZeroMorph skeleton * feat: provide a non-hiding variant of KZG Drafts a non-hiding variant of the KZG PCS on univariate Dense polynomial representations Details: - Changed access levels and ordering for various modules and functions like `sumcheck`, `cpu_best_multiexp`, and `mod kzg;`. - Upgraded `halo2curves` version and added new dependencies like `pairing`, `anyhow`, `rand`, and `group`. - Leveraged the UniPoly in sumcheck.rs - Created `non_hiding_kzg.rs` file, introducing new structures and functionalities like `UVUniversalKZGParam`, `UVKZGProverKey`, `UVKZGPoly`, and `UVKZGPCS` along with their implementation and tests. - Modified the import of `Engine` in `kzg.rs` from `halo2curves::pairing::Engine` to `pairing::Engine` (reflecting the new version of halo2curves). feat: add batch commit / open / prove * Small subtasks: params, phi, un function, EE Minor adjustments * chore: setup zeromorph refactor: Remove kzg and zeromorph provider modules * feat: add openings for Zeromorph - Added new data structures and their related functions including the `ZMProverKey`, `ZMVerifierKey`, `ZMCommitment`, `ZMEvaluation`, `ZMProof` and `ZMPCS` in `non_hiding_zeromorph.rs` - Implemented new functionalities in `non_hiding_zeromorph.rs` and `spartan/polynomial.rs` to provide better handling of polynomials, including fetching commitments for a polynomial, computing the quotient polynomials of an existing polynomial. - Enhanced the `UniPoly` struct in `spartan/sumcheck.rs` with `Index` and `IndexMut` for better coefficient access, and `AddAssign` and `MulAssign` for scalar and self types. - Removed and replaced certain elements in `UVUniversalKZGParam` struct in `non_hiding_kzg.rs` for improved flexibility, and included import and implementation of `TranscriptReprTrait`. draft ZM verify set up test structure defer test_quo * Include commits into ZMProof and adjust RO * feat: Refactor polynomial evaluation methods - Temporarily disabled some assertions in both `non_hiding_zeromorph.rs` and `polynomial.rs` for debugging purposes. - Introduced `evaluate_opt`, `fix_variables`, and `fix_one_variable_helper` functions in `polynomial.rs` to support multilinear polynomial evaluation and partial evaluation. fix: add domain separators - Integrated additional functionality into the ZMPCS implementation in `non_hiding_zeromorph.rs` by adding a protocol name function. - Improved security in `non_hiding_zeromorph.rs` by integrating domain separators into transcript's `open` and `verify` functions. refactor: Refactor code and test performance for NHZM - Implement `AsRef` trait for `UniPoly` structure in `src/spartan/sumcheck.rs` to facilitate direct access to its coefficients. - Refactor code in `src/provider/non_hiding_zeromorph.rs` to directly use `into_iter` in map function when creating `quotients_polys`, avoiding a large clone - Enhance test performance in `src/provider/non_hiding_zeromorph.rs` by pre-generating a `universal_setup` for tests, introducing a `max_vars` variable and RNG in the `commit_open_verify_with` test function, and adjusting the range of num_vars used for testing. fix: adjust APIs & Serialize impls for the Nova traits * feat: set up serde & abomonation bounds * add generic PCS errors * pp wip * fix: adjust type parameters for nontrivial_with_zm test fix: ignore panicking test fix: add doctest for evaluate_opt fix: remove obsolete comments chore: move UniPoly methods where they should be test: make clear current zeromorph operates in monomial basis - Added an additional test `test_dense_evaliations()` to provide more comprehensive testing for the `evaluate()` and `evaluate_opt()` functions in `MultilinearPolynomial`. refactor TranscriptReprTrait impl for compat with Commitments * feat: Implement KZG commitment trait and serialization features - Added serialization, deserialization, and Abomonation traits to UVUniversalKZGParam struct in `non_hiding_kzg.rs` file along with implementing comparison and length evaluation traits. - Created a new file `kzg_commitment.rs` which implements KZG Commitment Engine and setup, commit functions. - Integrated `kzg_commitment` in module `provider` and set up conversions between Commitment and UVKZGCommitment. - Enhanced assertion in `minroot_serde.rs` file from clone comparison to dereferenced comparison in MinRoot delay case. * Use the ZMPCS Evaluation Engine and the KZG Commitment Engine in tests. feat: Improve `prove` and `verify` methods in `ZMPCS` struct - Updated `prove` and `verify` methods in `ZMPCS<E>` struct within `non_hiding_zeromorph.rs` with actual logic for commitment, evaluation, and verification. - Adjusted the construction of `ZMCommitment` and `ZMEvaluation` within `prove` and `verify` methods. - Commented on portions of the code in `non_hiding_zeromorph.rs` that need further modification and refinement. - Modified test value for `test_pp_digest_with` in the `test_supernova_pp_digest` test case to match the new expected output. * fix evaluation reversal bug fix: remove superfluous eval functions fix: remove endianness shenanigans test: add evaluation unit test * fix: parallellize pp generation * feat: Enhance KZG commitment SRS generation efficiency using parallel computation - Introduced a new module `util` within the `provider` module, implementing fixed-base MSM, - New trait constraint has been imposed for `E::Fr: PrimeFieldBits` within the `non_hiding_zeromorph.rs` file, and usages have been adjusted in the `test` module. - Adding the `PrimeFieldBits` import from the `ff` crate and importing `provider::util` from the local crate. refactor: Refactor utility functions for elliptic curve operations - Renamed and moved `util.rs` to `fb_msm.rs` under the `provider` directory. - documented the FB MSM * Apply comments from code review Co-authored-by: Adrian Hamelink <adrian.hamelink@gmail.com> * Improve documentation and testing in non_hiding_zeromorph.rs - Enhanced the clarity and accuracy of code comments and documentation in `non_hiding_zeromorph.rs`, specifically within the `ZMPCS` struct's methods. - Enriched the `quotients` function's documentation, detailing its mathematical underpinnings - Implemented a new rigorous test, `test_quotients`, to ensure the `quotients` function's output satisfies the polynomial identity. * test: refactor random ML poly generation * Finish documenting / testing Zeromorph (#142) * fix: remove a TODO using RefCast * fix: remove unsightly clone * doc: comment intermediate ZM steps * refactor: Refactor `open` function in zeromorph provider Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code. * test: test batched_lifted_degree_quotient Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials. * refactor: Refine computations in non_hiding_zeromorph.rs - Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector, - correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`. - Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes. * test: test partially evaluated quotient \zeta_x - Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction. - correct an off-by-one error in the y_k terms * test: test partially evaluated quotient - Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs` - Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs` * fix: refine q_hat comment * fix: organize code more sparsely after rebase * refactor: Improve code readability and error handling in zeromorph - Updated and enhanced clarity and consistency of mathematical notation in comments across `non_hiding_kzg.rs` and `non_hiding_zeromorph.rs` files. - Implemented error handling in the `ZMPCS::verify` function within the `non_hiding_zeromorph.rs` file. * fix: compute quotient commitments in parallel * refactor: clean up --------- Co-authored-by: emmorais <eduardo.morais@gmail.com> Co-authored-by: Matej Penciak <matej.penciak@gmail.com> Co-authored-by: Adrian Hamelink <adrian.hamelink@gmail.com>

huitseeker added 8 commits November 27, 2023 11:15

fix: remove a TODO using RefCast

e611c87

fix: remove unsightly clone

d00273e

doc: comment intermediate ZM steps

20bb1ed

refactor: Refactor open function in zeromorph provider

1c87338

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

test: test batched_lifted_degree_quotient

0f1c63b

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

test: test partially evaluated quotient \zeta_x

3b2a2d2

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction. - correct an off-by-one error in the y_k terms

huitseeker requested review from adr1anh, porcuquine and mpenciak November 27, 2023 16:20

adr1anh reviewed Nov 28, 2023

View reviewed changes

adr1anh approved these changes Nov 28, 2023

View reviewed changes

fix: refine q_hat comment

1116f67

huitseeker merged commit 785a0a3 into zeromorph Nov 28, 2023
2 of 3 checks passed

huitseeker deleted the zeromorph_remove_warts branch November 28, 2023 16:38

huitseeker added a commit that referenced this pull request Nov 28, 2023

fixup! Finish documenting / testing Zeromorph (#142)

10f4f2d

huitseeker mentioned this pull request Nov 29, 2023

ZeroMorph: Polish and document code #35

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Finish documenting / testing Zeromorph #142

Finish documenting / testing Zeromorph #142

huitseeker commented Nov 27, 2023 •

edited

Loading

adr1anh Nov 28, 2023

Finish documenting / testing Zeromorph #142

Finish documenting / testing Zeromorph #142

Conversation

huitseeker commented Nov 27, 2023 • edited Loading

adr1anh Nov 28, 2023

Choose a reason for hiding this comment

huitseeker commented Nov 27, 2023 •

edited

Loading