Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial experimental HyperNova multifolding support #175

Draft
wants to merge 102 commits into
base: main
Choose a base branch
from

Commits on Jul 11, 2023

  1. Configuration menu
    Copy the full SHA
    789354a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5a025c6 View commit details
    Browse the repository at this point in the history
  3. feat: First cut of CCS (#14)

    Still quite rough and lots of bugs
    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    a3e2182 View commit details
    Browse the repository at this point in the history
  4. refactor: Remove digest

    Simplified in recent commit microsoft@b28aaf7
    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    5d4b110 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    60bbec2 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    4f5e784 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    f72c49b View commit details
    Browse the repository at this point in the history
  8. refactor(ccs): Add experimental hypernova feature flag (on by default)

    Also add docs to README
    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    4470d92 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    9182cc3 View commit details
    Browse the repository at this point in the history
  10. chore: update dep

    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    db6add0 View commit details
    Browse the repository at this point in the history
  11. chore: Cargo.toml

    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    2a59d80 View commit details
    Browse the repository at this point in the history
  12. add doc and tests for spartan/polynomial.rs

    1. Add documentation and test for spartan/polynomial.rs.
    - EqPolynomial: $\tilde{eq}$
    - MultilinearPolynomial: multilinear polynomial
    - SparsePolynomial
    
    2. Remove duplicate get_bits in SparsePolynomial::evaluate.
    wangtsiao authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    85f4e22 View commit details
    Browse the repository at this point in the history
  13. use pasta instead of defining new fp

    wangtsiao authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    5223fea View commit details
    Browse the repository at this point in the history
  14. wrap link with <> in comment

    wangtsiao authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    6a45489 View commit details
    Browse the repository at this point in the history
  15. fmt(spartan): cargo fmt

    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    9598d1d View commit details
    Browse the repository at this point in the history
  16. Remove: Clippy unused fn lint

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    64b7bd7 View commit details
    Browse the repository at this point in the history
  17. change: Impl SparseMatrix type

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    8db47a5 View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    ac57cb6 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    d373753 View commit details
    Browse the repository at this point in the history
  20. feat(ccs): Sketch out basic CCCS/LCCCS associated data structures

    Also add some comments
    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    ed36b4a View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    0262dc6 View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    f1a8834 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    d1e63a0 View commit details
    Browse the repository at this point in the history
  24. WIP CCCS

    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    0192d63 View commit details
    Browse the repository at this point in the history
  25. WIP: CCS MLE stuff

    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    49455f8 View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    1a9244e View commit details
    Browse the repository at this point in the history
  27. refactor(ccs): change c to be Scalar

    Makes some operations easier
    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    93fc061 View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    dbc0c22 View commit details
    Browse the repository at this point in the history
  29. wip(ccs): compute_g function

    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    48c1ac9 View commit details
    Browse the repository at this point in the history
  30. Configuration menu
    Copy the full SHA
    0a473b0 View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    e404742 View commit details
    Browse the repository at this point in the history
  32. Configuration menu
    Copy the full SHA
    987359b View commit details
    Browse the repository at this point in the history
  33. feat(ccs): compute_sum_Mz

    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    5ec547a View commit details
    Browse the repository at this point in the history
  34. fix: Correct SparseMatrix aux methods

    This updates and makes compile the `n_rows`/`n_cols` fns of
    SparseMatrix.
    It also updates `is_valid` which is transfromed from a closure into an
    associated fn for SparseMatrix.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    ab70853 View commit details
    Browse the repository at this point in the history
  35. fix: Update CCS-related structs & constructor

    This commit:
    - Removes attribute duplicity from `CCSShape`.
    - Fixes the reversed naming for col&row number.
    - Removes leftover comments that no longer apply.
    - Update the `CCSShape::new` to use `is_valid` on the new format as well
      as remove useless checks & update to new `CCSSape` format.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    d0a5160 View commit details
    Browse the repository at this point in the history
  36. change: Remove Result from Matrix/vec ops

    This removes the annoying results that were not needed from
    ops implementations and replaces them by leaving the "necessary"
    sanity-checks as assertions.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    fa0376e View commit details
    Browse the repository at this point in the history
  37. change: Update CCS.is_sat() to use iterators

    The impl gets cleaner and also easier to update later.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    0559644 View commit details
    Browse the repository at this point in the history
  38. Configuration menu
    Copy the full SHA
    10f1b9f View commit details
    Browse the repository at this point in the history
  39. Configuration menu
    Copy the full SHA
    659ef74 View commit details
    Browse the repository at this point in the history
  40. Configuration menu
    Copy the full SHA
    ff4b44b View commit details
    Browse the repository at this point in the history
  41. remove: Unused CCShape::matrix_mul fn

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    af1e3d5 View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    9b2f3c5 View commit details
    Browse the repository at this point in the history
  43. Configuration menu
    Copy the full SHA
    2a79b9e View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    1b8bba8 View commit details
    Browse the repository at this point in the history
  45. Configuration menu
    Copy the full SHA
    8075c15 View commit details
    Browse the repository at this point in the history
  46. Configuration menu
    Copy the full SHA
    b81e432 View commit details
    Browse the repository at this point in the history
  47. change: Move CCS stuff to it's own module

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    0f26a23 View commit details
    Browse the repository at this point in the history
  48. Configuration menu
    Copy the full SHA
    68afd5e View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    f48dda8 View commit details
    Browse the repository at this point in the history
  50. add: Add CCS -> CCCS conversions

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    71668c7 View commit details
    Browse the repository at this point in the history
  51. Configuration menu
    Copy the full SHA
    b2e8a6b View commit details
    Browse the repository at this point in the history
  52. fix: Include n_cos & n_rows info in SparseMatrix

    This prevents having issues as shown in the tests now failing for MLE
    conversions. The issue is that empty rows/columns at the end of the
    matrixes cause the algorithms to actually fail. As we have a sense of
    the matrix dimensions only when we are in the context of CCS. But we
    don't when we're outside of it.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    578431c View commit details
    Browse the repository at this point in the history
  53. change: Update pad and from_r1cs CCS methods.

    This solves some minor errors & issues with the CCS methods as well as
    updates the SparseMatrix calls to the new API/params used.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    bc94cb7 View commit details
    Browse the repository at this point in the history
  54. remove: coefficients field from HyperCube & refactor

    This helps to reduce the complexity of the implementation while at the
    same time removing the possibility of handling a boolean hypercube with
    coefficients which is not needed now at all.
    
    We can always bring this back later.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    dec29ae View commit details
    Browse the repository at this point in the history
  55. Configuration menu
    Copy the full SHA
    a11cf9c View commit details
    Browse the repository at this point in the history
  56. add: Port MLE support & tests passing

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    055751e View commit details
    Browse the repository at this point in the history
  57. fix: Vec * Sparse

    Matrix prod outp len
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    1450875 View commit details
    Browse the repository at this point in the history
  58. Configuration menu
    Copy the full SHA
    a057978 View commit details
    Browse the repository at this point in the history
  59. this code is cursed

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    1b60494 View commit details
    Browse the repository at this point in the history
  60. lift the curse:

    - BooleanHypercube iterator was reversing the bit representation
    - bound_poly_var_top is not fixing first variables but fixing last variables (=Espresso/hyperplonk's fix_last_variables method)
    - fix method fix_one_variable_helper
    
    Add also a couple of notes regarding sparse_vec_to_mle & bound_poly_var_top methods.
    
    Note: the title of this commit is in reference to the commit message of 22b3f6f
    arnaucube authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    827a1ff View commit details
    Browse the repository at this point in the history
  61. Configuration menu
    Copy the full SHA
    fe14cdd View commit details
    Browse the repository at this point in the history
  62. fix cccs.compute_sum_Mz method, small fix in compute_q, so by consequ…

    …ence compute_q & compute_Q now work fine
    arnaucube authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    ce646e6 View commit details
    Browse the repository at this point in the history
  63. Configuration menu
    Copy the full SHA
    af45a4c View commit details
    Browse the repository at this point in the history
  64. remove: CCS struct which is unused

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    c800b39 View commit details
    Browse the repository at this point in the history
  65. Configuration menu
    Copy the full SHA
    b714a20 View commit details
    Browse the repository at this point in the history
  66. change: Create utils folder under ccs

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    d436f87 View commit details
    Browse the repository at this point in the history
  67. change: Move Compute_sum_Mz into util mode

    This method was part of CCCS associated methods. But it doesn't make
    sense as LCCCS also requires it aside from other standalone functions.
    
    Hence, it is moved to a util module.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    8e2d99c View commit details
    Browse the repository at this point in the history
  68. Configuration menu
    Copy the full SHA
    99679fb View commit details
    Browse the repository at this point in the history
  69. Configuration menu
    Copy the full SHA
    8b73f47 View commit details
    Browse the repository at this point in the history
  70. Configuration menu
    Copy the full SHA
    eaadaf2 View commit details
    Browse the repository at this point in the history
  71. Configuration menu
    Copy the full SHA
    e3303cd View commit details
    Browse the repository at this point in the history
  72. Configuration menu
    Copy the full SHA
    f5a13a4 View commit details
    Browse the repository at this point in the history
  73. Configuration menu
    Copy the full SHA
    bf589be View commit details
    Browse the repository at this point in the history
  74. Configuration menu
    Copy the full SHA
    45e8391 View commit details
    Browse the repository at this point in the history
  75. fix: compute_all_sum_Mz_evals with test

    This adds a test for the `compute_all_sum_Mz_evals` fn and fixes it by
    reversing the order of iteration over `r_x` so that it matches the
    expectancies from Spartan/Polynomial lib impl.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    a14d27d View commit details
    Browse the repository at this point in the history
  76. fix: test_lcccs_v_j working

    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    3c1017c View commit details
    Browse the repository at this point in the history
  77. add: Basic multifolding functions & tests

    Includes fns to compute thetas & sigmas, C from thetas and sigmans and
    computation of `g` poly.
    
    It also includes tests for g computation.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    2a1fa7a View commit details
    Browse the repository at this point in the history
  78. Configuration menu
    Copy the full SHA
    a2d4408 View commit details
    Browse the repository at this point in the history
  79. fix: test_compute_sigmas_and_thetas

    This fixes the endianness of `r_x_prime` which was the issue that
    prevented the `test_compute_sigmas_and_thetas` to pass.
    CPerezz authored and oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    78a4059 View commit details
    Browse the repository at this point in the history
  80. Configuration menu
    Copy the full SHA
    d37afdb View commit details
    Browse the repository at this point in the history
  81. Configuration menu
    Copy the full SHA
    09cef38 View commit details
    Browse the repository at this point in the history
  82. test(multifolding): Fix CCCS assert

    Was using wrong witness. Also change z order and make variables more more eplicit.
    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    a33e7c3 View commit details
    Browse the repository at this point in the history
  83. refactor(ccs): Remove special CCCSWitness

    Should be the same according to current logic and multifolding-poc. Also change to to_cccs.
    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    216e0e6 View commit details
    Browse the repository at this point in the history
  84. tests(ccs): Fix test_lcccs_fold test

    Fix variable order and make used variables more explicit to correspond to multifolding-poc
    oskarth committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    f3bcf68 View commit details
    Browse the repository at this point in the history
  85. Configuration menu
    Copy the full SHA
    af959fb View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Merge upstream/main

    oskarth committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    f07e71b View commit details
    Browse the repository at this point in the history
  2. chore: fix merge artifact

    oskarth committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    4828a20 View commit details
    Browse the repository at this point in the history
  3. chore: Restore global warnings and unused

    Replace with local allow unused, works with hypernova flag on and off
    oskarth committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    6f1ed22 View commit details
    Browse the repository at this point in the history
  4. refactor: Enable requirement of even public IO by default

    Hacky solution with feature flag for experimental hypernova for now, need to adjust test case to fix better
    oskarth committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    20ba46c View commit details
    Browse the repository at this point in the history
  5. Hypernova first cut (#15)

    Opening WIP PR to make diff more clear
    
    Targets #13
    
    This merges the multifolding-poc porting into the Nova codebase
    CPerezz committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    ca55e73 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2023

  1. Configuration menu
    Copy the full SHA
    cde23f3 View commit details
    Browse the repository at this point in the history
  2. test(utils): Make generic over Field

    oskarth authored and CPerezz committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    3fdaeab View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4ae0cf7 View commit details
    Browse the repository at this point in the history
  4. fix: Address all Clippy suggestions/lints

    As seen in previous CI runs:
    - 4ae0cf7
    - cde23f3
    - ca55e73
    
    The clippy job is causing the CI to fail due to missing fixes on the
    code.
    This gets up to speed the codebase with Clippy recommendations and
    leaves the Uppercase warning disabled.
    CPerezz committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    3437fae View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2023

  1. Update bit_decompose def:

    - Move bit_decompose into hypercube.rs which is closer conceptually and
      only used there (except for a VirtualPolynomial test)
    - Remove unused bit_decompose imports from other files
    arnaucube committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    6fa448f View commit details
    Browse the repository at this point in the history
  2. refactor(virtual_poly): Remove unnecessary default from PhantomData (#39

    )
    
    Fixes additional clippy errors I see when I upgraded my Rust toolchain
    (had some issues with clippy and Homebrew)
    
    ```
    error: use of `default` to create a unit struct
       --> src/ccs/util/virtual_poly.rs:120:29
        |
    120 |         phantom: PhantomData::default(),
        |                             ^^^^^^^^^^^ help: remove this call to `default`
        |
        = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#default_constructed_unit_structs
    note: the lint level is defined here
    ```
    oskarth committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    a23c6ab View commit details
    Browse the repository at this point in the history
  3. refactor: Make tests generic w.r.t. curve/group (#40)

    Addresses #34
    
    Should cover all files
    oskarth committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    9a508b2 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. NIMFS-centric API refactor for multifolding impl (#41)

    This is a proposal for the new API of the Hypernova/multifolding current
    impl.
    
    The idea is that the API is
    [NIMFS](https://github.com/privacy-scaling-explorations/Nova/blob/change%2FNIMFS_centric_API/src/ccs/multifolding.rs#L39-L44)-centric.
    That means, the end user only needs to deal for now with `CCS`(this can
    be prevented) and the `NIMFS` object.
    
    The rest of interactions with `CCCS` & `LCCCS` have been "masked" or at
    the very least, is not necessary to import these structs to have full
    functionallity.
    
    The current workflow that this API brings can be clearly seen here: 
    ```rust
    let ccs = CCS::<G>::from_r1cs(r1cs_shape);
      let mles = ccs.matrix_mles();
      let mut nimfs = NIMFS::init(
        &mut rng,
        ccs,
        mles,
        // Note we constructed z on the fly with the previously-used witness.
        vec![
          G::Scalar::ONE,
          G::Scalar::from(3u64),
          G::Scalar::from(35u64),
        ],
      );
    
      // Now, the NIMFS should satisfy correctly as we have inputed valid starting inpuits for the first LCCCS contained instance:
      assert!(nimfs.is_sat().is_ok());
    
      // Now let's create a valid CCCS instance and fold it:
      let valid_cccs = nimfs.gen_cccs(vec![
        G::Scalar::ONE,
        G::Scalar::from(2u64),
        G::Scalar::from(15u64),
      ]);
      nimfs.fold(&mut rng, valid_cccs);
    
      // Since the instance was correct, the NIMFS should still be satisfied.
      assert!(nimfs.is_sat().is_ok());
    
      // Now let's create am invalid CCCS instance and fold it:
      let invalid_cccs = nimfs.gen_cccs(vec![
        G::Scalar::ONE,
        G::Scalar::from(5u64),
        G::Scalar::from(55u64),
      ]);
      nimfs.fold(&mut rng, invalid_cccs);
    
      // Since the instance was wrong, the NIMFS should not be satisfied correctly.
      assert!(nimfs.is_sat().is_err());
    ```
    
    This is part of a test, located in
    https://github.com/privacy-scaling-explorations/Nova/blob/change%2FNIMFS_centric_API/tests/nimfs.rs
    
    The idea is that the user needs to type as less as possible in order to
    get a fold done.
    CPerezz committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    1119702 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. Use polynomial.rs instead of virtual_poly for equality polynomial (#42)

    - Remove custom eq_poly and use EqPolynomial instead
    - Update polynomial docs
    - Ensures order of evaluation is consistent between for VirtualPoly and
    EqPolynomial
    - Remove old build_eq_x_r_vec and build_eq_x_r_helper
    - Tests around boolean hypercube and other introduced functions to make
    endianness explicit etc
    
    Addresses #19
    oskarth committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    bb7a651 View commit details
    Browse the repository at this point in the history
  2. Transcript usage inclusion into the codebase (#43)

    The PR introduces usage from the associated trait/type of `Group` called
    `TranscriptEngineTrait` to perform the Fiat-Shamir inside the
    multifolding for the Sigmas and Thetas computations as well as gammas.
    
    This should be rebased on the top of `main` once #41 is merged and then
    reviewed. As there could be places where FS should be applied but it
    isn't.
    CPerezz committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    0c8915b View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2023

  1. Remove witness from CCCS & LCCCS instances (#48)

    Since the Verifier in the protocol does not have access to the
    witnesses, we should not have `z`s inside of the LCCCS and CCCS
    instances and instead, work with the witness sepparatedly.
    
    This PR implements this sort of behaviour.
    Resolves: #46
    CPerezz committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    5f24446 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. Chore/sync upstream (#51)

    Sync upstream
    
    ---------
    
    Co-authored-by: François Garillot <4142+huitseeker@users.noreply.github.com>
    Co-authored-by: Srinath Setty <srinath@microsoft.com>
    3 people committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    a18afe0 View commit details
    Browse the repository at this point in the history