Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the 3x GPU prover #356

Merged
merged 36 commits into from
Aug 28, 2019
Merged

Add the 3x GPU prover #356

merged 36 commits into from
Aug 28, 2019

Conversation

mrmr1993
Copy link
Member

@mrmr1993 mrmr1993 commented Aug 19, 2019

This PR

  • pulls in the 3x GPU prover code (with changes to link it to OCaml, see this branch)
    • all of the new C++ and CUDA files are copied directly from there, the only changes should be to #include paths
    • some of the CMakeLists.txts, dune files and .ml files were changed to make things build
  • restructures the build system so that all of the C++/CUDA code lives beneath src/camlsnark_c/libsnark-caml
  • adds a meta-package to signal that CUDA is installed
    • this lives in ./snarky_cuda, can be installed with opam pin ./snarky_cuda from the project root

There is a test that can be run with dune exec src/camlsnark_c/groth16_gpu_prover/test/test.exe.

NOTE: This PR doesn't integrate the prover with the snarky API yet, but it should be possible to test it using generate_witness (from #322) to get the input and using the preprocessing and proving functions in Groth16_gpu_prover.
Update: This is no longer true, see the comment below.

@mrmr1993
Copy link
Member Author

Updated to compute the FFT domains properly and to add a GPU example (run with make examples-gpu)

@mrmr1993
Copy link
Member Author

This comment was supposed to be posted before, but I forgot to press the comment button 😑

Updated. This now includes a conditional module in snarky that extends a normal Snark_intf.Basic/Snark_intf.S to introduce the functions relevant to the GPU prover.

The basic structure now is:

  • the C++ and CUDA code all lives beneath src/camlsnark_c/libsnark-caml
    • GPU code is conditional on CUDA support via the CMakeLists.txt
    • cuda_prover has the C++ bindings for OCaml and the toplevel CUDA code
    • cuda-fixnum has the CUDA multiexp functions
    • prover_reference_functions has the C++ functions from the reference implementation, tweaked to support the CUDA code and OCaml bindings
  • there is a new 'library' snarky_cuda in the folder of the same name
    • this library is empty, designed to be opam pin-ed to tell snarky to rebuild with CUDA support
    • this is hidden from dune with data_only_dirs in the toplevel dune file, so that dune doesn't think it exists before it's pinned
    • it should fail to install (on Ubuntu and Debian) if cuda isn't installed
  • there is a new stubs builder in src/camlsnark_c/dune for the GPU code
    • this builds the CUDA code from the 3x GPU prover and the C++ stubs to link it to OCaml
    • this is conditional on the presence of the snarky_cuda package in opam
  • there is a new library groth16_gpu_prover in src/camlsnark_c/groth16_gpu_prover
    • this binds the stubs to OCaml functions (using the libffi ctypes API), using a functor over a partial Snarky.Libsnark to ensure that the opaque types (Field.t, G1.t, G2.t, Proof.t, etc.) match
    • this is also conditional on the presence of snarky_cuda ((optional) in dune parlance)
  • the main snarky dune file (src/dune) has a new 'alternative dependency' clause that selects between a no_gpu.ml and gpu.ml file, depending on whether groth16_gpu_prover is present or not
    • these have to be explicitly omitted from the snarky library's modules to avoid unconditionally building them both (and failing when groth16_gpu_prover wasn't build)
  • gpu.ml implements some functors that extend Snark_intf.Basic/Snark_intf.S with preprocessing and gpu-proving functions.

assert (Snarky.Libsnark.Mnt6753.Default.Proof.verify proof key public_input) ;
ignore proof

(* Test for the original gpu prover competition output. Disabled. *)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be deleted?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was keeping this around in case Hamish wanted to keep testing using his old set-up. Very happy to remove this if we think he should just be interfacing with the OCaml code (or something else) instead.

@mrmr1993 mrmr1993 merged commit 6d0df48 into master Aug 28, 2019
@mrmr1993 mrmr1993 deleted the feature/gpu-prover-support branch August 28, 2019 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants