Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: kill Turbo #2442

Merged
merged 14 commits into from
Sep 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 0 additions & 14 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -191,18 +191,6 @@ jobs:
command: cond_spot_run_test barretenberg-x86_64-linux-clang-assert 32 ./scripts/run_tests 1 proof_system_tests
- *save_logs

barretenberg-stdlib-recursion-turbo-tests:
docker:
- image: aztecprotocol/alpine-build-image
resource_class: small
steps:
- *checkout
- *setup_env
- run:
name: "Test"
command: cond_spot_run_test barretenberg-x86_64-linux-clang-assert 32 ./scripts/run_tests 1 stdlib_recursion_tests --gtest_filter=*turbo*
- *save_logs

barretenberg-stdlib-recursion-ultra-tests:
docker:
- image: aztecprotocol/alpine-build-image
Expand Down Expand Up @@ -1040,14 +1028,12 @@ workflows:
- barretenberg-dsl-tests: *bb_test
- barretenberg-tests: *bb_test
- barretenberg-stdlib-tests: *bb_test
- barretenberg-stdlib-recursion-turbo-tests: *bb_test
- barretenberg-stdlib-recursion-ultra-tests: *bb_test
- barretenberg-join-split-tests: *bb_test
# - barretenberg-benchmark-aggregator:
# requires:
# - barretenberg-tests
# - barretenberg-stdlib-tests
# - barretenberg-stdlib-recursion-turbo-tests
# - barretenberg-stdlib-recursion-ultra-tests
# - barretenberg-join-split-tests
# filters:
Expand Down
40 changes: 27 additions & 13 deletions barretenberg/cpp/docs/Fuzzing.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
# Fuzzing barretenberg

## Intro

We are gradually introducing fuzzing of various primitives into barretenberg, focusing first and foremost on in-cicruit types. If you are developing / patching a primitive and there is a fuzzer available for it, please take the time to update the fuzzer (if you've added new functionality) and run it for at least a few hours to increase security.

## Build

To build with standard clang:

```bash
cmake --preset fuzzing
cmake --build --preset fuzzing
```

Fuzzing build turns off building tests and benchmarks, since they are incompatible with libfuzzer interface.

To turn on address sanitizer add `-DADDRESS_SANITIZER=ON`. Note that address sanitizer can be used to explore crashes.
Expand All @@ -27,35 +31,43 @@ cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_C_COMPILER=<path to clang> -DCMA
```

## Currently supported
Currently we have fuzzers for bigfield, bit_array, bool, byte_array, field, safe_uint and uint. Each of them is available in 3 versions: StandardPlonk, TurboPlonk, ALL (differential fuzzing of 2 versions).

Currently we have fuzzers for bigfield, bit_array, bool, byte_array, field, safe_uint and uint. Each of them is available in 2 versions: StandardPlonk, ALL (differential fuzzing of 2 versions).
To compile all fuzzers just type `make`.

## Running the fuzzer

TODO: add information about saved testcases

You can simply run a fuzzer by executing the built executable, for example,

```bash
./bin/stdlib_primitives_bigfield_turbo_fuzzer
./bin/stdlib_primitives_bigfield_standard_fuzzer
```

This is useful if you added a feature/instruction or changed some logic and want to quickly test if there are any really bad bugs.
To run the fuzzer seriously, I'd recommend:

```bash
mkdir ../../../<fuzzer_type>_testcases;
mkdir crashes;
./bin/<fuzzer_executable> -timeout=1 -len_control=500 -workers=8 -jobs=8 -entropic=1 -shrink=1 -artifact_prefix=crashes/ -use_value_profile=1 ../../../<fuzzer_type>_testcases
```

You can watch the progress of the fuzzer in one of the generated logs fuzz-<number>.log
The purpose of each parameter:
+ -timeout=1 - If a testcase takes more than 1 second to execute, it will be treated as a crash
+ -len_control=500 - Slows down the increase of testcase size. Especially important for heavy classes like bigfield, keeps the number of executions per second at a decent rate
+ -worker=8 - The number of threads that can simultaneously execute testcases. Should be less or equal to the number of jobs
+ -jobs=8 - After how many crashes the fuzzer will stop fuzzing. If a crash is executed and the number of jobs is more than workers then the fuzzer will proceed to give the worker a new job. The 8/8 worker/job configuration ensures that the fuzzer will quit after 8 crashes and until the first crash all the workers are busy.
+ -entropic=1 - Entropic should be enabled by default, but in case it isn't, enable it. A better power schedule than the old one.
+ -shrink=1 - If a new testcase is encountered that has the same coverage as some previous one in the corpus and the testcase is smaller, replace the one in the corpus with the new one. Helps keep exec/s higher.
+ -artifact_prefix=crashes/ - Where to save crashes/timeouts/ooms.
+ -use_value_profile=1 - Leverage libfuzzer internal CMP analysis. Very useful, but blows the corpus up.
+ <PATH_TO_CORPUS> (../../../<fuzzer_type>_testcases) - The path to the folder, where corpus testcases are going to be saved and loaded from (also loads testcases from there at the start of fuzzing).

Log structure is described here https://llvm.org/docs/LibFuzzer.html

- -timeout=1 - If a testcase takes more than 1 second to execute, it will be treated as a crash
- -len_control=500 - Slows down the increase of testcase size. Especially important for heavy classes like bigfield, keeps the number of executions per second at a decent rate
- -worker=8 - The number of threads that can simultaneously execute testcases. Should be less or equal to the number of jobs
- -jobs=8 - After how many crashes the fuzzer will stop fuzzing. If a crash is executed and the number of jobs is more than workers then the fuzzer will proceed to give the worker a new job. The 8/8 worker/job configuration ensures that the fuzzer will quit after 8 crashes and until the first crash all the workers are busy.
- -entropic=1 - Entropic should be enabled by default, but in case it isn't, enable it. A better power schedule than the old one.
- -shrink=1 - If a new testcase is encountered that has the same coverage as some previous one in the corpus and the testcase is smaller, replace the one in the corpus with the new one. Helps keep exec/s higher.
- -artifact_prefix=crashes/ - Where to save crashes/timeouts/ooms.
- -use_value_profile=1 - Leverage libfuzzer internal CMP analysis. Very useful, but blows the corpus up.
- <PATH_TO_CORPUS> (../../../<fuzzer_type>\_testcases) - The path to the folder, where corpus testcases are going to be saved and loaded from (also loads testcases from there at the start of fuzzing).

Log structure is described here https://llvm.org/docs/LibFuzzer.html

If you've found an issue, stopped the fuzzer, you can minimize the corpus to get rid of repetitions and then start from a minimized corpus

Expand All @@ -67,10 +79,12 @@ cp ../../../<fuzzer_type>_testcases_minimized/* ../../../<fuzzer_type>_testcases
```

If you've found a crash, you can minimize the crash to make the root cause more obvious:

```bash
mkdir minimized_crashes
./bin/<fuzzer_executable> -minimize_crash=1 -artifact_prefix=minimized_crashes <crash_file>
```

Also, both bigfield and safeuint fuzzer containt the SHOW_INFORMATION preprocessor cases, which enable the printing of instructions and values to make debugging the crash easier.

# Coverage reports
Expand Down
1 change: 0 additions & 1 deletion barretenberg/cpp/src/barretenberg/barretenberg.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
#include "numeric/uint256/uint256.hpp"
#include "plonk/proof_system/types/proof.hpp"
#include "plonk/proof_system/verification_key/verification_key.hpp"
#include "proof_system/circuit_builder/turbo_circuit_builder.hpp"
#include "proof_system/circuit_builder/ultra_circuit_builder.hpp"
#include "proof_system/types/circuit_type.hpp"
// TODO(https://github.com/AztecProtocol/barretenberg/issues/491):
Expand Down
4 changes: 0 additions & 4 deletions barretenberg/cpp/src/barretenberg/common/fuzzer.hpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
#pragma once
#include "barretenberg/numeric/uint256/uint256.hpp"
#include "barretenberg/proof_system/circuit_builder/standard_circuit_builder.hpp"
#include "barretenberg/proof_system/circuit_builder/turbo_circuit_builder.hpp"
#include <concepts>

// NOLINTBEGIN(cppcoreguidelines-macro-usage, google-runtime-int)
Expand Down Expand Up @@ -693,9 +692,6 @@ constexpr void RunWithBuilders(const uint8_t* Data, const size_t Size, FastRando
if (Composers & 1) {
RunWithBuilder<Fuzzer, proof_system::StandardCircuitBuilder>(Data, Size, VarianceRNG);
}
if (Composers & 2) {
RunWithBuilder<Fuzzer, proof_system::TurboCircuitBuilder>(Data, Size, VarianceRNG);
}
}

// NOLINTEND(cppcoreguidelines-macro-usage, google-runtime-int)
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
#pragma once
#include <cstdint>

enum CircuitType : uint64_t {
Standard = 1 << 0,
Turbo = 1 << 1,
};
enum CircuitType : uint64_t { Standard = 1 << 0 };
14 changes: 4 additions & 10 deletions barretenberg/cpp/src/barretenberg/dsl/types.hpp
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
#pragma once
#include "barretenberg/plonk/composer/turbo_composer.hpp"
#include "barretenberg/plonk/composer/ultra_composer.hpp"

#include "barretenberg/plonk/proof_system/prover/prover.hpp"
#include "barretenberg/proof_system/circuit_builder/turbo_circuit_builder.hpp"
#include "barretenberg/stdlib/commitment/pedersen/pedersen.hpp"
#include "barretenberg/stdlib/commitment/pedersen/pedersen_plookup.hpp"
#include "barretenberg/stdlib/encryption/schnorr/schnorr.hpp"
Expand All @@ -30,15 +28,11 @@ namespace acir_format {
using Builder = proof_system::UltraCircuitBuilder;
using Composer = plonk::UltraComposer;

using Prover = std::conditional_t<
std::same_as<Composer, plonk::UltraComposer>,
plonk::UltraWithKeccakProver,
std::conditional_t<std::same_as<Composer, plonk::TurboComposer>, plonk::TurboProver, plonk::Prover>>;
using Prover =
std::conditional_t<std::same_as<Composer, plonk::UltraComposer>, plonk::UltraWithKeccakProver, plonk::Prover>;

using Verifier = std::conditional_t<
std::same_as<Composer, plonk::UltraComposer>,
plonk::UltraWithKeccakVerifier,
std::conditional_t<std::same_as<Composer, plonk::TurboComposer>, plonk::TurboVerifier, plonk::Verifier>>;
using Verifier =
std::conditional_t<std::same_as<Composer, plonk::UltraComposer>, plonk::UltraWithKeccakVerifier, plonk::Verifier>;

using RecursiveProver = plonk::UltraProver;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -368,54 +368,48 @@ TEST_F(UltraHonkComposerTests, non_trivial_tag_permutation_and_cycles)

TEST_F(UltraHonkComposerTests, bad_tag_permutation)
{
auto circuit_builder = proof_system::UltraCircuitBuilder();
fr a = fr::random_element();
fr b = -a;

auto a_idx = circuit_builder.add_variable(a);
auto b_idx = circuit_builder.add_variable(b);
auto c_idx = circuit_builder.add_variable(b);
auto d_idx = circuit_builder.add_variable(a + 1);
{
auto circuit_builder = proof_system::UltraCircuitBuilder();
fr a = fr::random_element();
fr b = -a;

circuit_builder.create_add_gate({ a_idx, b_idx, circuit_builder.zero_idx, 1, 1, 0, 0 });
circuit_builder.create_add_gate({ c_idx, d_idx, circuit_builder.zero_idx, 1, 1, 0, -1 });
auto a_idx = circuit_builder.add_variable(a);
auto b_idx = circuit_builder.add_variable(b);
auto c_idx = circuit_builder.add_variable(b);
auto d_idx = circuit_builder.add_variable(a + 1);

circuit_builder.create_tag(1, 2);
circuit_builder.create_tag(2, 1);
circuit_builder.create_add_gate({ a_idx, b_idx, circuit_builder.zero_idx, 1, 1, 0, 0 });
circuit_builder.create_add_gate({ c_idx, d_idx, circuit_builder.zero_idx, 1, 1, 0, -1 });

circuit_builder.assign_tag(a_idx, 1);
circuit_builder.assign_tag(b_idx, 1);
circuit_builder.assign_tag(c_idx, 2);
circuit_builder.assign_tag(d_idx, 2);
circuit_builder.create_tag(1, 2);
circuit_builder.create_tag(2, 1);

auto composer = UltraComposer();
prove_and_verify(circuit_builder, composer, /*expected_result=*/false);
}
circuit_builder.assign_tag(a_idx, 1);
circuit_builder.assign_tag(b_idx, 1);
circuit_builder.assign_tag(c_idx, 2);
circuit_builder.assign_tag(d_idx, 2);

// same as above but with turbocomposer to check reason of failue is really tag mismatch
TEST_F(UltraHonkComposerTests, bad_tag_turbo_permutation)
{
auto circuit_builder = proof_system::UltraCircuitBuilder();
fr a = fr::random_element();
fr b = -a;
auto composer = UltraComposer();
prove_and_verify(circuit_builder, composer, /*expected_result=*/false);
}
// Same as above but without tag creation to check reason of failure is really tag mismatch
{
auto circuit_builder = proof_system::UltraCircuitBuilder();
fr a = fr::random_element();
fr b = -a;

auto a_idx = circuit_builder.add_variable(a);
auto b_idx = circuit_builder.add_variable(b);
auto c_idx = circuit_builder.add_variable(b);
auto d_idx = circuit_builder.add_variable(a + 1);
auto a_idx = circuit_builder.add_variable(a);
auto b_idx = circuit_builder.add_variable(b);
auto c_idx = circuit_builder.add_variable(b);
auto d_idx = circuit_builder.add_variable(a + 1);

circuit_builder.create_add_gate({ a_idx, b_idx, circuit_builder.zero_idx, 1, 1, 0, 0 });
circuit_builder.create_add_gate({ c_idx, d_idx, circuit_builder.zero_idx, 1, 1, 0, -1 });
circuit_builder.create_add_gate({ a_idx, b_idx, circuit_builder.zero_idx, 1, 1, 0, 0 });
circuit_builder.create_add_gate({ c_idx, d_idx, circuit_builder.zero_idx, 1, 1, 0, -1 });

auto composer = UltraComposer();
// circuit_builder.create_add_gate({ a_idx, b_idx, circuit_builder.zero_idx, fr::one(), fr::neg_one(),
// fr::zero(), fr::zero() }); circuit_builder.create_add_gate({ a_idx, b_idx, circuit_builder.zero_idx,
// fr::one(), fr::neg_one(), fr::zero(), fr::zero() }); circuit_builder.create_add_gate({ a_idx, b_idx,
// circuit_builder.zero_idx, fr::one(), fr::neg_one(), fr::zero(), fr::zero() });
// auto prover = composer.create_prover(circuit_builder);
// auto verifier = composer.create_verifier(circuit_builder);
auto composer = UltraComposer();

prove_and_verify(circuit_builder, composer, /*expected_result=*/true);
prove_and_verify(circuit_builder, composer, /*expected_result=*/true);
}
}

TEST_F(UltraHonkComposerTests, sort_widget)
Expand Down
12 changes: 3 additions & 9 deletions barretenberg/cpp/src/barretenberg/join_split_example/types.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

#include "barretenberg/honk/composer/ultra_composer.hpp"
#include "barretenberg/plonk/composer/standard_composer.hpp"
#include "barretenberg/plonk/composer/turbo_composer.hpp"
#include "barretenberg/plonk/composer/ultra_composer.hpp"

#include "barretenberg/plonk/proof_system/prover/prover.hpp"
Expand All @@ -21,15 +20,10 @@ namespace join_split_example {
using Builder = proof_system::UltraCircuitBuilder;
using Composer = plonk::UltraComposer;

using Prover = std::conditional_t<
std::same_as<Composer, plonk::UltraComposer>,
plonk::UltraProver,
std::conditional_t<std::same_as<Composer, plonk::TurboComposer>, plonk::TurboProver, plonk::Prover>>;
using Prover = std::conditional_t<std::same_as<Composer, plonk::UltraComposer>, plonk::UltraProver, plonk::Prover>;

using Verifier = std::conditional_t<
std::same_as<Composer, plonk::UltraComposer>,
plonk::UltraVerifier,
std::conditional_t<std::same_as<Composer, plonk::TurboComposer>, plonk::TurboVerifier, plonk::Verifier>>;
using Verifier =
std::conditional_t<std::same_as<Composer, plonk::UltraComposer>, plonk::UltraVerifier, plonk::Verifier>;

using witness_ct = proof_system::plonk::stdlib::witness_t<Builder>;
using public_witness_ct = proof_system::plonk::stdlib::public_witness_t<Builder>;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ void compute_monomial_and_coset_selector_forms(plonk::proving_key* circuit_provi
barretenberg::polynomial selector_poly_fft(selector_poly, circuit_proving_key->circuit_size * 4 + 4);
selector_poly_fft.coset_fft(circuit_proving_key->large_domain);

// TODO(luke): For Standard/Turbo, the lagrange polynomials can be removed from the store at this point but this
// TODO(luke): For Standard, the lagrange polynomials can be removed from the store at this point but this
// is not the case for Ultra. Implement?
circuit_proving_key->polynomial_store.put(selector_properties[i].name, std::move(selector_poly));
circuit_proving_key->polynomial_store.put(selector_properties[i].name + "_fft", std::move(selector_poly_fft));
Expand Down
Loading