Skip to content

Commit

Permalink
📝 address review comments
Browse files Browse the repository at this point in the history
  • Loading branch information
nlohmann committed May 8, 2022
1 parent 13e9ab2 commit f92e6d0
Showing 1 changed file with 14 additions and 12 deletions.
26 changes: 14 additions & 12 deletions tests/fuzzing.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Fuzz testing

Each parser of the library (JSON, BJData, BSON, CBOR, MessagePack, and UBJSON) can be fuzz tested. Currently,
[LibFuzzer](https://llvm.org/docs/LibFuzzer.html) and [afl++](https://github.com/AFLplusplus/AFLplusplus) are supported.
[libFuzzer](https://llvm.org/docs/LibFuzzer.html) and [afl++](https://github.com/AFLplusplus/AFLplusplus) are supported.

## Corpus creation

For most effective fuzzing, a [corpus](https://llvm.org/docs/LibFuzzer.html#corpus) should be provided. A corpus is a
directory with some simple input files that cover several features of the parser and is hence a good starting point
directory with some simple input files that cover several features of the parser and is hence a good starting point
for mutations.

```shell
Expand All @@ -16,18 +16,19 @@ unzip v$TEST_DATA_VERSION.zip
rm v$TEST_DATA_VERSION.zip
for FORMAT in json bjdata bson cbor msgpack ubjson
do
rm -fr mkdir corpus_$FORMAT
rm -fr corpus_$FORMAT
mkdir corpus_$FORMAT
find json_test_data-$TEST_DATA_VERSION -size -5k -name "*.$FORMAT" -exec cp "{}" "corpus_$FORMAT" \;
done
rm -fr json_test_data-$TEST_DATA_VERSION
```

The generated corpus can be used with both LibFuzzer and afl++.
The generated corpus can be used with both libFuzzer and afl++. The remainder of this documentation assumes the corpus
directories have been created in the `tests` directory.

## LibFuzzer
## libFuzzer

To use LibFuzzer, you need to pass `-fsanitize=fuzzer` as `FUZZER_ENGINE`:
To use libFuzzer, you need to pass `-fsanitize=fuzzer` as `FUZZER_ENGINE`. In the `tests` directory, call

```shell
make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer"
Expand All @@ -36,14 +37,15 @@ make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer"
This creates a fuzz tester binary for each parser that supports these
[command line options](https://llvm.org/docs/LibFuzzer.html#options).

Note the compiler provided by Xcode (AppleClang) does not contain libFuzzer. Please install Clang via Homebrew calling
`brew install llvm` and add `CXX=$(brew --prefix llvm)/bin/clang` to the `make` call:
In case your default compiler is not a Clang compiler that includes libFuzzer (Clang 6.0 or later), you need to set the
`CXX` variable accordingly. Note the compiler provided by Xcode (AppleClang) does not contain libFuzzer. Please install
Clang via Homebrew calling `brew install llvm` and add `CXX=$(brew --prefix llvm)/bin/clang` to the `make` call:

```shell
make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer" CXX=$(brew --prefix llvm)/bin/clang
```

Then pass the corpus directory as command-line argument:
Then pass the corpus directory as command-line argument (assuming it is located in `tests`):

```shell
./parse_cbor_fuzzer corpus_cbor
Expand All @@ -55,13 +57,13 @@ a file starting with `crash-`.
## afl++

To use afl++, you need to pass `-fsanitize=fuzzer` as `FUZZER_ENGINE`. It will be replaced by a `libAFLDriver.a` to
re-use the same code written for LibFuzzer with afl++. Furthermore, set `afl-clang-fast++` as compiler.
re-use the same code written for libFuzzer with afl++. Furthermore, set `afl-clang-fast++` as compiler.

```shell
CXX=afl-clang-fast++ make fuzzers FUZZER_ENGINE="-fsanitize=fuzzer"
```

Then the fuzzer is called like this:
Then the fuzzer is called like this in the `tests` directory:

```shell
afl-fuzz -i corpus_cbor -o out -- ./parse_cbor_fuzzer
Expand All @@ -76,4 +78,4 @@ The library is further fuzz-tested 24/7 by Google's [OSS-Fuzz project](https://g
the same `fuzzers` target as above and also relies on the `FUZZER_ENGINE` variable. See the used
[build script](https://github.com/google/oss-fuzz/blob/master/projects/json/build.sh) for more information.

In case the build at OSS-Fuzz fails, and issue will be created automatically.
In case the build at OSS-Fuzz fails, an issue will be created automatically.

0 comments on commit f92e6d0

Please sign in to comment.