Add support for AFL

[Manishearth]

Would be nice if we could cargo fuzz --fuzzer afl on binary crates. Or something. Might need to set up the whole LLVM shebang.

cc @frewsxcv

[frewsxcv]

This would be pretty easy to do if we figured out https://github.com/rust-fuzz/libfuzzer-sys/issues/5

[frewsxcv]

I think the best strategy here is to make a change to rustc to add in the afl LLVM pass behind a rustc debug flag.

Relevant IRC rust-internals discussion: https://botbot.me/mozilla/rust-internals/2017-02-26/?msg=81593902&page=3

https://github.com/rust-lang/rust/tree/master/src/rustllvm

https://github.com/rust-lang/rust/tree/master/src/librustc_llvm

[whitequark]

What's the advantage of afl over libfuzzer?

[nagisa]

@whitequark the only one I know is that it, being a out-of-process fuzzer, can trivially run even in presence of crashes and avoids giving the crashing inputs it has already seen.

libfuzzer struggles with that use-case, sadly.

[frewsxcv]

What's the advantage of afl over libfuzzer?

In my opinion, the UI is a lot more informative and easier to understand: http://lcamtuf.coredump.cx/afl/status_screen.txt . Also from what I've seen, there's a larger ecosystem of tools built around AFL.

[frewsxcv]

okay, incorporating afl into cargo-fuzz is now doable.

yesterday, i opened this issue in rust-lang/rust. i was informed by @alex in that thread that it's now possible to use afl without relying on afl's own llvm pass and instead use llvm's trace-pc-guard feature. i wasn't aware of this, tried it out this morning, and got it working!

in particular, here's how to get it setup:

1. compile afl with the trace-pc feature
   1. download afl
   2. AFL_TRACE_PC=1 make clean all as per this
2. compile the afl runtime
   1. (from the afl source directory)
   2. gcc -c -O1 -fPIC -fno-omit-frame-pointer llvm_mode/afl-llvm-rt.o.c -fpermissive (copied some of these flags from oss-fuzz)
   3. ar r libafl-llvm-rt.a afl-llvm-rt.o.o
3. setup/compile a fuzz target
   1. just to demonstrate that this all works, just cargo new --bin foo printing 'hello world' is sufficient to show that afl can even run, albeit not very interesting
   2. export RUSTFLAGS="-Cllvm-args=-sanitizer-coverage-level=3 -Cllvm-args=-sanitizer-coverage-trace-pc-guard -Cpasses=sancov"
   3. cargo rustc -- -l afl-llvm-rt -L <directory where libafl-llvm-rt.a is>
4. run afl 🎉
   1. <afl source directory>/afl-fuzz -i <corpus dir> -o out target/debug/<crate name>

the main work left here is the integration into cargo-fuzz

some questions:

• in a world where cargo-fuzz can handle multiple fuzzers, what is the directory structure in fuzz/? for example, where should the output of afl go in the fuzz directory?
• if we want to automate most of the above steps, we'll need to handle afl compilation ourselves since we need to enable that AFL_TRACE_PC flag. compilation requirements are pretty minimal, but anyone have thoughts on the best way to organize this in cargo-fuzz? we'll need to compile afl-fuzz, build the runtime, and then use the outputs of both those steps when building the fuzz target

some thoughts:

• unlike libfuzzer, afl-fuzz relies on bytes coming in through stdin, so we'll need to update our fuzz_target macro to account for that
• can't get this to work on mac. upon attempting to run the cargo rustc command, i hit Invalid section names in Mach-O file on OS X w/ custom target spec rust-lang/rust#22915 . LLVM ERROR: Global variable '__sancov_gen_' has an invalid section specifier '__sancov_guards': mach-o section specifier requires a segment and section separated by a comma.

[frewsxcv]

Published afl.rs 0.2 yesterday: https://users.rust-lang.org/t/announcing-afl-rs-0-2-bindings-for-american-fuzzy-lop/13981 which should make it easier to complete this issue now

[Manishearth]

in a world where cargo-fuzz can handle multiple fuzzers, what is the directory structure in fuzz/? for example, where should the output of afl go in the fuzz directory?

We have an artifacts/ and corpus/ folder that turn up there. We can make the artifacts/ folder into artifacts/libfuzzer and artifacts/afl if we wish.

unlike libfuzzer, afl-fuzz relies on bytes coming in through stdin, so we'll need to update our fuzz_target macro to account for that

Can we make the macro paper over this to provide the same API?

[Manishearth]

Alternatively we can make AFL work on cargo examples directly.