Skip to content

Commit

Permalink
Document LIBAFL_DEBUG_OUTPUT in Launcher (#1485)
Browse files Browse the repository at this point in the history
* Document LIBAFL_DEBUG_OUTPUT in Launcher

* fmt

* more doc

* fork

* unix
  • Loading branch information
domenukk authored Aug 29, 2023
1 parent 51e4d81 commit 5710c8b
Show file tree
Hide file tree
Showing 4 changed files with 21 additions and 8 deletions.
2 changes: 1 addition & 1 deletion docs/src/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@ feel free to use and mutate an Abstract Syntax Tree instead, for structured fuzz
- `scalable`: As part of LibAFL, we developed `Low Level Message Passing`, `LLMP` for short, which allows LibAFL to scale almost linearly over cores. That is, if you chose to use this feature - it is your fuzzer, after all.
Scaling to multiple machines over TCP is also possible, using LLMP's `broker2broker` feature.
- `fast`: We do everything we can at compile time so that the runtime overhead is as minimal as it can get.
- `bring your own target`: We support binary-only modes, like QEMU-Mode and Frida-Mode with ASAN and CmpLog, as well as multiple compilation passes for sourced-based instrumentation.
- `bring your own target`: We support binary-only modes, like (full-system) QEMU-Mode and Frida-Mode with ASan and CmpLog, as well as multiple compilation passes for sourced-based instrumentation.
Of course, we also support custom instrumentation, as you can see in the Python example based on Google's Atheris.
- `usable`: This one is on you to decide. Dig right in!
2 changes: 1 addition & 1 deletion docs/src/message_passing/configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Configurations for individual fuzzer nodes are relevant for multi node fuzzing.
The chapter describes how to run nodes with different configurations
in one fuzzing cluster.
This allows, for example, a node compiled with ASAN, to know that it needs to rerun new testcases for a node without ASAN, while the same binary/configuration does not.
This allows, for example, a node compiled with ASan, to know that it needs to rerun new testcases for a node without ASan, while the same binary/configuration does not.

Fuzzers with the same configuration can exchange Observers for new testcases and reuse them without rerunning the input.
A different configuration indicates, that only the raw input can be exchanged, it must be rerun on the other node to capture relevant observations.
8 changes: 6 additions & 2 deletions docs/src/message_passing/spawn_instances.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Multiple fuzzer instances can be spawned using different ways.

## Manually, via a TCP port

The straightforward way to do Multi-Threading is to use the `LlmpRestartingEventManager`, specifically to use `setup_restarting_mgr_std`.
The straightforward way to do Multi-Threading is to use the [`LlmpRestartingEventManager`](https://docs.rs/libafl/latest/libafl/events/llmp/struct.LlmpRestartingEventManager.html), specifically to use [`setup_restarting_mgr_std`](https://docs.rs/libafl/latest/libafl/events/llmp/fn.setup_restarting_mgr_std.html).
It abstracts away all the pesky details about restarts on crash handling (for in-memory fuzzers) and multi-threading.
With it, every instance you launch manually tries to connect to a TCP port on the local machine.

Expand All @@ -13,7 +13,7 @@ If the port is not yet bound, this instance becomes the broker, binding itself t
If the port is already bound, the EventManager will try to connect to it.
The instance becomes a client and can now communicate with all other nodes.

Launching nodes manually has the benefit that you can have multiple nodes with different configurations, such as clients fuzzing with and without ASAN.
Launching nodes manually has the benefit that you can have multiple nodes with different configurations, such as clients fuzzing with and without `ASan``.

While it's called "restarting" manager, it uses `fork` on Unix-like operating systems as optimization and only actually restarts from scratch on Windows.

Expand Down Expand Up @@ -42,13 +42,17 @@ To use launcher, first you need to write an anonymous function `let mut run_clie
This first starts a broker, then spawns `n` clients, according to the value passed to `cores`.
The value is a string indicating the cores to bind to, for example, `0,2,5` or `0-3`.
For each client, `run_client` will be called.
If the launcher uses `fork`, it will hide child output, unless the settings indicate otherwise, or the `LIBAFL_DEBUG_OUTPUT` env variable is set.
On Windows, the Launcher will restart each client, while on Unix-alikes, it will use `fork`.

Advanced use-cases:

1. To connect multiple nodes together via TCP, you can use the `remote_broker_addr`. this requires the `llmp_bind_public` compile-time feature for `LibAFL`.
2. To use multiple launchers for individual configurations, you can set `spawn_broker` to `false` on all instances but one.
3. Launcher will not select the cores automatically, so you need to specify the `cores` that you want.
4. On `Unix`, you can chose between a forking and non-forking version of Launcher by setting the `fork` feature in LibAFL. Some targets may not like forking, but it is faster than restarting processes from scratch. Windows will never fork.
5. For simple debugging, first set the `LIBAFL_DEBUG_OUTPUT` env variable to see if a child process printed anything.
6. For further debugging of fuzzer failures, it may make sense to replace `Launcher` temporarily with a [`SimpleEventManager`](https://docs.rs/libafl/latest/libafl/events/simple/struct.SimpleEventManager.html#method.new) and call your harness fn (`run_client(None, mgr, 0);`) directly, so that fuzzing runs in the same thread and is easier to debug, before moving back to `Launcher` after the bugfix.

For more examples, you can check out `qemu_launcher` and `libfuzzer_libpng_launcher` in [`./fuzzers/`](https://github.com/AFLplusplus/LibAFL/tree/main/fuzzers).

Expand Down
17 changes: 13 additions & 4 deletions libafl/src/events/launcher.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
//! The [`Launcher`] launches multiple fuzzer instances in parallel.
//! Thanks to it, we won't need a `for` loop in a shell script...
//!
//! It will hide child output, unless the settings indicate otherwise, or the `LIBAFL_DEBUG_OUTPUT` env variable is set.
//!
//! To use multiple [`Launcher`]`s` for individual configurations,
//! we can set `spawn_broker` to `false` on all but one.
//!
Expand Down Expand Up @@ -54,7 +56,13 @@ use crate::{
/// The (internal) `env` that indicates we're running as client.
const _AFL_LAUNCHER_CLIENT: &str = "AFL_LAUNCHER_CLIENT";

/// Provides a Launcher, which can be used to launch a fuzzing run on a specified list of cores
/// The env variable to set in order to enable child output
#[cfg(all(feature = "fork", unix))]
const LIBAFL_DEBUG_OUTPUT: &str = "LIBAFL_DEBUG_OUTPUT";

/// Provides a [`Launcher`], which can be used to launch a fuzzing run on a specified list of cores
///
/// Will hide child output, unless the settings indicate otherwise, or the `LIBAFL_DEBUG_OUTPUT` env variable is set.
#[cfg(feature = "std")]
#[allow(
clippy::type_complexity,
Expand Down Expand Up @@ -168,7 +176,7 @@ where
.map(|filename| File::create(filename).unwrap());

#[cfg(feature = "std")]
let debug_output = std::env::var("LIBAFL_DEBUG_OUTPUT").is_ok();
let debug_output = std::env::var(LIBAFL_DEBUG_OUTPUT).is_ok();

// Spawn clients
let mut index = 0_u64;
Expand Down Expand Up @@ -277,7 +285,8 @@ where
Ok(core_conf) => {
let core_id = core_conf.parse()?;

//todo: silence stdout and stderr for clients
// TODO: silence stdout and stderr for clients
// let debug_output = std::env::var(LIBAFL_DEBUG_OUTPUT).is_ok();

// the actual client. do the fuzzing
let (state, mgr) = RestartingMgr::<MT, S, SP>::builder()
Expand Down Expand Up @@ -493,7 +502,7 @@ where
.stderr_file
.map(|filename| File::create(filename).unwrap());

let debug_output = std::env::var("LIBAFL_DEBUG_OUTPUT").is_ok();
let debug_output = std::env::var(LIBAFL_DEBUG_OUTPUT).is_ok();

// Spawn centralized broker
self.shmem_provider.pre_fork()?;
Expand Down

0 comments on commit 5710c8b

Please sign in to comment.