Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor InProcessExecutor, merge timeout executors #1789

Merged
merged 104 commits into from
Jan 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
047c392
move windows, inprocess fork to a different file, try new hook mechan…
tokatoka Jan 10, 2024
6ec89ce
fix
tokatoka Jan 11, 2024
4782df6
even more
tokatoka Jan 11, 2024
fc74ec0
more
tokatoka Jan 11, 2024
89eb38a
more
tokatoka Jan 11, 2024
6d7c08f
fix
tokatoka Jan 12, 2024
76c7080
fix
tokatoka Jan 12, 2024
fd4f661
macosgit add -ugit add -u
tokatoka Jan 12, 2024
a091f4b
windows!
tokatoka Jan 12, 2024
0f75463
windows!
tokatoka Jan 12, 2024
0709f93
aa
tokatoka Jan 12, 2024
c6f250f
aa
tokatoka Jan 12, 2024
2ec674a
macos
tokatoka Jan 12, 2024
f0c0fbc
std
tokatoka Jan 12, 2024
a2e2ae2
wtf unresolved?
tokatoka Jan 12, 2024
e0416d5
Copy, Clone
tokatoka Jan 12, 2024
1507896
why you just don't have the same API!
tokatoka Jan 12, 2024
5f14bff
inproc
tokatoka Jan 12, 2024
c69f367
next; inprocess
tokatoka Jan 12, 2024
cba8a81
windows?
tokatoka Jan 12, 2024
4da3ade
ci
tokatoka Jan 12, 2024
e55892c
ci
tokatoka Jan 12, 2024
7e59300
ci
tokatoka Jan 12, 2024
42e6369
unused
tokatoka Jan 12, 2024
c0bd738
ci
tokatoka Jan 12, 2024
c2974a8
unused
tokatoka Jan 12, 2024
4856b1d
no_std
tokatoka Jan 12, 2024
c4491f6
Merge branch 'main' of https://github.com/AFLplusplus/LibAFL into inp…
tokatoka Jan 12, 2024
11e95c0
Merge branch 'inproc_ref' of https://github.com/AFLplusplus/LibAFL in…
tokatoka Jan 12, 2024
9b61297
windows no std
tokatoka Jan 13, 2024
24b89dd
fix
tokatoka Jan 13, 2024
5d04b14
inprocess
tokatoka Jan 13, 2024
88c95fc
fix
tokatoka Jan 13, 2024
d4b786b
windows
tokatoka Jan 13, 2024
5c354ca
fuzzers
tokatoka Jan 13, 2024
c989e0d
macos , book
tokatoka Jan 13, 2024
8ffd858
fix
tokatoka Jan 13, 2024
ee80de5
aa
tokatoka Jan 13, 2024
334db4e
allow
tokatoka Jan 13, 2024
9cd3927
fix
tokatoka Jan 13, 2024
c62eefe
stop suggesting wrong lint AAAAAAAAAAAAAAAAA!!!
tokatoka Jan 14, 2024
aabc0ff
stop suggesting wrong lint AAAAAAAAAAAAAAAAA!!!
tokatoka Jan 14, 2024
6e1bf14
Merge branch 'main' into inproc_ref
tokatoka Jan 14, 2024
7723de3
win
tokatoka Jan 14, 2024
75968b4
fix
tokatoka Jan 14, 2024
bf6809a
wip
tokatoka Jan 15, 2024
caa0726
wip2
tokatoka Jan 15, 2024
6ade91d
windows done?
tokatoka Jan 15, 2024
6bdd738
Merge branch 'main' into inproc_ref
tokatoka Jan 17, 2024
2cdbc0b
remove TimeoutExecutor
tokatoka Jan 18, 2024
0d9726b
Merge branch 'inproc_ref' of github.com:AFLplusplus/LibAFL into inpro…
tokatoka Jan 18, 2024
60539d2
ci
tokatoka Jan 18, 2024
c57bedc
ci
tokatoka Jan 18, 2024
db94861
miri
tokatoka Jan 18, 2024
15fa5fa
fixfi
tokatoka Jan 18, 2024
8a4d4d0
compile on windows
tokatoka Jan 18, 2024
57b2a80
Merge branch 'main' of github.com:AFLplusplus/LibAFL into inproc_ref
tokatoka Jan 18, 2024
15cf451
a
tokatoka Jan 18, 2024
5199778
Merge branch 'inproc_ref' of github.com:AFLplusplus/LibAFL into inpro…
tokatoka Jan 18, 2024
25b176a
clp
tokatoka Jan 18, 2024
9a8f241
no_std stuff
tokatoka Jan 19, 2024
7ef3aad
windows no_std
tokatoka Jan 19, 2024
3949e9b
mac stuff
tokatoka Jan 19, 2024
8f72c06
Merge branch 'main' into inproc_ref
tokatoka Jan 19, 2024
c75cd9d
m
tokatoka Jan 19, 2024
292e53e
a
tokatoka Jan 19, 2024
663a0de
ci
tokatoka Jan 19, 2024
7eb20df
ci
tokatoka Jan 19, 2024
643d631
deleting timeoutexecutor, gradually
tokatoka Jan 19, 2024
8bf9af9
fucking macos
tokatoka Jan 19, 2024
c0f197a
ci
tokatoka Jan 19, 2024
db1af11
test
tokatoka Jan 19, 2024
e497254
ci
tokatoka Jan 19, 2024
58f785f
ci
tokatoka Jan 19, 2024
befeae4
batch mode constructor
tokatoka Jan 19, 2024
222b4e6
fix
tokatoka Jan 19, 2024
d50acf3
ci
tokatoka Jan 19, 2024
1e73d93
aa
tokatoka Jan 19, 2024
f384f1c
miri
tokatoka Jan 19, 2024
99d1d87
aaa
tokatoka Jan 19, 2024
37000b5
tmate again
tokatoka Jan 19, 2024
64d1cd4
fix windows stuff
tokatoka Jan 19, 2024
6b2d288
final fix
tokatoka Jan 19, 2024
20acc22
another win fix
tokatoka Jan 19, 2024
f8e300f
add
tokatoka Jan 19, 2024
4965c95
let's add the new fix later
tokatoka Jan 19, 2024
054007b
more
tokatoka Jan 19, 2024
eb01071
fi
tokatoka Jan 19, 2024
dd41bfd
parse
tokatoka Jan 20, 2024
f3bbc24
win clippy
tokatoka Jan 20, 2024
8c9e34a
win no std
tokatoka Jan 20, 2024
95babe2
safety
tokatoka Jan 20, 2024
f25159e
fix
tokatoka Jan 20, 2024
0677214
Merge branch 'inproc_ref' of github.com:AFLplusplus/LibAFL into inpro…
tokatoka Jan 20, 2024
4b6e2af
DEFAULT
tokatoka Jan 20, 2024
fbc1cfa
final fix
tokatoka Jan 21, 2024
4bf44f1
libafl_libfuzzer
tokatoka Jan 22, 2024
d67e6b5
comments
tokatoka Jan 22, 2024
d405208
fix
tokatoka Jan 23, 2024
45fe122
fix fuzzres
tokatoka Jan 23, 2024
9cf5fd6
fixxxxx
tokatoka Jan 23, 2024
3367eaa
fixxxxx
tokatoka Jan 23, 2024
6163ec4
last fix
tokatoka Jan 23, 2024
3dffea3
change name
tokatoka Jan 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,5 @@ libafl_nyx/packer
.gdb_history
# No llvm IR
*.ll

.tar.gz
15 changes: 11 additions & 4 deletions docs/listings/baby_fuzzer/listing-04/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
extern crate libafl;
extern crate libafl_bolts;

use std::path::PathBuf;

use libafl::{
corpus::{InMemoryCorpus, OnDiskCorpus},
events::SimpleEventManager,
Expand All @@ -13,8 +15,7 @@ use libafl::{
schedulers::QueueScheduler,
state::StdState,
};
use libafl_bolts::{current_nanos, rands::StdRand, AsSlice};
use std::path::PathBuf;
use libafl_bolts::{current_nanos, rands::StdRand, tuples::tuple_list, AsSlice};
/* ANCHOR_END: use */

fn main() {
Expand Down Expand Up @@ -70,8 +71,14 @@ fn main() {

/* ANCHOR: executor */
// Create the executor for an in-process function
let mut executor = InProcessExecutor::new(&mut harness, (), &mut fuzzer, &mut state, &mut mgr)
.expect("Failed to create the Executor");
let mut executor = InProcessExecutor::new(
&mut harness,
(),
&mut fuzzer,
&mut state,
&mut mgr,
)
.expect("Failed to create the Executor");
/* ANCHOR_END: executor */

/* ANCHOR: generator */
Expand Down
2 changes: 0 additions & 2 deletions docs/src/core_concepts/executor.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ In Rust, we bind this concept to the [`Executor`](https://docs.rs/libafl/latest/

By default, we implement some commonly used Executors such as [`InProcessExecutor`](https://docs.rs/libafl/latest/libafl/executors/inprocess/type.InProcessExecutor.html) in which the target is a harness function providing in-process crash detection. Another Executor is the [`ForkserverExecutor`](https://docs.rs/libafl/latest/libafl/executors/forkserver/struct.ForkserverExecutor.html) that implements an AFL-like mechanism to spawn child processes to fuzz.

A common pattern when creating an Executor is wrapping an existing one, for instance [`TimeoutExecutor`](https://docs.rs/libafl/latest/libafl/executors/timeout/struct.TimeoutExecutor.html) wraps an executor and installs a timeout callback before calling the original `run` function of the wrapped executor.

## InProcessExecutor
Let's begin with the base case; `InProcessExecutor`.
This executor executes the harness program (function) inside the fuzzer process.
Expand Down
3 changes: 2 additions & 1 deletion fuzzers/baby_fuzzer_with_forkexecutor/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#[cfg(windows)]
use std::ptr::write_volatile;
use std::{path::PathBuf, ptr::write};
use std::{path::PathBuf, ptr::write, time::Duration};

use libafl::{
corpus::{InMemoryCorpus, OnDiskCorpus},
Expand Down Expand Up @@ -110,6 +110,7 @@ pub fn main() {
&mut fuzzer,
&mut state,
&mut mgr,
core::time::Duration::from_millis(5000),
shmem_provider,
)
.expect("Failed to create the Executor");
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use std::path::PathBuf;
use std::{path::PathBuf, time::Duration};

use libafl::{
corpus::{InMemoryCorpus, OnDiskCorpus},
Expand Down Expand Up @@ -98,6 +98,7 @@ pub fn main() {
&mut fuzzer,
&mut state,
&mut mgr,
Duration::from_millis(5000),
shmem_provider,
)
.expect("Failed to create the Executor");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ pub fn main() {
&mut fuzzer,
&mut state,
&mut mgr,
core::time::Duration::from_millis(5000),
shmem_provider,
)
.expect("Failed to create the Executor");
Expand Down
26 changes: 12 additions & 14 deletions fuzzers/fuzzbench/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ use clap::{Arg, Command};
use libafl::{
corpus::{Corpus, InMemoryOnDiskCorpus, OnDiskCorpus},
events::SimpleRestartingEventManager,
executors::{inprocess::InProcessExecutor, ExitKind, TimeoutExecutor},
executors::{inprocess::InProcessExecutor, ExitKind},
feedback_or,
feedbacks::{CrashFeedback, MaxMapFeedback, TimeFeedback},
fuzzer::{Fuzzer, StdFuzzer},
Expand Down Expand Up @@ -327,29 +327,27 @@ fn fuzz(
let mut tracing_harness = harness;

// Create the executor for an in-process function with one observer for edge coverage and one for the execution time
let mut executor = TimeoutExecutor::new(
InProcessExecutor::new(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
)?,
let mut executor = InProcessExecutor::with_timeout(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
timeout,
);
)?;

// Setup a tracing stage in which we log comparisons
let tracing = TracingStage::new(TimeoutExecutor::new(
InProcessExecutor::new(
let tracing = TracingStage::new(
InProcessExecutor::with_timeout(
&mut tracing_harness,
tuple_list!(cmplog_observer),
&mut fuzzer,
&mut state,
&mut mgr,
timeout * 10,
)?,
// Give it more time!
timeout * 10,
));
);

// The order of the stages matter!
let mut stages = tuple_list!(calibration, tracing, i2s, power);
Expand Down
2 changes: 2 additions & 0 deletions fuzzers/fuzzbench_fork_qemu/src/fuzzer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ use std::{
io::{self, Write},
path::PathBuf,
process,
time::Duration,
};

use clap::{Arg, Command};
Expand Down Expand Up @@ -342,6 +343,7 @@ fn fuzz(
&mut state,
&mut mgr,
shmem_provider,
Duration::from_millis(5000),
)?;

// Show the cmplog observer
Expand Down
6 changes: 3 additions & 3 deletions fuzzers/fuzzbench_qemu/src/fuzzer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ use clap::{Arg, Command};
use libafl::{
corpus::{Corpus, InMemoryOnDiskCorpus, OnDiskCorpus},
events::SimpleRestartingEventManager,
executors::{ExitKind, ShadowExecutor, TimeoutExecutor},
executors::{ExitKind, ShadowExecutor},
feedback_or,
feedbacks::{CrashFeedback, MaxMapFeedback, TimeFeedback},
fuzzer::{Fuzzer, StdFuzzer},
Expand Down Expand Up @@ -351,17 +351,17 @@ fn fuzz(
),
);

// Create the executor for an in-process function with one observer for edge coverage and one for the execution time
let executor = QemuExecutor::new(
&mut hooks,
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
timeout,
)?;

// Create the executor for an in-process function with one observer for edge coverage and one for the execution time
let executor = TimeoutExecutor::new(executor, timeout);
// Show the cmplog observer
let mut executor = ShadowExecutor::new(executor, tuple_list!(cmplog_observer));

Expand Down
68 changes: 29 additions & 39 deletions fuzzers/fuzzbench_text/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ use content_inspector::inspect;
use libafl::{
corpus::{Corpus, InMemoryOnDiskCorpus, OnDiskCorpus},
events::SimpleRestartingEventManager,
executors::{inprocess::InProcessExecutor, ExitKind, TimeoutExecutor},
executors::{inprocess::InProcessExecutor, ExitKind},
feedback_or,
feedbacks::{CrashFeedback, MaxMapFeedback, TimeFeedback},
fuzzer::{Fuzzer, StdFuzzer},
Expand Down Expand Up @@ -394,29 +394,24 @@ fn fuzz_binary(
let mut tracing_harness = harness;

// Create the executor for an in-process function with one observer for edge coverage and one for the execution time
let mut executor = TimeoutExecutor::new(
InProcessExecutor::new(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
)?,
let mut executor = InProcessExecutor::with_timeout(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
timeout,
);
)?;

// Setup a tracing stage in which we log comparisons
let tracing = TracingStage::new(TimeoutExecutor::new(
InProcessExecutor::new(
&mut tracing_harness,
tuple_list!(cmplog_observer),
&mut fuzzer,
&mut state,
&mut mgr,
)?,
// Give it more time!
let tracing = TracingStage::new(InProcessExecutor::with_timeout(
&mut tracing_harness,
tuple_list!(cmplog_observer),
&mut fuzzer,
&mut state,
&mut mgr,
timeout * 10,
));
)?);

// The order of the stages matter!
let mut stages = tuple_list!(calibration, tracing, i2s, power);
Expand Down Expand Up @@ -621,29 +616,24 @@ fn fuzz_text(
let generalization = GeneralizationStage::new(&edges_observer);

// Create the executor for an in-process function with one observer for edge coverage and one for the execution time
let mut executor = TimeoutExecutor::new(
InProcessExecutor::new(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
)?,
let mut executor = InProcessExecutor::with_timeout(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
timeout,
);

)?;
// Setup a tracing stage in which we log comparisons
let tracing = TracingStage::new(TimeoutExecutor::new(
InProcessExecutor::new(
&mut tracing_harness,
tuple_list!(cmplog_observer),
&mut fuzzer,
&mut state,
&mut mgr,
)?,
let tracing = TracingStage::new(InProcessExecutor::with_timeout(
&mut tracing_harness,
tuple_list!(cmplog_observer),
&mut fuzzer,
&mut state,
&mut mgr,
// Give it more time!
timeout * 10,
));
)?);

// The order of the stages matter!
let mut stages = tuple_list!(generalization, calibration, tracing, i2s, power, grimoire);
Expand Down
18 changes: 8 additions & 10 deletions fuzzers/libafl_atheris/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use clap::{Arg, ArgAction, Command};
use libafl::{
corpus::{Corpus, InMemoryCorpus, OnDiskCorpus},
events::{launcher::Launcher, EventConfig},
executors::{inprocess::InProcessExecutor, ExitKind, TimeoutExecutor},
executors::{inprocess::InProcessExecutor, ExitKind},
feedback_or,
feedbacks::{CrashFeedback, MaxMapFeedback, TimeFeedback, TimeoutFeedback},
fuzzer::{Fuzzer, StdFuzzer},
Expand Down Expand Up @@ -197,16 +197,14 @@ pub extern "C" fn LLVMFuzzerRunDriver(
};

// Create the executor for an in-process function with one observer for edge coverage and one for the execution time
let mut executor = TimeoutExecutor::new(
InProcessExecutor::new(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
)?,
let mut executor = InProcessExecutor::with_timeout(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut mgr,
Duration::from_millis(timeout_ms),
);
)?;

// Secondary harness due to mut ownership
let mut harness = |input: &BytesInput| {
Expand Down
20 changes: 9 additions & 11 deletions fuzzers/libfuzzer_libpng/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ use std::{env, path::PathBuf};
use libafl::{
corpus::{Corpus, InMemoryCorpus, OnDiskCorpus},
events::{setup_restarting_mgr_std, EventConfig, EventRestarter},
executors::{inprocess::InProcessExecutor, ExitKind, TimeoutExecutor},
executors::{inprocess::InProcessExecutor, ExitKind},
feedback_or, feedback_or_fast,
feedbacks::{CrashFeedback, MaxMapFeedback, TimeFeedback, TimeoutFeedback},
fuzzer::{Fuzzer, StdFuzzer},
Expand Down Expand Up @@ -173,17 +173,15 @@ fn fuzz(corpus_dirs: &[PathBuf], objective_dir: PathBuf, broker_port: u16) -> Re
};

// Create the executor for an in-process function with one observer for edge coverage and one for the execution time
let mut executor = TimeoutExecutor::new(
InProcessExecutor::new(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut restarting_mgr,
)?,
// 10 seconds timeout
let mut executor = InProcessExecutor::with_timeout(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut restarting_mgr,
Duration::new(10, 0),
);
)?;
// 10 seconds timeout

// The actual target run starts here.
// Call LLVMFUzzerInitialize() if present.
Expand Down
19 changes: 8 additions & 11 deletions fuzzers/libfuzzer_libpng_accounting/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ use clap::Parser;
use libafl::{
corpus::{Corpus, InMemoryCorpus, OnDiskCorpus},
events::{EventConfig, Launcher},
executors::{inprocess::InProcessExecutor, ExitKind, TimeoutExecutor},
executors::{inprocess::InProcessExecutor, ExitKind},
feedback_or, feedback_or_fast,
feedbacks::{CrashFeedback, MaxMapFeedback, TimeFeedback, TimeoutFeedback},
fuzzer::{Fuzzer, StdFuzzer},
Expand Down Expand Up @@ -205,17 +205,14 @@ pub extern "C" fn libafl_main() {
};

// Create the executor for an in-process function with one observer for edge coverage and one for the execution time
let mut executor = TimeoutExecutor::new(
InProcessExecutor::new(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut restarting_mgr,
)?,
// 10 seconds timeout
let mut executor = InProcessExecutor::with_timeout(
&mut harness,
tuple_list!(edges_observer, time_observer),
&mut fuzzer,
&mut state,
&mut restarting_mgr,
opt.timeout,
);
)?;

// The actual target run starts here.
// Call LLVMFUzzerInitialize() if present.
Expand Down
Loading
Loading