Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MutatedTransform to the input type in TMinMutationalStage (#1251) #1971

Merged
merged 4 commits into from
Mar 27, 2024
Merged

Add MutatedTransform to the input type in TMinMutationalStage (#1251) #1971

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
411f84a
Support `MutatedTransform` in `TMinMutationalStage`.
am009 Mar 25, 2024
12b3849
Run `MutatedTransformPost` for the replaced testcase.
am009 Mar 25, 2024
a5ee00d
Add clone trait bound for `MutatedTransformPost`.
am009 Mar 25, 2024
48a5235
Return an error instead of using unwrap.
am009 Mar 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 47 additions & 22 deletions libafl/src/stages/tmin.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! The [`TMinMutationalStage`] is a stage which will attempt to minimize corpus entries.

use alloc::string::{String, ToString};
use core::{fmt::Debug, hash::Hash, marker::PhantomData};
use core::{borrow::BorrowMut, fmt::Debug, hash::Hash, marker::PhantomData};

use ahash::RandomState;
use libafl_bolts::{HasLen, Named};
Expand All @@ -16,7 +16,10 @@ use crate::{
mutators::{MutationResult, Mutator},
observers::{MapObserver, ObserversTuple},
schedulers::{RemovableScheduler, Scheduler},
stages::{ExecutionCountRestartHelper, Stage},
stages::{
mutational::{MutatedTransform, MutatedTransformPost},
ExecutionCountRestartHelper, Stage,
},
start_timer,
state::{
HasCorpus, HasCurrentTestcase, HasExecutions, HasMaxSize, HasMetadata, HasSolutions, State,
Expand All @@ -30,7 +33,7 @@ use crate::{monitors::PerfFeature, state::HasClientPerfMonitor};
/// Mutational stage which minimizes corpus entries.
///
/// You must provide at least one mutator that actually reduces size.
pub trait TMinMutationalStage<CS, E, EM, F1, F2, M, OT, Z>:
pub trait TMinMutationalStage<CS, E, EM, F1, F2, I, IP, M, OT, Z>:
Stage<E, EM, Z> + FeedbackFactory<F2, CS::State, OT>
where
Self::State: HasCorpus + HasSolutions + HasExecutions + HasMaxSize,
Expand All @@ -40,12 +43,14 @@ where
EM: EventFirer<State = Self::State>,
F1: Feedback<Self::State>,
F2: Feedback<Self::State>,
M: Mutator<Self::Input, Self::State>,
M: Mutator<I, Self::State>,
OT: ObserversTuple<CS::State>,
Z: ExecutionProcessor<OT, State = Self::State>
+ ExecutesInput<E, EM>
+ HasFeedback<Feedback = F1>
+ HasScheduler<Scheduler = CS>,
IP: MutatedTransformPost<Self::State> + Clone,
I: MutatedTransform<Self::Input, Self::State, Post = IP> + Clone,
{
/// The mutator registered for this stage
fn mutator(&self) -> &M;
Expand Down Expand Up @@ -77,7 +82,10 @@ where
- usize::try_from(self.execs_since_progress_start(state)?).unwrap();

start_timer!(state);
let transformed = I::try_transform_from(state.current_testcase_mut()?.borrow_mut(), state)?;
let mut base = state.current_input_cloned()?;
// potential post operation if base is replaced by a shorter input
let mut base_post = None;
let base_hash = RandomState::with_seeds(0, 0, 0, 0).hash_one(&base);
mark_feature_time!(state, PerfFeature::GetInputFromCorpus);

Expand All @@ -93,20 +101,21 @@ where
}

let mut next_i = i + 1;
let mut input = base.clone();
let mut input_transformed = transformed.clone();

let before_len = input.len();
let before_len = base.len();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be input_transformed.len()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of the awkward places.

The input_transformed is now of the wrapped type MutatedTransform<Input, State>. I think here we want the length of the internal input (if I understand correctly). If we use try_transform_into to get the internal input (so that we can call .len()), MutatedTransform<Input, State> will be consumed. But we still need to keep MutatedTransform<Input, State> to pass into the mutator.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok got it


state.set_max_size(before_len);

start_timer!(state);
let mutated = self.mutator_mut().mutate(state, &mut input)?;
let mutated = self.mutator_mut().mutate(state, &mut input_transformed)?;
mark_feature_time!(state, PerfFeature::Mutate);

if mutated == MutationResult::Skipped {
continue;
}

let (input, post) = input_transformed.try_transform_into(state)?;
let corpus_idx = if input.len() < before_len {
// run the input
let exit_kind = fuzzer.execute_input(state, executor, manager, &input)?;
Expand Down Expand Up @@ -134,6 +143,7 @@ where
if feedback.is_interesting(state, manager, &input, observers, &exit_kind)? {
// we found a reduced corpus entry! use the smaller base
base = input;
base_post = Some(post.clone());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to avoid this clone() ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function MutatedTransformPost.post_exec consumes self. For example, the post is a metadata, and its ownership is transferred to the metadata map in the post_exec. So I guess it can't be avoided?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could set a bool to true here and then do base_post conversion and post_exec below, after calling post.post_exec

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So do we still need to clone? I think the post object is no longer valid after post_exec, because it is comsumed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooh...
Well I guess in that case not much we can do..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no i removed it


// do more runs! maybe we can minify further
next_i = 0;
Expand All @@ -149,6 +159,7 @@ where

start_timer!(state);
self.mutator_mut().post_exec(state, corpus_idx)?;
post.post_exec(state, corpus_idx)?;
mark_feature_time!(state, PerfFeature::MutatePostExec);

i = next_i;
Expand All @@ -171,6 +182,11 @@ where
fuzzer
.scheduler_mut()
.on_replace(state, base_corpus_idx, &prev)?;
// perform the post operation for the new testcase, e.g. to update metadata.
// base_post should be updated along with the base (and is no longer None)
base_post
.ok_or_else(|| Error::empty_optional("Failed to get the MutatedTransformPost"))?
.post_exec(state, Some(base_corpus_idx))?;
}

state.set_max_size(orig_max_size);
Expand All @@ -184,7 +200,7 @@ where

/// The default corpus entry minimising mutational stage
#[derive(Clone, Debug)]
pub struct StdTMinMutationalStage<CS, E, EM, F1, F2, FF, M, OT, Z> {
pub struct StdTMinMutationalStage<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z> {
/// The mutator(s) this stage uses
mutator: M,
/// The factory
Expand All @@ -194,22 +210,24 @@ pub struct StdTMinMutationalStage<CS, E, EM, F1, F2, FF, M, OT, Z> {
/// The progress helper for this stage, keeping track of resumes after timeouts/crashes
restart_helper: ExecutionCountRestartHelper,
#[allow(clippy::type_complexity)]
phantom: PhantomData<(CS, E, EM, F1, F2, OT, Z)>,
phantom: PhantomData<(CS, E, EM, F1, F2, I, IP, OT, Z)>,
}

impl<CS, E, EM, F1, F2, FF, M, OT, Z> UsesState
for StdTMinMutationalStage<CS, E, EM, F1, F2, FF, M, OT, Z>
impl<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z> UsesState
for StdTMinMutationalStage<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z>
where
CS: Scheduler,
M: Mutator<CS::Input, CS::State>,
M: Mutator<I, CS::State>,
Z: ExecutionProcessor<OT, State = CS::State>,
CS::State: HasCorpus,
IP: MutatedTransformPost<CS::State> + Clone,
I: MutatedTransform<CS::Input, CS::State, Post = IP> + Clone,
{
type State = CS::State;
}

impl<CS, E, EM, F1, F2, FF, M, OT, Z> Stage<E, EM, Z>
for StdTMinMutationalStage<CS, E, EM, F1, F2, FF, M, OT, Z>
impl<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z> Stage<E, EM, Z>
for StdTMinMutationalStage<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z>
where
CS: Scheduler + RemovableScheduler,
CS::State: HasCorpus + HasSolutions + HasExecutions + HasMaxSize + HasCorpus + HasMetadata,
Expand All @@ -219,12 +237,14 @@ where
F1: Feedback<CS::State>,
F2: Feedback<CS::State>,
FF: FeedbackFactory<F2, CS::State, OT>,
M: Mutator<CS::Input, CS::State>,
M: Mutator<I, CS::State>,
OT: ObserversTuple<CS::State>,
Z: ExecutionProcessor<OT, State = CS::State>
+ ExecutesInput<E, EM>
+ HasFeedback<Feedback = F1>
+ HasScheduler<Scheduler = CS>,
IP: MutatedTransformPost<CS::State> + Clone,
I: MutatedTransform<CS::Input, CS::State, Post = IP> + Clone,
{
fn perform(
&mut self,
Expand All @@ -250,8 +270,8 @@ where
}
}

impl<CS, E, EM, F1, F2, FF, M, OT, Z> FeedbackFactory<F2, Z::State, OT>
for StdTMinMutationalStage<CS, E, EM, F1, F2, FF, M, OT, Z>
impl<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z> FeedbackFactory<F2, Z::State, OT>
for StdTMinMutationalStage<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z>
where
F2: Feedback<Z::State>,
FF: FeedbackFactory<F2, Z::State, OT>,
Expand All @@ -262,8 +282,8 @@ where
}
}

impl<CS, E, EM, F1, F2, FF, M, OT, Z> TMinMutationalStage<CS, E, EM, F1, F2, M, OT, Z>
for StdTMinMutationalStage<CS, E, EM, F1, F2, FF, M, OT, Z>
impl<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z> TMinMutationalStage<CS, E, EM, F1, F2, I, IP, M, OT, Z>
for StdTMinMutationalStage<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z>
where
CS: Scheduler + RemovableScheduler,
E: HasObservers<Observers = OT, State = CS::State> + Executor<EM, Z>,
Expand All @@ -272,13 +292,15 @@ where
F2: Feedback<CS::State>,
FF: FeedbackFactory<F2, CS::State, OT>,
<CS::State as UsesInput>::Input: HasLen + Hash,
M: Mutator<CS::Input, CS::State>,
M: Mutator<I, CS::State>,
OT: ObserversTuple<CS::State>,
CS::State: HasCorpus + HasSolutions + HasExecutions + HasMaxSize + HasMetadata,
Z: ExecutionProcessor<OT, State = CS::State>
+ ExecutesInput<E, EM>
+ HasFeedback<Feedback = F1>
+ HasScheduler<Scheduler = CS>,
IP: MutatedTransformPost<CS::State> + Clone,
I: MutatedTransform<CS::Input, CS::State, Post = IP> + Clone,
{
/// The mutator, added to this stage
#[inline]
Expand All @@ -302,12 +324,15 @@ where
}
}

impl<CS, E, EM, F1, F2, FF, M, OT, Z> StdTMinMutationalStage<CS, E, EM, F1, F2, FF, M, OT, Z>
impl<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z>
StdTMinMutationalStage<CS, E, EM, F1, F2, FF, I, IP, M, OT, Z>
where
CS: Scheduler,
M: Mutator<CS::Input, CS::State>,
M: Mutator<I, CS::State>,
Z: ExecutionProcessor<OT, State = CS::State>,
CS::State: HasCorpus,
IP: MutatedTransformPost<CS::State> + Clone,
I: MutatedTransform<CS::Input, CS::State, Post = IP> + Clone,
{
/// Creates a new minimizing mutational stage that will minimize provided corpus entries
pub fn new(mutator: M, factory: FF, runs: usize) -> Self {
Expand Down
Loading