add the metrics(pending,pend_fav, own_finds,imported) #1351

ToSeven · 2023-07-06T11:49:08Z

No description provided.

ToSeven · 2023-07-06T11:53:36Z

Sorry to close this PR #1332 by accident

libafl/src/fuzzer/mod.rs

tokatoka · 2023-07-06T11:58:56Z

libafl/src/events/llmp.rs

@@ -624,7 +627,13 @@ impl<E, S, SP, Z> EventManager<E, Z> for LlmpEventManager<S, SP>
 where
    E: HasObservers<State = S> + Executor<Self, Z>,
    for<'a> E::Observers: Deserialize<'a>,
-    S: UsesInput + HasExecutions + HasClientPerfMonitor + HasMetadata,
+    S: UsesInput


you don't need these type constraints

There is an error.

hm but why? can you give me full error log?

because here you don't even use those methods then why is it necessary?

but you don't need this in eventmanager no?

I mean you could just store imported, pending in state, and don't make any traits.
then you don't need these type constraints anymore

hmm, I merge all 4 metrics into one trait HasAFLStats, then I just need to type one constraint. how about?

libafl/src/state/mod.rs

tokatoka · 2023-07-06T12:00:47Z

can you remove the unnecessary traits?

tokatoka · 2023-07-07T13:16:56Z

is the UI supposed to run on a different process other than the main fuzzer?
@vanhauser-thc

vanhauser-thc · 2023-07-07T13:22:01Z

is the UI supposed to run on a different process other than the main fuzzer?
@vanhauser-thc

imho out of the fuzzer process is best, @domenukk was first fighting hard for in broker process then thought about it and … not sure what he would want now :)

ToSeven · 2023-07-08T06:57:42Z

Please review my code.

vanhauser-thc · 2023-07-13T13:48:41Z

@ToSeven can you please resolve the conflicts?

libafl/src/state/mod.rs

tokatoka

I said "how about putting code in calibration stage" before.

but now i think it feels odd to do the stats stuff in calibration stage.
can you separate the code in calibration stage into another stage like StatsReportsStage? (You need to make another stage for this)

tokatoka · 2023-07-13T17:27:55Z

@ToSeven

Merge https://github.com/AFLplusplus/LibAFL/tree/stats_stage to your code.
I've made a stage where you should put all your stats-related code.

ToSeven · 2023-07-14T10:08:31Z

@ToSeven can you please resolve the conflicts?

ok

libafl/src/events/centralized.rs

vanhauser-thc · 2023-07-17T05:05:02Z

also please run cargo fmt and cargo clippy to ensure clean code. maybe you did that but I see things in the code that I think clippy would not allow :)
but besides this and my one comment I think we are close :)
@tokatoka ?

domenukk · 2023-07-18T22:59:04Z

is the UI supposed to run on a different process other than the main fuzzer?
@vanhauser-thc

imho out of the fuzzer process is best, @domenukk was first fighting hard for in broker process then thought about it and … not sure what he would want now :)

As long as it adheres to the Monitor API it can be out of process or in process without minimal code changes ( we can create a Proxy monitor that forwards to an out of process monitor)

domenukk · 2023-07-18T23:02:08Z

libafl/src/state/mod.rs

@@ -404,6 +439,55 @@ impl<I, C, R, SC> HasExecutions for StdState<I, C, R, SC> {
    }
 }

+impl<I, C, R, SC> HasAFLStats for StdState<I, C, R, SC> {


Should be HasAflStats for consistency.
That being said, maybe there is an even nicer name? We'll need that guy everywhere.

domenukk · 2023-07-18T23:03:41Z

libafl/src/stages/stats.rs

+        if cur.checked_sub(self.last_report_time).unwrap_or_default() > self.stats_report_interval {
+            manager.fire(
+                state,
+                Event::UpdateUserStats {


I think we should fire one message with all stats in them, else this could lead to more overhead on the receiving end since UI might have to be updated for each new UserStats message, etc

tokatoka · 2023-07-18T23:06:41Z

libafl/src/state/mod.rs

@@ -787,6 +871,10 @@ where
        let mut state = Self {
            rand,
            executions: 0,
+            pending: 0,


can you add this member in stages/stats.rs?

instead of putting in state

how about adding the "pending" " pend_favor" "own_finds" to stages/stats.rs and remaining the "imported" in state?
I found it not easy to calculate the value of the "imported" in stats.rs. @tokatoka

tokatoka · 2023-07-18T23:07:21Z

ideally your change should be only inside stages/stats.rs

domenukk · 2023-08-22T09:00:07Z

libafl/src/stages/stats.rs

+
+/// The [`AFLStatsStage`] is a simple stage that computes and reports some stats.
+#[derive(Debug, Clone)]
+pub struct AFLStatsStage<E, EM, Z>


We usually use Afl (lowercase fl)

tokatoka · 2023-08-22T20:33:20Z

libafl/src/stages/mod.rs

@@ -256,6 +259,7 @@ where
        + HasMetadata
        + HasRand
        + HasCorpus
+        + HasImported


push adapter doesn't need this right?

domenukk · 2023-08-23T06:42:49Z

We moved the libafl::bolts module to the libafl_bolts create, that's a big reason this PR is failing right now :)

domenukk · 2023-08-28T12:09:11Z

We should try this PR together with #1432, maybe create (or extend) a fuzzer in ./fuzzers that uses the StatsStage

ToSeven · 2023-09-04T10:13:19Z

We should try this PR together with #1432, maybe create (or extend) a fuzzer in ./fuzzers that uses the StatsStage

I will create a fuzzer with the AFL-Style UI in ./fuzzers

tokatoka

we can merge this after you resolve the conflict and fix CI.

domenukk · 2023-09-06T01:38:29Z

Thanks!

tokatoka reviewed Jul 6, 2023

View reviewed changes

libafl/src/fuzzer/mod.rs Outdated Show resolved Hide resolved

tokatoka reviewed Jul 6, 2023

View reviewed changes

libafl/src/state/mod.rs Outdated Show resolved Hide resolved

ToSeven force-pushed the main branch 5 times, most recently from a5c3d56 to 5e9c5d7 Compare July 7, 2023 13:12

add the metrics(pending,own_finds,imported)

297e848

ToSeven force-pushed the main branch from 5e9c5d7 to 297e848 Compare July 8, 2023 06:37

ToSeven changed the title ~~add the metrics(pending,own_finds,imported)~~ add the metrics(pending,pend_fav, own_finds,imported) Jul 8, 2023

add the pend_fav metrics

f79240d

ToSeven force-pushed the main branch from fd4efbc to f79240d Compare July 8, 2023 07:06

tokatoka reviewed Jul 13, 2023

View reviewed changes

libafl/src/state/mod.rs Outdated Show resolved Hide resolved

tokatoka requested changes Jul 13, 2023

View reviewed changes

push

7211c5e

toseven added 2 commits July 16, 2023 13:18

pull the latest source code and fix all conflicts

686529b

Merge branch 'stats_stage' of https://github.com/AFLplusplus/LibAFL

29b1122

vanhauser-thc reviewed Jul 17, 2023

View reviewed changes

libafl/src/events/centralized.rs Outdated Show resolved Hide resolved

ToSeven force-pushed the main branch from dbb8bee to 7e3f6f6 Compare July 18, 2023 08:41

Add the feature that AFLStats is computed and reported in AFLStatsStage

4ae8c9b

ToSeven force-pushed the main branch from 7e3f6f6 to 4ae8c9b Compare July 18, 2023 08:50

toseven added 2 commits July 18, 2023 17:23

fix some cicd errors

64fe9e3

Merge branch 'main' of https://github.com/AFLplusplus/LibAFL

0315bc9

domenukk reviewed Jul 18, 2023

View reviewed changes

tokatoka reviewed Jul 18, 2023

View reviewed changes

toseven added 3 commits July 24, 2023 22:16

AFLStats migrates to stage/stats.rs

4a57a73

fix the cicd error

cf6cbe7

Merge branch 'main' of https://github.com/AFLplusplus/LibAFL

40a63fa

ToSeven force-pushed the main branch from 877ae27 to 40a63fa Compare July 26, 2023 13:19

domenukk added 2 commits August 1, 2023 11:26

Merge branch 'main' into main

25a1303

Merge branch 'main' into main

b048dfe

domenukk enabled auto-merge (squash) August 11, 2023 11:56

Merge branch 'AFLplusplus:main' into main

37df814

domenukk reviewed Aug 22, 2023

View reviewed changes

tokatoka reviewed Aug 22, 2023

View reviewed changes

Merge branch 'main' into main

ff20b1b

tokatoka approved these changes Sep 4, 2023

View reviewed changes

fix some bugs and resolve the conflicts

a5b3462

auto-merge was automatically disabled September 5, 2023 08:58
Head branch was pushed to by a user without write access

fix some typos

0a48eaa

ToSeven mentioned this pull request Sep 5, 2023

Add an example fuzzer with AFL-Style UI #1501

Merged

domenukk merged commit 04aecd9 into AFLplusplus:main Sep 6, 2023
17 checks passed

add the metrics(pending,pend_fav, own_finds,imported) #1351

add the metrics(pending,pend_fav, own_finds,imported) #1351

Conversation

ToSeven commented Jul 6, 2023

ToSeven commented Jul 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tokatoka Jul 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ToSeven Jul 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tokatoka commented Jul 6, 2023

tokatoka commented Jul 7, 2023

vanhauser-thc commented Jul 7, 2023

ToSeven commented Jul 8, 2023

vanhauser-thc commented Jul 13, 2023

tokatoka left a comment

Choose a reason for hiding this comment

tokatoka commented Jul 13, 2023

ToSeven commented Jul 14, 2023

vanhauser-thc commented Jul 17, 2023

domenukk commented Jul 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tokatoka commented Jul 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenukk commented Aug 23, 2023

domenukk commented Aug 28, 2023

ToSeven commented Sep 4, 2023

tokatoka left a comment • edited Loading

Choose a reason for hiding this comment

domenukk commented Sep 6, 2023

tokatoka Jul 6, 2023 •

edited

Loading

ToSeven Jul 6, 2023 •

edited

Loading

domenukk commented Jul 18, 2023 •

edited

Loading

tokatoka left a comment •

edited

Loading