The Guidance interface

One of JQF's main features is the ability to extend the feedback-directed input generation technique, via the Guidance interface. Fuzzing with AFL is just one of the many ways that this interface can be used. You can implement this interface to provide your own input-generation technique and run JQF using the static method GuidedFuzzing.run().

In a nutshell, the Guidance interface specifies the following methods:

public interface Guidance {
    boolean hasInput();
    InputStream getInput();
    void handleResult(Result result, Throwable error);
    Consumer<TraceEvent> generateCallBack(Thread thread);
}

The first three methods are invoked once per iteration of the main fuzzing. In pseudo-code, JQF runs the following fuzzing loop:

while (guidance.hasInput()) {
    // Create a pseudo-random-number generator from the input file
    Random rng = new StreamBackedRandom(guidance.getInput());
    // Generate the args to the test function using QuickCheck and custom RNG
    Object[] args = generateInput(rng);
    try {
        // Execute the test with generated inputs (i.e. args)
        runTest(args); // this generates many trace events
        guidance.handleResult(SUCCESS, null);
    } catch (AssumptionViolatedException e) {
        guidance.handleResult(INVALID, e);
    } catch (Throwable t) {
        if (isExpected(e)) {
            guidance.handleResult(SUCCESS, e);
        } else {
            guidance.handleResult(FAILURE, e);
        }
    }
}

Where:

• StreamBackedRandom is an extension of java.util.Random that picks values from an input stream instead of using a pseudo-random number generator. Although seemingly convoluted, the reason for doing this is that we can use existing quickcheck input generators, which have been written to work with a source of "random" values.
• generateInput generates an input for the test program using the quickcheck generators corresponding to the arguments of the test function (see Writing a JQF test).
• runTest is a single execution of the test with the generated inputs, called a trial, which generates trace events using program instrumentation (see Implementation details).
• Guidance.hasInput() tells JQF if there are more inputs to run. Fuzzing stops when this method returns false.
• Guidance.getInput() returns a reference to the input stream containing "random" values for the quickcheck generators.
• Guidance.handleResult() is invoked at the end of a trial run. The guidance object receives a Result and an optional throwable. Note that assumption violations are treated differently from test errors, giving the guidance to choose to update its input-generation technique to avoid assumption failures. A successful trial is one which terminates without any exceptions or if an exception listed in the throws clause of the test method is thrown. Assertion failures or other unexpected exceptions thrown during the test run classify as failures.
• Guidance.generateCallBack(): In order to provide a feedback-directed guidance, a Guidance instance can handle trace events generated during the test method's execution (e.g. function calls or program branches), which will help it generate future inputs. To do this, the Guidance instance must provide a callback -- a method that takes as its argument a trace event. As JQF supports multi-threaded execution of test methods, these callbacks can be different for different application threads (e.g. perhaps you only want to monitor the main thread, or you want to maintain per-thread call stacks). The method Guidance.generateCallBack(thread) is invoked exactly once per thread, whenever the first trace event for that thread is generated. JQF guarantees that for a given thread, the callbacks for trace events will be invoked in the order of the execution of the corresponding events in their respective threads; no ordering guarantees are given for trace events across distinct threads. Here's a silly example of a callback that prints trace events to standard output:

@Override
public Consumer<TraceEvent> generateCallBack(Thread thread) {
    return (event) -> {
        System.out.println(String.format("Thread %s produced event %s",
            thread.getName(), event));
    };
}

Sample implementations

• NoGuidance - the most trivial implementation is that of having no guidance at all. This class is effectively a blind random input generation; it discards trace events completely.
• AFLGuidance - guided fuzzing using AFL as the fuzzing engine. This class performs inter-process communication with an AFL process running alongside it, via a proxy.
• ReproGuidance - this is the mechanism used to replay test cases discovered by AFL or other guidance engines. It simply provides inputs from one or more provided files; the trace events may be optionally logged to file for further debugging or analysis.
• PerfFuzzGuidance - this class extends AFLGuidance to provide feedback about execution counts, so as to generate inputs that degrade performance of a test application (e.g. worst-case inputs for some algorithm). This implementation inpsired our work on the ISSTA'18 paper for PerfFuzz.
• ZestGuidance - a home-grown fuzzing engine that biases input generation towards semantic validity. Check out the ISSTA'19 paper for more details on the Zest algorithm.