Agamotto: Accelerating Kernel Driver Fuzzing with Lightweight Virtual Machine Checkpoints

Prerequisite

• CMake 3.7.2 or higher (cmake -version)
• Go 1.12.3 or higher (go version)
• Python 3

Setup

Download source code

git clone --recursive https://github.com/securesystemslab/agamotto.git
cd agamotto
export AGPATH=$PWD # assumed by commands that follow
./setup.sh

Change the host Linux kernel for custom hypercall support

Build the host Linux kernel with our patch applied, and with CONFIG_KVM_AGAMOTTO=y, and install & reboot it.

Tested environment:

• Ubuntu-hwe-4.18.0-18.19_18.04.1 on AMD EPYC 7601

Download and build Syzkaller

# Get Syzkaller source code
go get -u -d github.com/google/syzkaller
cd $GOPATH/src/github.com/google/syzkaller
git checkout ddc3e85997efdad885e208db6a98bca86e5dd52f

# Apply patch and build
cd $GOPATH/src/github.com/google/syzkaller
patch -p0 <$AGPATH/syzkaller.patch
make

Build project and generate necessary files

# Build project
cd $AGPATH/build
cmake ..
make

Setup QEMU

# Apply patch
cd $AGPATH/qemu
patch -p0 <$AGPATH/qemu.patch

# Build
mkdir $AGPATH/build/qemu
cd $AGPATH/build/qemu
$AGPATH/qemu/configure --prefix=$AGPATH/build/qemu/install --target-list=x86_64-softmmu --with-agamotto=$AGPATH/build/libagamotto --enable-debug
make -j4 install

Setup VM

• Patch and build Linux kernel

  cd $AGPATH/guest/linux/kernel
  patch -p0 <../kernel.patch

  cd $AGPATH/scripts
  ./build-linux-guest.sh all ../guest/linux/kernel/

• Create a Debian image

  cd $AGPATH/scripts
  ./create-debian-image.sh             # Create an image
  ./copy-modules.py all -d stretch.img # Copy necessary files into the image

Start fuzzing

# Generate Syzkaller config files
cd $AGPATH
make -C configs/syzkaller VMCNT=<number of fuzzing instances> -B

# Run Syzkaller USB fuzzing
cd $GOPATH/src/github.com/google/syzkaller
export PATH=$AGPATH/build/qemu/install/bin:$PATH
export LD_LIBRARY_PATH=$AGPATH/build/libagamotto:$LD_LIBRARY_PATH
./bin/syz-manager -config $AGPATH/configs/syzkaller/generated/<CFG_FILE>.cfg

# Run AFL PCI fuzzing
cd $AGPATH/scripts
./create-overlay-image.py rtl8139 -d stretch.img
export PATH=$AGPATH/build/qemu/install/bin:$PATH
export LD_LIBRARY_PATH=$AGPATH/build/libagamotto:$LD_LIBRARY_PATH
./fuzz.py rtl8139 -g linux-prog05 -i seed/ -N <number of fuzzing instances>

Citing our work

@inproceedings{song2020agamotto,
  title =        {{Agamotto}: Accelerating Kernel Driver Fuzzing with
                  Lightweight Virtual Machine Checkpoints},
  author =       {Song, Dokyung and Hetzelt, Felicitas and Kim, Jonghwan and
                  Kang, Brent Byunghoon and Seifert, Jean-Pierre and Franz,
                  Michael},
  booktitle =    {{USENIX} Security Symposium},
  year =         {2020}
}