JAttack is a framework that enables template-based testing for compilers. Using JAttack, compiler developers can write templates in the same language as the compiler they are testing (Java), enabling them to leverage their domain knowledge to set up a code structure likely to lead to compiler optimizations while leaving holes representing expressions they want explored. JAttack executes templates, exploring possible expressions for holes and filling them in, generating programs to later be run on compilers. JAttack blends the power of developers insights, who are providing templates, and random testing to detect critical bugs.
This demo reproduces a bug of OpenJDK jdk-11.0.8+10 C2 JIT compiler
using template T.java.
- Developers write a template program using JAttack's DSL fully
embedded in Java, for example,
T.java.
import jattack.annotation.Entry;
import static jattack.Boom.*;
public class T {
static int s1;
static int s2;
@Entry
public static int m() {
int[] arr = { s1++, s2, 1, 2, intVal().eval() };
for (int i = 0; i < arr.length; ++i) {
if (intIdOrIntArrAccessExp().eval() <= s2
|| relation(intId("s2"), intIdOrIntArrAccessExp(), LE).eval()) {
arr[i] &= arithmetic(intId(), intArrAccessExp(), ADD, MUL).eval();
}
}
return s1 + s2;
}
}- JAttack executes the given template to generate concrete Java
programs. For example, one generated program from the
template
T.javacan beTGen1.java.
import jattack.annotation.Entry;
import static jattack.Boom.*;
import org.csutil.checksum.WrappedChecksum;
public class TGen1 {
static int s1;
static int s2;
public static int m() {
int[] arr = { s1++, s2, 1, 2, -1170105035 };
for (int i = 0; i < arr.length; ++i) {
if (i <= s2 || (s2 <= arr[2])) {
arr[i] &= (s2 + arr[0]);
}
}
return s1 + s2;
}
public static long main0(String[] args) {
int N = 100000;
if (args.length > 0) {
N = Math.min(Integer.parseInt(args[0]), N);
}
WrappedChecksum cs = new WrappedChecksum();
for (int i = 0; i < N; ++i) {
try {
cs.update(m());
} catch (Throwable e) {
if (e instanceof jattack.exception.InvokedFromNotDriverException) {
throw e;
}
cs.update(e.getClass().getName());
}
}
cs.updateStaticFieldsOfClass(TGen1.class);
return cs.getValue();
}
public static void main(String[] args) {
System.out.println(main0(args));
}
}- JAttack runs every generated program across Java JIT compilers
under test. For example, running the generated program
TGen1.javacrashes C2 in openjdk-11.0.8.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f55deedd845, pid=432431, tid=432442
#
# JRE version: OpenJDK Runtime Environment AdoptOpenJDK (11.0.8+10) (build 11.0.8+10)
# Java VM: OpenJDK 64-Bit Server VM AdoptOpenJDK (11.0.8+10, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0xd60845] ok_to_convert(Node*, Node*)+0x15
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to /home/zzq/projects/jattack/core.432431)
#
# If you would like to submit a bug report, please visit:
# https://github.com/AdoptOpenJDK/openjdk-support/issues
#
To run the demo, please run ./demo.sh. Sample output:
Download JDK...
Build JAttack jar...
Install python packages...
1..3
[10:28:28E]su.__main__: bash: line 1: 2414756 Aborted (core dumped) /home/zzq/projects/jattack/.downloads/jdk-11.0.8+10/bin/java -cp /home/zzq/projects/jattack/tool/jattack-all.jar:/home/zzq/projects/jattack/.jattack/T/build -XX:TieredStopAtLevel=4 -XX:ErrorFile=/home/zzq/projects/jattack/.jattack/T/output/TGen1/he_err_pid%p.log -XX:ReplayDataFile=/home/zzq/projects/jattack/.jattack/T/output/TGen1/replay_pid%p.log TGen1 > /home/zzq/projects/jattack/.jattack/T/output/TGen1/java_env0.txt 2> /dev/null
not ok 1 - TGen1
---
message: 'Found a potential crash bug'
data: CrashBugData(type=<BugType.CRASH: 'crash'>, crashed_java_envs=[JavaEnv(java_home=PosixPath('/home/zzq/projects/jattack/.downloads/jdk-11.0.8+10'), java_opts=['-XX:TieredStopAtLevel=4'])])
...
ok 2 - TGen2
ok 3 - TGen3
- Linux with GNU Bash (tested on Ubuntu 20.04)
- JDK >=11
- Python 3.8
cd tool
./install.shThe install.sh script builds JAttack jar, installs python packages
and creates an executable jattack in tools.
cd tool
./jattack --clz TEMPLATE_CLASS_NAME --n_gen NUM_OF_GENERATED_PROGRAMS \
[--java_envs JAVA_ENVIRONMENTS_UNDER_TEST]
[--src TEMPLATE_SOURCE_PATH]
[--n_itrs NUM_OF_ITERATIONS_TO_TRIGGER_JIT]
[--seed RANDOM_SEED]-
Provide only two required arguments
--clzand--n_gen../tool/jattack --clz T --n_gen 3
This command generates 3 programs from template
T.javaand uses the 3 generated programs to test default java environments found in$JAVA_HOMEat level 4 and level 1, which are:$JAVA_HOME/bin/java -XX:TieredStopAtLevel=4$JAVA_HOME/bin/java -XX:TieredStopAtLevel=1
-
Specify java environments and associated java options to be tested using
--java_envs../tool/jattack --clz T --n_gen 3 \ --java_envs "[\ [/home/zzq/opt/jdk-11.0.15,[-Xbatch,-Xcomp,-XX:-TieredCompilation]],\ [/home/zzq/opt/jdk-17.0.3,[-Xbatch,-Xcomp,-XX:TieredStopAtLevel=1]],\ [/home/zzq/opt/jdk-17.0.3,[]]]"The
java_envsargument is a list, which can also be appended using--java_envs+=, for example, the command above can be rewritten as:./tool/jattack --clz T --n_gen 3 \ --java_envs [[/home/zzq/opt/jdk-11.0.15,[-Xbatch,-Xcomp,-XX:-TieredCompilation]]] \ --java_envs+=[[/home/zzq/opt/jdk-17.0.3,[-Xbatch,-Xcomp,-XX:TieredStopAtLevel=1]]] \ --java_envs+=[[/home/zzq/opt/jdk-17.0.3,[]]]The
java_envsargument can also be given using a config file, i.e.,./tool/jattack --config config.yaml --clz T --n_gen 3
where
config.yamlis:# config.yaml java_envs: - - /home/zzq/opt/jdk-11.0.15 - - -Xbatch - -Xcomp - -XX:-TieredCompilation - - /home/zzq/opt/jdk-17.0.3 - - -Xbatch - -Xcomp - -XX:TieredStopAtLevel=1 - - /home/zzq/opt/jdk-17.0.3 - []
This command generates 3 programs from template
T.javaand uses the 3 generated programs to test given java environments with given options, which are/home/zzq/opt/jdk-11.0.15/bin/java -Xbatch -Xcomp -XX:-TieredCompilation/home/zzq/opt/jdk-17.0.3/bin/java -Xbatch -Xcomp -XX:TieredStopAtLevel=1/home/zzq/opt/jdk-17.0.3/bin/java
-h, --help Show this help message and exit.
--config CONFIG Path to a configuration file.
--print_config[=flags]
Print the configuration after applying all
other arguments and exit. The optional flags
are one or more keywords separated by comma
which modify the output. The supported flags
are: comments, skip_default, skip_null.
--clz CLZ the fully qualified class name of the
template, separated with "." (required, type:
str)
--n_gen N_GEN the total number of generated programs
(required, type: int)
--src SRC the path to the source file of the template.
By default, `./{clz}.java` is used. (type:
Optional[str], default: null)
--n_itrs N_ITRS the number of iterations to trigger JIT (type:
int, default: 100000)
--seed SEED the random seed used by JAttack during
generation, fix this to reproduce a previous
generation. (type: Optional[int], default:
null)
--java_envs JAVA_ENVS, --java_envs+ JAVA_ENVS
the java environments to be differentially
tested, which should be provided as a list of
a tuple of java home string and a list of java
option strings, e.g., `--java_envs=[[/home/zzq
/opt/jdk-11.0.15,[-XX:TieredStopAtLevel=4]],[/
home/zzq/opt/jdk-17.0.3,[-XX:TieredStopAtLevel
=1]]]` means we want to differentially test
java 11 at level 4 and java 17 at level 1.
Note, the first java environment of the list
will be used to compile the template and
generated programs, which means the version of
the first java environment has to be less than
or equal to the remaining ones. Also, the
first java environment is used to run JAttack
itself, which means its version should be at
least 11. By default, $JAVA_HOME in the system
with level 4 and level 1 are used, i.e., `--ja
va_envs=[[$JAVA_HOME,[-XX:TieredStopAtLevel=4]
],[$JAVA_HOME,[-XX:TieredStopAtLevel=1]]]`
(type: Optional[List[Tuple[str, List[str]]]],
default: null)
JAttack's command-line output is in TAP format, so you can make it prettier using any TAP consumer, like tapview:
$ ./tool/jattack --clz T --n_gen 3 --seed 42 \
--java_envs "[\
[.downloads/jdk-11.0.8+10,[-XX:TieredStopAtLevel=4]],\
[.downloads/jdk-11.0.8+10,[-XX:TieredStopAtLevel=1]]]" \
| tapview
F..
not ok 1 - TGen1
---
message: 'Found a potential crash bug'
data: CrashBugData(type=<BugType.CRASH: 'crash'>, crashed_java_envs=[JavaEnv(java_home=PosixPath('/home/zzq/projects/jattack/.downloads/jdk-11.0.8+10'), java_opts=['-XX:TieredStopAtLevel=4'])])
...
3 tests, 1 failures.
After the run, a hidden directory .jattack is created under
current working directory with the following structure:
.jattack
- logs # logs of runs
- 1679956565593918684.log
- T
- build # Java class files
- TGen3.class
- gen # Generated programs from the template
- Gen1
- TGen1.java
- Gen2
- TGen2.java
- Gen3
- TGen3.java
- output # Outputs of generated programs executed on different java environments
- Gen1
- he_err_pid693345.log # error data file of the crash
- java_env1.txt # Output from execution on java_envs[1]
- replay_pid693345.log # replay data file of the crash
- Gen2
- java_env0.txt # Output from execution on java_envs[0]
- java_env1.txt
- Gen3
- java_env0.txt
- java_env1.txt
The following steps build javadoc for JAttack jar. Please refer to
class Boom for how to use provided APIs to write your own template.
-
Build javadoc from source code.
cd tool/api ./gradlew javadoc -
Open
tool/api/build/docs/javadoc/index.htmlin your favorite browser.
Directory bugs contains all the JIT bugs we found using JAttack,
each of which contains a template program, a generated program and a
minimized program to expose the bug.
If you find JIT bugs using JAttack, we would be happy to add your findings to this list. Please open a PR with a link to your bug.
- JDK-8239244 (Login required): See CVE-2020-14792
- JDK-8258981: JVM crash with # Problematic frame: # V [libjvm.so+0xdc0df5] ok_to_convert(Node*, Node*)+0x15
- JDK-8271130 (Login required): See CVE-2022-21305
- JDK-8271276: C2: Wrong JVM state used for receiver null check
- JDK-8271459: C2: Missing NegativeArraySizeException when creating StringBuilder with negative capacity
- JDK-8271926: Crash related to Arrays.copyOf with # Problematic frame: # V [libjvm.so+0xc1b83d] NodeHash::hash_delete(Node const*)+0xd
- JDK-8297730: C2: Arraycopy intrinsic throws incorrect exception
If you use JAttack in your research, we request you to cite our ASE'22 paper (which won an ACM SIGSOFT Distinguished Paper Award) and ICSE'23 Demo paper.
@inproceedings{zang22jattack,
author = {Zang, Zhiqiang and Wiatrek, Nathaniel and Gligoric, Milos and Shi, August},
title = {Compiler Testing using Template {J}ava Programs},
booktitle = {International Conference on Automated Software Engineering},
pages = {23:1--23:13},
year = {2022},
doi = {10.1145/3551349.3556958},
}
@inproceedings{zang23jattacktool,
author = {Zang, Zhiqiang and Yu, Fu-Yao and Wiatrek, Nathaniel and Gligoric, Milos and Shi, August},
title = {{JA}ttack: {J}ava {JIT} Testing using Template Programs},
booktitle = {International Conference on Software Engineering, Tool Demonstrations Track},
pages = {6--10},
year= {2023},
doi = {10.1109/ICSE-Companion58688.2023.00014},
}Let me (Zhiqiang Zang) know if you have any questions.