update.

cmu-pasta · Jul 26, 2024 · ee066d3 · ee066d3
1 parent 43cc30a
commit ee066d3
Show file tree

Hide file tree

Showing 4 changed files with 121 additions and 72 deletions.
diff --git a/README.md b/README.md
@@ -1,83 +1,122 @@
-# Run Fray
+# Fray: An Efficient General-Purpose Concurrency Testing Platform for the JVM
 
-- To run Fray, you first need to build the project and create a instrumented JDK.
+## Build Fray
+
+- Run the following commands to build Fray
 
 ```bash
-./gradlew build
+./gradlew build -x test
 ./gradlew jlink
 ./gradlew shadowJar
 ```
 
-- Next, you can run Fray!
-  - You may follow the `examples` folder to understand how to run sfuzz using gradle.
-  - You can also run sfuzz through command line. We created a wrapper `bin/sfuzz` for you
-    to setup environment.
-    - Command: `bin/sfuzz CLASS_PATH ARGS_TO_FRAY`
-    - For example, to run `bin/sfuzz` for `examples` program, you need
-        - `./bin/sfuzz ./examples/build/classes/java/main example.Main -o ./examples/build/report --scheduler fifo`
-    - To understand Fray args you can:
-        - `./bin/sfuzz dummy --help`
-
-
-# Architecture
-
-![image](./assets/fray-arch.png)
-
-
+## Run Fray
 
 
-# Runtime
+### Application Configuration
 
-Runtime should be **super** light weight. It should not include any dependencies. It should not call any JDK methods.
-It is a delegate between **core** and **instrumentation**.
+First you need to provide a configuration file for the application you want to test. The configuration file should be in the following format:
 
-# Scheduling
-
-It is operations' responsibility to decide whether an operation triggers a reschedule.
-
-# Threads
-
-Each thread should only try to block itself.
-
-We should reschedule a thread **before** a non-local operation. This gives us the opportunity to
-schedule operations based on the knowledge of their effects.
+```json
+{
+  "executor": {
+    "clazz": "com.example.Main",
+    "method": "main",
+    "args": ["arg1", "arg2"],
+    "classpaths": ["path/to/your/application.jar"],
+    "properties": {"PROPERTY1": "VALUE1", "PROPERTY2": "VALUE2"}
+  },
+  "ignore_unhandled_exceptions": false,
+  "timed_op_as_yield": false,
+  "interleave_memory_ops": false,
+  "max_scheduled_step": -1
+}
+```
 
-Fray should proactively maintain the state of each thread instead of reactively.
+- `executor` defines the entrypoint and environment of the application you want to test.
+  - `clazz`: the main class of the application.
+  - `method`: the main method of the application.
+  - `args`: the arguments to the main method.
+  - `classpaths`: the classpaths of the application.
+  - `properties`: the system properties of the application.
+- `ignore_unhandled_exceptions`: whether to treat unhandled exceptions as failures.
+- `timed_op_as_yield`: whether to treat timed operations as yields otherwise they will be treated as no timeout op.
+- `interleave_memory_ops`: whether to interleave memory operations.
+- `max_scheduled_step`: the maximum number of scheduled steps. And Fray will throw `LivenessException` if the number of scheduled steps exceeds this value. If the value is -1, then there is no limit.
 
 
+### Fray Configuration
 
-# Locks
+You may use the following gradle task to run Fray:
 
-Thread blocking/resuming should be managed at Thread level instead of sync information level.
+```bash
+./gradlew runFray -PconfigPath=path/to/your/application_config.json -PextraArgs="extra args passed to Fray"
+```
 
-We should use `wati` and `notify` to block/unblock a thread because we will instrument semaphores as well.
+Here are the available extra args:
 
-```java
-synchronized (monitor) {
-    monitor.wait();
-}
-synchronized (monitor) {
-        monitor.notify();
-}
+```
+Options:
+  -o=<text>                Report output directory.
+  -i, --iter=<int>         Number of iterations.
+  -f, --full               If the report should save full schedule. Otherwise,
+                           Fray only saves schedules points if there are more
+                           than one runnable threads.
+  -l, --logger=(json|csv)  Logger type.
+  --scheduler=(replay|fifo|pos|random|pct)
+                           Scheduling algorithm.
+  --no-fray                Runnning in no-Fray mode.
+  --explore                Running in explore mode and Fray will continue if a
+                           failure is found.
+  --no-exit-on-bug         Fray will not immediately exit when a failure is
+                           found.
+  --run-config=(cli|json)  Run configuration for the application.
+  -h, --help               Show this message and exit
 ```
 
-`Thread.join` should be handled the same way as `Object.wait`
+### Output 
 
-# Instrumentation
+The output of Fray will be saved in the `output` directory. The output directory contains the following files:
+
+- `output.txt`: the Fray of the testing.
+- `schedule_{id}.json/csv`: the schedule you can replay.
 
-Instrumentation should be as simple as possible. Ideally only static method calls.
+## Replay a buggy schedule
 
-We should have different classes to instrument different types of code so that they can be disabled easily.
+Once Fray finds a bug as indicated in `output.txt`. You may replay it by providing the corresponding schedule.
 
-# Synchronization
+```bash
+./gradlew runFray -PconfigPath=path/to/your/application_config.json -PextraArgs="--scheduler=replay --path=path/to/schedule.json"
+```
 
-We should **strictly** enforce sequential execution. If a new thread is created/resumed, we
-should wait for them to finish bootstrap and fully stopped before proceeding.
+## Example
 
-Thread rescheduling can only be called by the current scheduled thread.
+We provide an example application in the `integration-tests/src/main/java/example/FrayExample.java`. You may run the following command to test it:
 
+```bash
+mkdir out
+javac integration-tests/src/main/java/example/FrayExample.java -d out
+echo '{
+  "executor": {
+    "clazz": "example.FrayExample",
+    "method": "main",
+    "args": [],
+    "classpaths": ["CURRENT_DIR/out/"],
+    "properties": {}
+  },
+  "ignore_unhandled_exceptions": false,
+  "timed_op_as_yield": false,
+  "interleave_memory_ops": false,
+  "max_scheduled_step": -1
+}' | sed "s|CURRENT_DIR|$(pwd)|g" > out/config.json
+./gradlew runFray -PconfigPath="out/config.json" -PextraArgs="--iter=1000 --logger=json --scheduler=random -o=/tmp/fray-example/"
+```
 
-# Limitations
+And you may see a failure `AssertionError` or `DeadlockException` in stdout. In `/tmp/report/`, you will see one schedule is 
+saved which will lead to the failure.
 
-- We cannot remove monitors in JDK, but we can for applications.
+To replay that schedule, you may run the following command:
 
+```bash
+./gradlew runFray -PconfigPath="out/config.json" -PextraArgs="--iter=1000 --logger=json --scheduler=replay --path=/tmp/fray-example/schedule_XXX.json"
+```
diff --git a/core/src/main/kotlin/cmu/pasta/fray/core/command/Configuration.kt b/core/src/main/kotlin/cmu/pasta/fray/core/command/Configuration.kt
@@ -107,7 +107,7 @@ sealed class ScheduleAlgorithm(name: String) : OptionGroup(name) {
 }
 
 class Replay : ScheduleAlgorithm("replay") {
-  val path by option().file().required()
+  val path by option("-p", "--path", help = "Path to the saved schedule.").file().required()
 
   override fun getScheduler(): Scheduler {
     return ReplayScheduler(Schedule.fromString(path.readText(), path.extension == "json", false))
@@ -141,24 +141,37 @@ class PCT : ScheduleAlgorithm("pct") {
 }
 
 class MainCommand : CliktCommand() {
-  val report by option("-o").default("/tmp/report")
-  val iter by option("-i", "--iter", help = "Number of iterations").int().default(1)
-  val fullSchedule by option("-f", "--full").flag()
+  val report by option("-o", "--output", help = "Report output directory.").default("/tmp/report")
+  val iter by option("-i", "--iter", help = "Number of iterations.").int().default(1)
+  val fullSchedule by
+      option(
+              "-f",
+              "--full",
+              help =
+                  "If the report should save full schedule. Otherwise, Fray only saves schedules points if there are more than one runnable threads.")
+          .flag()
   val logger by
-      option("-l", "--logger").groupChoice("json" to JsonLoggerOption(), "csv" to CsvLoggerOption())
+      option("-l", "--logger", help = "Logger type.")
+          .groupChoice("json" to JsonLoggerOption(), "csv" to CsvLoggerOption())
   val scheduler by
-      option()
+      option(help = "Scheduling algorithm.")
           .groupChoice(
               "replay" to Replay(),
               "fifo" to Fifo(),
               "pos" to POS(),
               "random" to Rand(),
               "pct" to PCT())
-  val noFray by option("--no-fray").flag()
-  val exploreMode by option("--explore").flag()
-  val noExitWhenBugFound by option("--no-exit-on-bug").flag()
+  val noFray by option("--no-fray", help = "Runnning in no-Fray mode.").flag()
+  val exploreMode by
+      option(
+              "--explore",
+              help = "Running in explore mode and Fray will continue if a failure is found.")
+          .flag()
+  val noExitWhenBugFound by
+      option("--no-exit-on-bug", help = "Fray will not immediately exit when a failure is found.")
+          .flag()
   val runConfig by
-      option()
+      option(help = "Run configuration for the application.")
           .groupChoice(
               "cli" to CliExecutionConfig(),
               "json" to JsonExecutionConfig(),

diff --git a/core/src/main/kotlin/cmu/pasta/fray/core/scheduler/ReplayScheduler.kt b/core/src/main/kotlin/cmu/pasta/fray/core/scheduler/ReplayScheduler.kt
@@ -17,19 +17,13 @@ class ReplayScheduler(val schedule: Schedule) : Scheduler {
       // scheduler")
     }
     val choice = schedule.choices[index]
-    if (threads.map { it.index }.toList() != choice.enabledIds) {
-      println("?")
-    }
     assert(threads.map { it.index }.toList() == choice.enabledIds)
     assert(choice.enabled == threads.size)
 
     val selected = threads[choice.selected % threads.size]
-    //    assert(choice.threadId == selected.index)
+    assert(choice.threadId == selected.index)
     index += 1
-    if (selected.pendingOperation.toString().split("@")[0] != choice.operation.split("@")[0]) {
-      println("?")
-    }
-    assert(selected.pendingOperation.toString() == choice.operation)
+    assert(selected.pendingOperation.toString().split("@")[0] == choice.operation.split("@")[0])
     return selected
   }
 

diff --git a/examples/build.gradle.kts b/examples/build.gradle.kts
@@ -41,7 +41,7 @@ tasks.withType<JavaExec> {
 }
 
 tasks.register<JavaExec>("runFray") {
-  val configPath = properties["configPath"] as String? ?: ""
+  var configPath = properties["configPath"] as String? ?: ""
   val extraArgs = when (val extraArgs = properties["extraArgs"]) {
     is String -> {
       val pattern = Pattern.compile("""("[^"]+"|\S+)""")
@@ -54,6 +54,9 @@ tasks.register<JavaExec>("runFray") {
     }
     else -> emptyList()
   }
+  if (!File(configPath).isAbsolute) {
+    configPath = System.getProperty("user.dir") + "/" + configPath
+  }
   args = listOf("--run-config", "json", "--config-path", configPath) + extraArgs
 }