Add support for async fixtures

Previously, it was not possible to create fixtures that loaded
asynchronously. It was possible to work around this limitation on the
JVM by awaiting on futures, but there was no workaround for Scala.js.
This commit adds support to return futures (and anything that converts
to futures) from the beforeAll/beforeEach/afterEach/afterAll methods.

Supersedes #418. This commit is
inspired by that PR but uses a different approach:

- No new `AsyncFixture[T]` type. This simplies the logic in
  `MUnitRunner` and reduces the size of the MUnit public API.
  The downside is that we introduce a breaking change to the existing
  `Fixture[T]` API, which will require bumping the version to v1.0.0
  for the next release.
- This approach supports any `Future`-like values by hooking into the
  default evaluation of test bodies via `munitValueTransforms`.

Co-authored-by: Daniel Esik <e.danicheg@yandex.ru>

docs/fixtures.md

@@ -57,12 +57,12 @@ to reusable or ad-hoc fixtures when necessary.
 ## Reusable test-local fixtures
 
 Reusable test-local fixtures are more powerful than functional test-local
-fixtures because they can declare custom logic that gets evaluted before each
+fixtures because they can declare custom logic that gets evaluated before each
 local test case and get torn down after each test case. These increased
 capabilities come at the price of ergonomics of the API.
 
-Override the `beforeEach()` and `afterEach()` methods in the `Fixture[T]` trait
-to configure a reusable test-local fixture.
+Override the `beforeEach()`, `afterEach()` and `munitFixtures` methods in the
+`Fixture[T]` trait to configure a reusable test-local fixture.
 
 ```scala mdoc:reset
 import java.nio.file._
@@ -120,6 +120,58 @@ class MySuite extends munit.FunSuite {
 }
 ```
 
+## Asynchronous fixtures
+
+Return a `Future`-like value from the methods `beforeAll`, `beforeEach`,
+`afterEach` and `afterAll` to make an asynchronous fixture. By default, only
+`Future[_]` values are recognized. Override `munitValueTransforms` to add
+support for writing async fixture with other `Future`-like types, see
+[declare async tests](tests.md#declare-async-test) for more details.
+
+```scala mdoc:reset
+import java.nio.file._
+import java.sql.Connection
+import java.sql.DriverManager
+import munit._
+import scala.concurrent.Future
+import scala.concurrent.ExecutionContext.Implicits.global
+
+class AsyncFilesSuite extends FunSuite {
+
+  // Test-local async fixture
+  val file = new Fixture[Path]("files") {
+    var file: Path = null
+    def apply() = file
+    override def beforeEach(context: BeforeEach): Future[Unit] = Future {
+      file = Files.createTempFile("files", context.test.name)
+    }
+    override def afterEach(context: AfterEach): Future[Unit] = Future {
+      // Always gets called, even if test failed.
+      Files.deleteIfExists(file)
+    }
+  }
+
+  // Suite-local async fixture
+  val db = new Fixture[Connection]("database") {
+    private var connection: Connection = null
+    def apply() = connection
+    override def beforeAll(): Future[Unit] = Future {
+      connection = DriverManager.getConnection("jdbc:h2:mem:", "sa", null)
+    }
+    override def afterAll(): Future[Unit] = Future {
+      connection.close()
+    }
+  }
+
+  override def munitFixtures = List(file, db)
+
+  test("exists") {
+    // `file` is the temporary file that was created for this test case.
+    assert(Files.exists(file()))
+  }
+}
+```
+
 ## Ad-hoc test-local fixtures
 
 Override `beforeEach()` and `afterEach()` to add custom logic that should run

docs/tests.md

@@ -86,8 +86,8 @@ test("buggy-task") {
 ```
 
 Since tasks are lazy, a test that returns `LazyFuture[T]` will always pass since
-you need to call `run()` to start the task execution. Override `munitValueTransforms`
-to make sure that `LazyFuture.run()` gets called.
+you need to call `run()` to start the task execution. Override
+`munitValueTransforms` to make sure that `LazyFuture.run()` gets called.
 
 ```scala mdoc
 import scala.concurrent.ExecutionContext.Implicits.global
@@ -118,9 +118,10 @@ parallel test suite execution in sbt, add the following setting to `build.sbt`.
 Test / parallelExecution := false
 ```
 
-In case you do not run your tests in parallel, you can also disable buffered logging, 
-which is on by default to prevent test results of multiple suites from appearing interleaved.
-Switching buffering off would give you immediate feedback on the console while a suite is running.
+In case you do not run your tests in parallel, you can also disable buffered
+logging, which is on by default to prevent test results of multiple suites from
+appearing interleaved. Switching buffering off would give you immediate feedback
+on the console while a suite is running.
 
 ```sh
 Test / testOptions += Tests.Argument(TestFrameworks.MUnit, "-b")
@@ -199,9 +200,9 @@ bug report but don't have a solution to fix the issue yet.
 
 ## Customize evaluation of tests with tags
 
-Override `munitTestTransforms()` to extend the default behavior for how test bodies are
-evaluated. For example, use this feature to implement a `Rerun(N)` modifier to
-evaluate the body multiple times.
+Override `munitTestTransforms()` to extend the default behavior for how test
+bodies are evaluated. For example, use this feature to implement a `Rerun(N)`
+modifier to evaluate the body multiple times.
 
 ```scala mdoc
 case class Rerun(count: Int) extends munit.Tag("Rerun")
@@ -228,9 +229,9 @@ class MyRerunSuite extends munit.FunSuite {
 }
 ```
 
-The `munitTestTransforms()` method is similar to `munitValueTransforms()` but is different in
-that you also have access information about the test in `TestOptions` such as
-tags.
+The `munitTestTransforms()` method is similar to `munitValueTransforms()` but is
+different in that you also have access information about the test in
+`TestOptions` such as tags.
 
 ## Customize test name based on a dynamic condition
 
@@ -316,46 +317,3 @@ abstract class BaseSuite extends munit.FunSuite {
 class MyFirstSuite extends BaseSuite { /* ... */ }
 class MySecondSuite extends BaseSuite { /* ... */ }
 ```
-
-## Roll our own testing library with `munit.Suite`
-
-The `munit.FunSuite` class comes with a lot of built-in functionality such as
-assertions, fixtures, `munitTimeout()` helpers and more. These features may not
-be necessary or even desirable when writing tests. You may sometimes prefer a
-smaller API.
-
-Extend the base class `munit.Suite` to implement a minimal test suite that
-includes no optional MUnit features. At its core, MUnit operates on a data
-structure `GenericTest[TestValue]` where the type parameter `TestValue`
-represents the return value of test bodies. This type parameter can be
-customized per-suite. In `munit.FunSuite`, the type parameter `TestValue` is
-defined as `Any` and `type Test = GenericTest[Any]`.
-
-Below is an example custom test suite with `type TestValue = Future[String]`.
-
-```scala
-class MyCustomSuite extends munit.Suite {
-  override type TestValue = Future[String]
-  override def munitTests() = List(
-    new Test(
-      "name",
-      // compile error if it's not a Future[String]
-      body = () => Future.successful("Hello world!"),
-      tags = Set.empty[Tag],
-      location = implicitly[Location]
-    )
-  )
-}
-```
-
-Some use-cases where you may want to define a custom `munit.Suite`:
-
-- implement APIs that mimic testing libraries to simplify the migration to MUnit
-- design stricter APIs that don't use `Any`
-- design purely functional APIs with no publicly facing side-effects
-
-In application code, it's desirable to use strong types avoid mutable state.
-However, it's not clear that those best practices yield the same cost/benefit
-ratio when writing test code. MUnit intentionally exposes types such `Any` and
-side-effecting methods like `test("name") { ... }` because they subjectively
-make the testing API nice-to-use.

munit/shared/src/main/scala/munit/BeforeEach.scala

@@ -0,0 +1,9 @@
+package munit
+
+class BeforeEach(
+    val test: Test
+) extends Serializable
+
+class AfterEach(
+    val test: Test
+) extends Serializable

munit/shared/src/main/scala/munit/Fixture.scala

@@ -0,0 +1,27 @@
+package munit
+
+/**
+ * @param name The name of this fixture, used for displaying an error message if
+ * `beforeAll()` or `afterAll()` fail.
+ */
+abstract class Fixture[T](val fixtureName: String) {
+
+  /** The value produced by this suite-local fixture that can be reused for all test cases. */
+  def apply(): T
+
+  /** Runs once before the test suite starts */
+  def beforeAll(): Any = ()
+
+  /**
+   * Runs before each individual test case.
+   * An error in this method aborts the test case.
+   */
+  def beforeEach(context: BeforeEach): Any = ()
+
+  /** Runs after each individual test case. */
+  def afterEach(context: AfterEach): Any = ()
+
+  /** Runs once after the test suite has finished, regardless if the tests failed or not. */
+  def afterAll(): Any = ()
+
+}

munit/shared/src/main/scala/munit/FunSuite.scala

@@ -17,8 +17,6 @@ abstract class FunSuite
     with SuiteTransforms
     with ValueTransforms { self =>
 
-  final type TestValue = Future[Any]
-
   final val munitTestsBuffer: mutable.ListBuffer[Test] =
     mutable.ListBuffer.empty[Test]
   def munitTests(): Seq[Test] = {

munit/shared/src/main/scala/munit/FutureFixture.scala

@@ -0,0 +1,15 @@
+package munit
+
+import scala.concurrent.Future
+
+/**
+ * Extend this class if you want to create a Fixture where all methods have `Future[Any]` as the result type.
+ */
+abstract class FutureFixture[T](name: String) extends Fixture[Future[T]](name) {
+  override def beforeAll(): Future[Any] = Future.successful(())
+  override def beforeEach(context: BeforeEach): Future[Any] =
+    Future.successful(())
+  override def afterEach(context: AfterEach): Future[Any] =
+    Future.successful(())
+  override def afterAll(): Future[Any] = Future.successful(())
+}