ZZSpec

A Scala library with great ZIO integration to help you easily write high-level integration/black box tests. zzspec helps you grow your confidence in the correctness of software with ZIO (test).

Why?

Testing at a high level means freedom of implementation and refactoring, this is something all engineers love. Also, we love functional programming, Scala and ZIO ❤️.

Tests should be the compass to study and prove the correctness of the behaviour of a system.

Getting started

To get started using zzspec, a good place to look is the integration tests of zzspec itself.

See the PostgreSQL spec PostgreSQL spec here.

See the OpenSearch spec OpenSearch spec here.

See the Kafka spec Kafka spec here.

See the MockServer spec MockServer here.

See an example “real world” test which combines MockServer and PostgreSQL here, with prepared DB state and post-run DB checks: docs/examples/real-world-postgre-and-mockserver.scala.

Vision

zzspec considers the following to be essential for good testing:

• allow engineers to focus on the actual test flow at a high level of reasoning, with an expressive language (DSL), abstracting away certain details (always with an escape hatch).
• concurrent testing should work and be the default (with random order of text execution to prove correctness)
• speed of test execution, having ZIO test really helps us by leveraging layers (per suite, per test etc.)
• tests should be first class citizens of your endeavours, and a good way to teach more details about the system to others
• up to date dependencies (e.g. docker container versions, 3rd party code)
• pragmatism as starting point and on a case by case basis, decide what kind of tests do enough coverage of the system and control flow. traditional code coverage metrics are not always applicable.
• contract-driven testing is a great way to ensure data schemas and values remain consistent, when testing with external systems for example. zzspec can help and has functionalities for this.
• mocks are most of the time not the right tool to test our systems and logic
• no marrying the test structure to code structure. No testing every individual class when not needed.
• ensure we test functionalities, almost never test how the code is written / implemented (this gives flexibility and freedom to refactor).
• attempt to test all “user paths” and possible interactions with the system, good and bad, low load high load, etc.
• write many unit tests and property based tests where it makes sense. Preferably auto-generated test cases, with a set of inputs.

Installing

You can install zzspec just like any other Maven/Nexus package, by adding it as dependency to your project:

libraryDependencies += ("io.github.jjba23" %% "zzspec" % "0.9.3" % Test) // or newer

ZZSpec will also pull some libraries with it, including ZIO, Circe and testcontainers. Artifacts are hosted on Maven central.

Status of zzspec

Currently, this codebase and API is in a semi-stable state, with no major backward-compatible changes planned. zzspec will start to use semantic versioning and commit to a stable API starting at v1.0.0 (WIP).

Unit testing has its place (aided by property based)

Unit testing is an invaluable tool that should also be used in parallel to zzspec and other high level tests. It is very easy to use, useful and gives also good information about a system.

Ideally, your “core domain” should be fully tested, and your “business logic” should be encoded in more pure code, ideally as data.

When relevant, you should leverage great property based testing tools and data generators. E.g. ZIO test, to ensure all edge cases are caught in tests.

Dealing with dependencies

When we start using external dependencies in a system like databases, caches, external APIs, etc. You have 3 choices:

• create and use mocks ☹️
• create and use stubs (dummy implementations) 😼
• use real dependencies 👍

Mocks are inherently evil : Mocks are generally painful to write, read, debug and maintain.

They should be avoided when possible, we should use real implementations for most tests to a system. testcontainers is a great library for this purpose, and we use it extensively.

Often mocks are used for lack of better tooling, or simply due to habit (specially if you have Java experience). Lots of developers coming from a traditional enterprisy JDK environment know what we mean.

This tendency of doing one class per file, and creating tests for every single class and every single method along with extensive Mockito ideology.

What’s in it for me ?

Easier and simpler tests of the entire system, tests have lower complexity.

Testing a system becomes simpler and we can cover many more “real” edge cases.

Easy to cover 100% of a “user flow” or a “data flow”.

Low chance of false positives (partly thanks to avoiding mocks too). This allows for a good test-driven development approach, and more confidence in product.

Testers require less technical knowledge, programming or IT skills and do not need to learn all nitty gritty implementation details of the system.

More loose coupling from the code means more freedom of implementation + refactor

Contributing

Please feel free to open a pull request, GitHub issue or reach out to me personally (Joe - jjbigorra@gmail.com).

By contributing, your work will be protected under the GNU Lesser General Public License v3.0.

Project management - Backlog

Developing benchmarking capabilities (HTTP, Kafka, IO, Elastic, PostgreSQL)

Add Kafka and Postgres windowing test capabilities

Use more capabilities of ZIO test and its data generators

Make container layers more customizable and configurable

Add Kafka Schema Registry container and Protobuf testing facilities

Auto-generate and publish Scaladoc and documentation in Github Pages (with CI)

Allow “initial state” in PostgreSQL and in Opensearch more easily

Considering not using Jackson to work with JSON

Auto-tag and publish artifacts to Maven (with CI)

sbt publishSigned -> sbt sonatypeBundleRelease

✅ Work done

Move to Slick instead of Scalikejdbc

Finalize initial phase Kafka testing

Add vision statement and improve README