BAR :: Behaviour ARchive

Table of Contents

• 2 Minute API Consumer Intro
  • Generating Test Data (Consumer)
• 2 Minute API Provider Intro
  • Managing Test Data (Provider)
• Credentials
• Full Dependency Injection
• SYSTEM Level Tests
• Full Documentation

Code-first, low ceremony Consumer Driven Contracts test tool for Java natives.

Let’s pick that apart:

"code-first": there’s a lot of debate about designing APIs schema-first (a.k.a. API-first) vs. code-first. I absolutely agree that (not only for public APIs) it’s very important to design APIs in a consistent and easily approachable way. I even think that it’s actually more important to capture the real use-cases of real consumers of an API. It’s just way too easy to design something that looks good as a schema but is clumsy to use in practice. And by lowering the bar to specify the (changes to an) API as much as possible, consumers are invited to help evolve the API in a pragmatic way that is easy to use.

"low ceremony": use it just like you’d use Mockito to test your client code. Easily scale that from Unit to Integration Test^[1], i.e. integrate your client code with an actual http server mocking the responses the tests expect, in order to also cover all the (de)serialization involved. Or even go for System Tests, where your service runs in a production-like environment and calls a pre-deployed mock service. Either way, this helps clients to test their own code and…​

"Consumer Driven Contracts": …​ as a side effect, WunderBar also records the expected http interactions into a Behavior ARchive bar, a simple zip file containing properties files for the method, uri, and headers, and json files for the bodies^[2]. Hand this file over to the API providers, so they can use it to verify compliance with the clients' requirements. See the article linked above. Just to make it clear: just because the consumer drives the API contract doesn’t mean that the provider is expected to comply blindly. It’s only a starting point for the "contract negotiation". The provider has to evolve a domain model that is consistent over all consumers: the BAR files are not a replacement for talking; they just bring it to the precision that the machines foolishly insist on; and a long-term test suite of acceptance tests. Starting from the API Consumer perspective is actually just a very natural way of defining an API: from the real requirements. But you must generally take care that no client application logic seeps into your backend domain, i.e. views, flows, etc.

"Java native": The consumer/client can use REST via MicroProfile REST Client or GraphQL via SmallRye GraphQL Client, while the server can use any technology stack that can run JUnit 5 tests (or you can use the bar files directly). For the details see below.

2 Minute API Consumer Intro

Say you’re developing an Order service that uses data from a Product service, i.e. it consumes an API. You probably have a ProductsGateway or ProductsResolver class that uses a ProductsClient interface with annotations from MP Rest Client or SmallRye GraphQL Client that you unit-test with Mockito:

@RegisterRestClient @Path("/products")
public interface ProductsClient {
    @GET @Path("/{id}")
    Product product(@PathParam("id") String id);
}

// or alternatively

@GraphQLClientApi
public interface ProductsClient {
    Product product(String id);
}

// test

@ExtendWith(MockitoExtension.class)
class ProductsGatewayMockitoTest {
    @Mock ProductsClient products;
    @InjectMocks ProductsGateway gateway;

    @Test void shouldGetProduct() {
        given(products.product(PRODUCT_ID)).willReturn(PRODUCT);

        var response = gateway.product(ORDER_ITEM);

        then(response).usingRecursiveComparison().isEqualTo(PRODUCT);
    }
}

Instead of using the Mockito extension with its annotations and the static given method import, you can simply use those from WunderBar. They are more limited than Mockito, but have the same style, same behavior, just different logs:

@WunderBarApiConsumer
class ProductsGatewayTest {
    @Service ProductsClient products;
    @SystemUnderTest ProductsGateway gateway;

    @Test void shouldGetProduct() {
        given(products.product(PRODUCT_ID)).returns(PRODUCT);

        var response = gateway.product(ITEM);

        then(response).usingRecursiveComparison().isEqualTo(PRODUCT);
    }
}

Note that you can also use the Mockito syntax given(…​).willReturn(…​), but using returns makes sure you don’t accidentally use the given from Mockito.

To make things interesting, you can change the @WunderBarApiConsumer annotation to @WunderBarApiConsumer(level = INTEGRATION) (or simply change the test name to end with IT, short for Integration Test): WunderBar now starts a mock server exposing the behavior you just stubbed, i.e. it will reply to your real http request with the proper product http response. No code changes needed, and now it fully tests your REST or GraphQL client annotations and (de)serialization of your POJOs.

This is nice, but as a welcome side effect, it records the requests and responses you need for your code to work, and saves it in a wunder.bar file (Behavior ARchive). Give this file to your API provider, so they can check if their service complies to your requirements. You can even deploy this file, together with your other maven artefacts, e.g. by using the attach-artifact goal of the build-helper-maven-plugin; for an example, look at the pom in the demo/order submodule. Note that you’ll get a Failed to install artifact error when running mvn clean install -DskipTests, because you skip the tests that generate the bar files, so the plugin can’t attach them. Most of the time, executing clean is not necessary (and slows your build down); and running install before releasing your software is only useful for libraries, not for applications; simply run mvn package -DskipTests instead. If you still need the install, you can also pass -Dbuildhelper.skipAttach.

You can disable recording for one stub by calling withoutRecording, e.g. when testing the error handling in your consumer code, so this interaction is not an expected behavior of the API provider.

Generating Test Data (Consumer)

Generating good test data can become tedious; numbers should be rather small, positive, and (above all) unique, so the value are easy to handle and recognize. WunderBar provides an extensible mechanism to generate test data that fulfills these requirements, using a simple counter starting for every test. And it logs the values it generated for what purpose, so you can easily find the source for a value. You can generate a value for a field or parameter by annotating it as @Some.

@WunderBarApiConsumer
class ProductResolverTest {
    @Test void shouldUpdateProduct(@Some int newPrice) {
        given(/*...*/).returns(product.withPrice(newPrice));

        var resolvedProduct = resolver.productWithPriceUpdate(item, newPrice);

        then(resolvedProduct).usingRecursiveComparison()
            .isEqualTo(product.withPrice(newPrice));
    }
}

@Some int newPrice could also be a field, and it would work exactly the same.

Note how the newPrice parameter is used throughout the test. This makes it easier for the reader of your code to understand which test value is which. Sometimes, you’ll want to provide some constant value; as they should be distinct from the generated values, use values below 100, which is where @Some will start counting from by default. And the generator will fail, if you generate values beyond Short.MAX_VALUE = 32767 = 215-1 = 0x7FFF; generating a byte fails sooner, obviously.

Out-of-the-box, you can generate the primitive types byte, char, short, int, long, float, double (or their wrapper types Integer, etc.), and some basic types like String, URI, LocalDateTime, etc. (see here for the full list), but obviously not boolean: they can hardly be considered unique. You can change the starting point by calling SomeBasics#reset, e.g. in a @BeforeEach. You can also generate List or Set, which will contain exactly one generated element, and instances of arbitrary classes, which will recursively generate values for every field.

To generate your own data, e.g., @Some Product product, you can register your own generator class: @Register(SomeProducts.class), where SomeProducts implements SomeData. The @Some annotation takes an optional list of String tags that are passed to custom generators, along with the AnnotatedElement location, so it can fine-control what data it should generate, e.g. to generate invalid objects.

You can also inject an instance of SomeGenerator into your generator’s constructor to dynamically generate other values you depend on, or to look up the location or tags of an actual value. For a full example see here.

2 Minute API Provider Intro

When you implement an API (i.e. you provide it), you can load a suite of tests that has been stored in a wunder.bar file, and run them against your service:

@WunderBarApiProvider(baseUri = "http://localhost:8080")
class ConsumerDrivenAT {
    @TestFactory DynamicNode orderTests() {
        return findTestsIn("wunder.bar");
    }
}

There are several ways to load bar files; e.g., you can also load them from maven coordinates. See the public methods in the WunderBarTestFinder class for details.

The requirements will be more specific than your service, but that’s a good thing: thankfully, your service will be lenient in some cases; e.g. it accepts different content type encodings, like ISO-8859-1 or utf-8. In this way, a client can change some details of its technical requirements, e.g. by requesting a different encoding or even content type (e.g. json instead of xml); as long as your service supports it, the tests continue to pass. And if it doesn’t support it, it will show up as soon as the new version of the bar file runs.

If the test data in your service is static and matches the expectations of your clients/consumers, that’s it! But to be honest, managing test data is generally a nastily complex issue, and WunderBar can help, but can’t make it go away completely.

Managing Test Data (Provider)

Consumer Driven Contract testing is about the structure of the data, the API. But the requests and responses in a bar file also contain some more or less random data itself. The most common reflex is to create exactly that data in your test system, which is okay as long as the data is very static. But test data often changes or is even deleted for various reasons: some data simply times out, other data is changed by manual as well as automated tests, etc. This demands coordination between different teams, resulting in high effort and brittle tests: they sporadically break without exposing a real bug anywhere but in this communication between people.

Your tests will be much more maintainable, if set up (and maybe clean up) data in your service to match the consumers' requirements, i.e. mostly putting the expected response into your system. You can do so by using some mutating APIs of your service, or by storing and deleting the data directly into your database, or by defining an extra test backdoor API for your service: either way, you’ll need do this kind of test setup before every test in the BAR (and maybe some cleanup thereafter). To do so, just define a method, annotated as @BeforeInteraction^[3], taking a single parameter of type HttpRequest, HttpResponse, or HttpInteraction (which basically just bundles a request and response).

In addition to storing the data in your system, you can also manipulate the request or the expected response by returning an HttpInteraction, HttpRequest, or HttpResponse from your BeforeInteraction method to modify the interaction, e.g. to replace the dummy credentials from the bar file (see below) with real credentials your service will accept. The HttpRequest and HttpResponse classes help here with a bunch of convenient methods.

This works nicely when reading data, but you’ll need more for mutating operations; e.g. when a test creates a record in a database, it most often will also generate something like a primary key, which will not match the key in the expected response.^[4] To manipulate the expected response to match a value from the actual response, write a method annotated as @AfterInteraction. You can’t return a request here anymore (as it’s already done), but get the actual response with a @Actual annotated HttpResponse parameter and use that to manipulate the expected result as needed.

You can also filter the tests to actually run, by annotating a method as @BeforeDynamicTest, and returning the List<HttpInteraction> with tests removed as you wish. You could even add your own, e.g. by duplicating (and then probably modifying) an existing one.

Writing your acceptance tests in this way makes your testing more robust, as you don’t have to agree with the consumers of your APIs on any volatile and intransparent assumptions about the test data, e.g. what ids or data fields result in what behavior. For a fully running example, see the demo ConsumerDrivenAT.

Credentials

bar files never contain the secrets of a real Authorization header ^[5]. They could contain random values for integration tests, without adding any benefit; for system-level tests (see below) against a real service, the interactions would even contain real credentials. So WunderBar only writes dummy values instead.

For a GraphQL client, you can use the @AuthorizationHeader annotation to read the configuration from an MP Config property; but you don’t have to actually provide those for an integration test, as they won’t be written anyway; a dummy value will be written instead. OTOH, a @Header(name = "Authorization") works normally (but won’t be written either).

On the API provider side, the acceptance test has to replace this value with real credentials, e.g. by returning a modified HttpRequest in a @BeforeInteraction method.

Full Dependency Injection

Using the @SystemUnderTest annotation performs only a very limited form of dependency injection. For more complex dependency requirements, it may be appropriate to use, e.g., weld-junit5 as a fully blown CDI testing environment. To do so, do the following steps:

1. add a test scope dependency on org.jboss.weld:weld-junit5,

2. annotate your test class with @EnableWeld after (this is important) the @WunderBarApiConsumer annotation,

3. instead of @SystemUnderTest, use the CDI @Inject annotation, and

4. build a WeldInitiator with your classes, and for the services, add a mock bean with a delayed create producer of the WunderBar-mocked service field.

This sums up like this:

@WunderBarApiConsumer
@EnableWeld
class ProductsResolverWeldIT {
    @Service Products products;
    @Inject ProductsResolver resolver;

    @WeldSetup
    WeldInitiator weld = WeldInitiator.from(ProductsResolver.class, Products.class)
        .addBeans(MockBean.builder().types(Products.class).create(ctx -> products).build())
        .build();
}

In this way, WunderBar produces the service proxy, and Weld can inject it into your system under test. For a complete example, take a look at ProductsResolverWeldIT.

SYSTEM Level Tests

To go one step further than integration tests, you can use the test level SYSTEM, maybe by renaming your test class suffix from IT to ST. This means that you actually deploy your service to a full environment, often called 'dev stage'. Then your service needs to call a running instance of the target systems' API. WunderBar provides the wunderbar-mock-server war artifact that you can deploy so your system under test service can reach it and configure your service to do so; no code changes needed. Configure the @Service#endpoint to the address of this mock service. If you call a given on the stub that’s injected into your test, WunderBar prepares this

You can use system-level tests to test a real system, as long as you only test with the data that exists in that service, as calling given will try

Full Documentation

The full documentation is in the JavaDoc, mainly in the @WunderBarApiConsumer annotation, the Level enum and the WunderbarExpectationBuilder for the API consumer (client) side and in the @WunderBarApiProvider annotation and the WunderBarTestFinder for the API provider (server) side.

The demo module contains two example projects: order consumes an API that the product service provides. Both in REST and GraphQL and on all test levels.

If you have further questions, don’t hesitate to ask questions on Stack Overflow tagged with wunderbar. Contributions are also very welcome, of course: start discussions, open issues, add comments, share it online or offline, and if you like it, give it a star on GitHub, please 😁

---

1. The terms "Integration Test", "System Test", and "Acceptance Test" are used in other contexts with slightly different meaning. This is definitively confusing, but introducing new terms or even numbers would make it even harder to understand. So this is the lesser of two evils.

2. We currently don’t see the necessity to support other content types, open an issue if you do need xml or whatever.

3. JUnit invokes the standard JUnit @Before/AfterEach methods only once for every test method, not for every test in a DynamicNode. WunderBar also calls methods annotated as @BeforeDynamicTest / @AfterDynamicTest; the difference is that, in some cases, there can be several subsequent interactions within one dynamic test, so methods with Before/AfterDynamicTest work on Lists.

4. You actually could create the data in a setup method, manipulate the expected response accordingly, and rely on your service being idempotent, so the real call will return the same data, but this is not only more work but also contra-intuitive. There’s a better way.

5. They used to say that the username was a secret, too, but when you use good passwords (i.e. really random and really long), this is not necessary anymore, but it makes life so much easier to see the username.