Reproducing a failing test case quickly

Recipes and recipe hashes

Here's a tricky bit of implementation:

class Tiers<Element extends Comparable<Element>> {
    final int worstTier;

    final List<Element> ringBuffer;

    public Tiers(int worstTier) {
        this.worstTier = worstTier;
        ringBuffer = new ArrayList<>(worstTier) {
            private int ringOffset = 0;

            @Override
            public Element get(int index) {
                return super.get(size() < worstTier ? index
                                                    : (ringOffset + index) %
                                                      worstTier);
            }

            @Override
            public void add(int index, Element element) {
                if (size() < worstTier) {
                    super.add(index, element);
                } else {
                    super.set((ringOffset + index) % worstTier, element);
                    ringOffset = (1 + ringOffset) % worstTier;
                }
            }
        };
    }

    void add(Element element) {
        final int index = Collections.binarySearch(ringBuffer, element);

        if (0 > index) {
            ringBuffer.add(-(index + 1), element);
        } else {
            ringBuffer.add(index, element);
        }
    }

    Optional<Element> at(int tier) {
        return 0 < tier && tier <= ringBuffer.size()
               ? Optional.of(ringBuffer.get(tier - 1))
               :
               Optional.empty();
    }
}

The purpose of Tiers is to take a series of elements, and arrange the elements by tier, where tier 1 is the highest element, tier 2 is the next highest and so on down to a fixed worst tier. Elements that don't make the grade (or are surpassed later) are summarily ejected.

Testing this is quite involved and allows a demonstration of a realistic property-based test. There is a lot of code to follow, but the gist of it is to start with a list of query values that we expect the tiers instance to end up with, then present them in a feed sequence to the tiers instance, surrounding each query value with a run of background values that are constructed to be less than all of the query values.

This approach explores the full range of possibilities - we may have duplicates in the query values (so presumably they should occupy adjacent tiers). Query values may be interspersed with empty lists, in which case they can start or end the feed sequence, or come in adjacent clumps. The query values aren't presented in any particular order, nor are the background values. All we can say is that because the query values are the highest by construction, then as long as we set the number of tiers to be the number of query values, then they should all make it through to the final assessment.

So, on with the test:

final Trials<ImmutableList<Integer>> queryValueLists = api()
        .integers(-1000, 1000)
        .immutableLists()
        .filter(list -> !list.isEmpty());


final Trials<Tuple2<ImmutableList<Integer>, ImmutableList<Integer>>>
        testCases =
        queryValueLists.flatMap(queryValues -> {
            final int minimumQueryValue =
                    queryValues.stream().min(Integer::compareTo).get();

            // A background is a (possibly empty) run of values that are
            // all less than the query values.
            final Trials<ImmutableList<Integer>> backgrounds = api()
                    .integers(Integer.MIN_VALUE, minimumQueryValue - 1)
                    .immutableLists();

            // A section is either a query value in a singleton list, or
            // a background.
            final List<Trials<ImmutableList<Integer>>> sectionTrials =
                    queryValues
                            .stream()
                            .flatMap(queryValue ->
                                             Stream.of(api().only(
                                                               ImmutableList.of(
                                                                       queryValue)),
                                                       backgrounds))
                            .collect(Collectors.toList());

            sectionTrials.add(0, backgrounds);

            // Glue the trials together and flatten the sections they
            // yield into a single feed sequence per trial.
            final Trials<ImmutableList<Integer>> feedSequences =
                    api().immutableLists(sectionTrials).map(sections -> {
                        final ImmutableList.Builder<Integer> builder =
                                ImmutableList.builder();
                        sections.forEach(builder::addAll);
                        return builder.build();
                    });
            return feedSequences.map(feedSequence -> Tuple.tuple(queryValues,
                                                                 feedSequence));
        });


testCases.withLimit(40).supplyTo(testCase -> {
    final ImmutableList<Integer> queryValues = testCase._1();
    final ImmutableList<Integer> feedSequence = testCase._2();

    final int worstTier = queryValues.size();

    final Tiers<Integer> ranking = new Tiers<>(worstTier);

    feedSequence.forEach(ranking::add);

    final ImmutableList.Builder<Integer> builder =
            ImmutableList.<Integer>builder();

    int tier = worstTier;

    do {
        final Integer ranked = ranking.at(tier).get();

        builder.add(ranked);
    } while (1 < tier--);

    final ImmutableList<Integer> arrangedByRank = builder.build();

    assertThat(arrangedByRank, equalTo(queryValues));
});

By now, we won't be surprised to find that it doesn't work:

Trial exception with underlying cause:
java.lang.AssertionError: 
Expected: <[0]>
     but: was <[-1]>
Case:
[[0],[0, -1]]
Reproduce via Java property:
trials.recipeHash=d8d11a95f6c6f5e408a0898868cb289e
Reproduce via Java property:
trials.recipe="[{\"ChoiceOf\":{\"index\":1}},{\"FactoryInputOf\":{\"input\":0}},{\"ChoiceOf\":{\"index\":0}},{\"ChoiceOf\":{\"index\":0}},{\"ChoiceOf\":{\"index\":1}},{\"FactoryInputOf\":{\"input\":-1}},{\"ChoiceOf\":{\"index\":0}}]"