DEV Community

Tomer Figenblat
Tomer Figenblat

Posted on

Property-Based Matrix Testing in Java

Property-Based testing with a dynamic matrix

A while back, I had to roll out a dynamic matrix to fully test some feature I was working on at the time.

I can't even remember what was the case or the project, but ever since then, I've been using the same technique to accomplish matrix testing.
It's not very pretty, or clean, but it gets the job done. 😎

I've been using JUnit's ParameterizedTest annotation to incorporate property-based testing (PBT), and a MethodSource annotation pointing to a method that spits out a stream representing the matrix.
It's pretty straightforward, but the more parameter types I have in my matrix, the harder it is to read or write the method supplying them. 😵

Lately, I discovered JUnit Pioneer, which is a JUnit 5 Extension Pack, amongst all the goodies introduced, you can find the CartesianProductTest annotation.
Incorporated with a couple of a CartesianEnumSource annotations and the matrix implementation becomes much more simple and elegant. 😃

Let's have a look.

Creating a dynamic property-based matrix from the following Enums should spit out 6 tests:

  enum Direction {
    INCOMING,
    OUTGOING
  }

  enum Status {
    SUCCESS,
    FAILURE,
    WAITING
  }
Enter fullscreen mode Exit fullscreen mode

Implementing a matrix from these Enums with JUnit's ParameterizedTest and a MethodSource:

  @ParameterizedTest
  @MethodSource("getArguments")
  void using_junit_parameterized_test_with_method_source(
      final Direction direction, final Status status) {
    assertTrue(true);
  }

  static Stream<Arguments> getArguments() {
    return Stream.of(Direction.values())
        .flatMap(d -> Stream.of(Status.values()).map(s -> arguments(d, s)));
  }
Enter fullscreen mode Exit fullscreen mode

As you can see, adding members to the existing Enums will dynamically increase the matrix and therefore the number of tests performed.

But, adding a third element to the matrix, and the getArguments method, will start losing its readability.
Nevertheless, that gets the job done, and I've been using this technique throughout my projects for a long time now.

Now, let's accomplish the same with JUnit Pioneer's CartesianProductTest and a couple of CartesianEnumSources:

  @CartesianProductTest
  @CartesianEnumSource(Direction.class)
  @CartesianEnumSource(Status.class)
  void using_junit_pioneer_cartesian_product_test_with_enum_source(
      final Direction direction, final Status status) {
    assertTrue(true);
  }
Enter fullscreen mode Exit fullscreen mode

This will spit out the same matrix, only now, adding a third element is quite simple, just add another annotation, you can find other types of sources beside Enums, in JUnit Pioneer's Documentation.

As demonstrated in this repository, executing both test cases in the same test class, using JUnit's platform, will print out:

[INFO] '-- JUnit Jupiter [OK]
[INFO]   '-- Property Based Matrix Test [OK]
[INFO]     +-- using junit pioneer cartesian product test with enum source (Direction, Status) [OK]
[INFO]     | +-- [1] INCOMING, SUCCESS [OK]
[INFO]     | +-- [2] INCOMING, FAILURE [OK]
[INFO]     | +-- [3] INCOMING, WAITING [OK]
[INFO]     | +-- [4] OUTGOING, SUCCESS [OK]
[INFO]     | +-- [5] OUTGOING, FAILURE [OK]
[INFO]     | '-- [6] OUTGOING, WAITING [OK]
[INFO]     '-- using junit parameterized test with method source (Direction, Status) [OK]
[INFO]       +-- [1] INCOMING, SUCCESS [OK]
[INFO]       +-- [2] INCOMING, FAILURE [OK]
[INFO]       +-- [3] INCOMING, WAITING [OK]
[INFO]       +-- [4] OUTGOING, SUCCESS [OK]
[INFO]       +-- [5] OUTGOING, FAILURE [OK]
[INFO]       '-- [6] OUTGOING, WAITING [OK]
Enter fullscreen mode Exit fullscreen mode

You can check out the code for this tutorial in Github.

👋 See you in the next tutorial 👋

Discussion (0)