In this article, I’m showing you
- What’s the difference between white box and black testing
- What are the benefits of the latter
- How you can implement it in your Spring Boot application
- How to configure the OpenAPI generator to simplify code and reduce duplications
You can find the code examples in this repository.
Domain
We’re developing a restaurant automatization system. There are two domain classes. Fridge
and Product
. A fridge can have many products, whilst a product belongs to a single fridge. Look at the classes declaration below.
@Entity
@Table(name = "fridge")
public class Fridge {
@Id
@GeneratedValue(strategy = IDENTITY)
private Long id;
private String name;
@OneToMany(fetch = LAZY, mappedBy = "fridge")
private List<Product> products = new ArrayList<>();
}
@Entity
@Table(name = "product")
public class Product {
@Id
@GeneratedValue(strategy = IDENTITY)
private Long id;
@Enumerated(STRING)
private Type type;
private int quantity;
@ManyToOne(fetch = LAZY)
@JoinColumn(name = "fridge_id")
private Fridge fridge;
public enum Type {
POTATO, ONION, CARROT
}
}
I'm using Spring Data JPA as a persistence framework. Therefore, those classes are Hibernate entities.
White box testing
This type of testing makes an assumption that we know some implementation details and may interact them. We have 4 REST API endpoints in the system:
- Create a new
Fridge
. - Add a new
Product
. - Change the
Product
quantity. - Remove the
Product
from theFridge
.
Suppose we want to test the one that changes the Product
quantity. Take a look at the example of the test below.
@SpringBootTest(webEnvironment = RANDOM_PORT)
class ProductControllerWhiteBoxTest extends IntegrationSuite {
@Autowired
private FridgeRepository fridgeRepository;
@Autowired
private ProductRepository productRepository;
@Autowired
private TestRestTemplate rest;
@BeforeEach
void beforeEach() {
productRepository.deleteAllInBatch();
fridgeRepository.deleteAllInBatch();
}
@Test
void shouldUpdateProductQuantity() {
final var fridge = fridgeRepository.save(Fridge.newFridge("someFridge"));
final var productId = productRepository.save(Product.newProduct(POTATO, 10, fridge)).getId();
assertDoesNotThrow(() -> rest.put("/api/product/{productId}?newQuantity={newQuantity}", null, Map.of(
"productId", productId,
"newQuantity", 20
)));
final var product = productRepository.findById(productId).orElseThrow();
assertEquals(20, product.getQuantity());
}
}
Let’s examine this piece of code step by step.
We inject repositories to manipulate rows in the database. Then the TestRestTemplate
comes into play. This bean is used to send HTTP requests. Then you can see that the @BeforeEach
callback deletes all rows from the database. So, each test runs deterministically. And finally, here is the test itself:
- We create a new
Fridge
. - Then we create a new
Product
with a quantity of10
that belongs to the newly createdFridge
. - Afterwards, we invoke the REST endpoint to increase the
Product
quantity from10
to20
. - Eventually we select the same
Product
from the database and check that the quantity has been increased.
The test works fine. Anyway, there are nuances that should be taken into account:
- Though the test verifies the entire system behavior (aka functional test) there is a coupling on implementation details (i.e. the database).
- What the test validates is not the actual use case. If somebody wants to interact with our service, they won’t be able to insert and update rows in the database directly.
As a matter of fact, if we want to test the system from the user perspective, we can only use the public API that the service exposes.
Black box testing
This type of testing means loose coupling on the system's implementation details. Therefore, we can only depend on the public API (i.e. REST API).
Check out the previous white box test example. How can we refactor it into the black box kind? Look at the @BeforeEach
implementation below.
@BeforeEach
void beforeEach() {
productRepository.deleteAllInBatch();
fridgeRepository.deleteAllInBatch();
}
A black box test should not interact with the persistence provider directly. Meaning that there should be a separate REST endpoint clearing all data. Look at the code snippet below.
@RestController
@RequiredArgsConstructor
@Profile("qa")
public class QAController {
private final FridgeRepository fridgeRepository;
private final ProductRepository productRepository;
@DeleteMapping("/api/clearData")
@Transactional
public void clearData() {
productRepository.deleteAllInBatch();
fridgeRepository.deleteAllInBatch();
}
}
Now we have a particular controller that encapsulates all the clearing data logic. If the test depends only on this endpoint, then we can safely put changes and refactor the method as the application grows. And our black box tests won’t break. The @Profile(“qa”)
annotation is crucial. We don’t want to expose an endpoint that can delete all user data in production or even development environment. So, we register this endpoint, if qa
profile is active. We’ll use it only in tests.
The
qa
abbreviation stands for the quality assurance.
And now we should refactor the test method itself. Have a look at its implementation below again.
@Test
void shouldUpdateProductQuantity() {
final var fridge = fridgeRepository.save(Fridge.newFridge("someFridge"));
final var productId = productRepository.save(Product.newProduct(POTATO, 10, fridge)).getId();
assertDoesNotThrow(() -> rest.put("/api/product/{productId}?newQuantity={newQuantity}", null, Map.of(
"productId", productId,
"newQuantity", 20
)));
final var product = productRepository.findById(productId).orElseThrow();
assertEquals(20, product.getQuantity());
}
There are 3 operations that should be replaced with direct REST API invocations. These are:
- Creating new
Fridge
. - Creating new
Product
. - Checking the the
Product
quantity has been increased.
Look at the whole black box test example below.
@SpringBootTest(webEnvironment = RANDOM_PORT)
@ActiveProfiles("qa")
class ProductControllerBlackBoxTest extends IntegrationSuite {
@Autowired
private TestRestTemplate rest;
@BeforeEach
void beforeEach() {
rest.delete("/api/qa/clearData");
}
@Test
void shouldUpdateProductQuantity() {
// create new Fridge
final var fridgeResponse =
rest.postForEntity("/api/fridge?name={fridgeName}", null, FridgeResponse.class, Map.of(
"fridgeName", "someFridge"
));
assertTrue(fridgeResponse.getStatusCode().is2xxSuccessful(), "Error during creating new Fridge: " + fridgeResponse.getStatusCode());
// create new Product
final var productResponse = rest.postForEntity(
"/api/product/fridge/{fridgeId}",
new ProductCreateRequest(
POTATO,
10
),
ProductResponse.class,
Map.of(
"fridgeId", fridgeResponse.getBody().id()
)
);
assertTrue(productResponse.getStatusCode().is2xxSuccessful(), "Error during creating new Product: " + productResponse.getStatusCode());
// call the API that should be tested
assertDoesNotThrow(
() -> rest.put("/api/product/{productId}?newQuantity={newQuantity}",
null,
Map.of(
"productId", productResponse.getBody().id(),
"newQuantity", 20
))
);
// get the updated Product by id
final var updatedProductResponse = rest.getForEntity(
"/api/product/{productId}",
ProductResponse.class,
Map.of(
"productId", productResponse.getBody().id()
)
);
assertTrue(
updatedProductResponse.getStatusCode().is2xxSuccessful(),
"Error during retrieving Product by id: " + updatedProductResponse.getStatusCode()
);
// check that the quantity has been changed
assertEquals(20, updatedProductResponse.getBody().quantity());
}
}
The benefits of black box testing in comparison to white box testing are:
- The test checks the path that a user shall do to retrieve the expected result. Therefore, the verification behavior becomes more robust.
- Black box tests are highly stable against refactoring. As long as the API contract remains the same, the test should not break.
- If you accidentally break the backward compatibility (e.g. adding a new mandatory parameter to an existing REST endpoint), the black box test will fail and you'll determine the issue way before the artefact is being deployed to any environment.
However, there is a slight problem with the code that you have probably noticed. The test is rather cumbersome. It’s hard to read and maintain. If I didn’t put in the explanatory comments, you would probably spend too much time figuring out what’s going on. Besides, the same endpoints might be called for different scenarios, which can lead to code duplication.
Luckily there is solution.
OpenAPI and code generation
Spring Boot comes with a brilliant OpenAPI support. All you have to do is to add two dependencies. Look at the Gradle configuration below.
implementation 'org.springframework.boot:spring-boot-starter-actuator'
implementation 'org.springdoc:springdoc-openapi-ui:1.6.12'
After adding these dependencies, the OpenAPI specification is available by GET /v3/api-docs
endpoint.
The SpringDoc library comes with lots of annotations to tune your REST API specification precisely. Anyway, that's out of context of this article.
If we have the OpenAPI specification, it means that we can generate Java classes to call the endpoints in a type-safe manner. What’s even more exciting is that we can apply those generated classes in our black box tests!
Firstly, let's define the requirements for the upcoming OpenAPI Java client:
- The generated classes should be put into
.gitignore
. Otherwise, if you have Checkstyle, PMD, or SonarQube in your project, then generated classes can violate some rules. Besides, if you don't put them into.gitignore
, then each pull request might become huge due to the fact that even a slightest fix can lead to lots of changes in the generated classes. - Each pull request build should guarantee that generated classes are always up to date with the actual OpenAPI specification.
How can we get the OpenAPI specification itself during the build phase? The easiest way is to write a separate test that creates the web part of the Spring context, invokes the /v3/api-docs
endpoint, and put the retrieved specification into build
folder (if you are Maven user, then it will be target
folder). Take a look at the code example below.
@SpringBootTest(
webEnvironment = RANDOM_PORT
)
@AutoConfigureTestDatabase
@ActiveProfiles("qa")
public class OpenAPITest {
@Autowired
private TestRestTemplate rest;
@Test
@SneakyThrows
void generateOpenApiSpec() {
final var response = rest.getForEntity("/v3/api-docs", String.class);
assertTrue(response.getStatusCode().is2xxSuccessful(), "Unexpected status code: " + response.getStatusCode());
// the specification will be written to 'build/classes/test/open-api.json'
Files.writeString(
Path.of(getClass().getResource("/").getPath(), "open-api.json"),
response.getBody()
);
}
}
The
@AutoConfigureTestDatabase
configures the in-memory database (e.g. H2), if you have one in the classpath. Since the database provider does not affect the result OpenAPI specification, we can make the test run a bit faster by not using Testcontainers.
Now we have the result specification. How can we generate the Java classes based on it? We have another Gradle plugin for that. Take a look at the build.gradle
configuration below.
plugins {
...
id "org.openapi.generator" version "6.2.0"
}
...
openApiGenerate {
inputSpec = "$buildDir/classes/java/test/open-api.json".toString()
outputDir = "$rootDir/open-api-java-client".toString()
apiPackage = "com.example.demo.generated"
invokerPackage = "com.example.demo.generated"
modelPackage = "com.example.demo.generated"
configOptions = [
dateLibrary : "java8",
openApiNullable: "false",
]
generatorName = 'java'
groupId = "com.example.demo"
globalProperties = [
modelDocs: "false"
]
additionalProperties = [
hideGenerationTimestamp: true
]
}
In this article, I'm showing you how to configure the corresponding Gradle plugin. Anyway, there is Maven plugin as well and the approach won't be different much.
There is an important detail about OpenAPI generator plugin. It creates the whole Gradle/Maven/SBT project (containing build.gradle
, pom.xml
, and build.sbt
files) but not just Java classes. So, we set the theoutputDir
property as $rootDir/open-api-java-client
. Therefore, the generated Java classes go into the Gradle subproject.
We should also mark the open-api-java-client
directory as a subproject in the settings.gradle
. Look at the code snippet below.
rootProject.name = 'demo'
include 'open-api-java-client'
All you have to do to generate OpenAPI Java client is to run these Gradle commands:
gradle test --tests "com.example.demo.controller.OpenAPITest.generateOpenApiSpec"
gradle openApiGenerate
Applying the Java client
Now let’s try our brand new Java client in action. We’ll create a separate @TestComponent
for convenience. Look at the code snippet below.
@TestComponent
public class TestRestController {
@Autowired
private Environment environment;
public FridgeControllerApi fridgeController() {
return new FridgeControllerApi(newApiClient());
}
public ProductControllerApi productController() {
return new ProductControllerApi(newApiClient());
}
public QaControllerApi qaController() {
return new QaControllerApi(newApiClient());
}
private ApiClient newApiClient() {
final var apiClient = new ApiClient();
apiClient.setBasePath("http://localhost:" + environment.getProperty("local.server.port", Integer.class));
return apiClient;
}
}
Finally, we can refactor our black box test. Look at the final version below.
@SpringBootTest(webEnvironment = RANDOM_PORT)
@ActiveProfiles("qa")
@Import(TestRestControllers.class)
class ProductControllerBlackBoxGeneratedClientTest extends IntegrationSuite {
@Autowired
private TestRestControllers rest;
@BeforeEach
@SneakyThrows
void beforeEach() {
rest.qaController().clearData();
}
@Test
void shouldUpdateProductQuantity() {
final var fridgeResponse = assertDoesNotThrow(
() -> rest.fridgeController()
.createNewFridge("someFridge")
);
final var productResponse = assertDoesNotThrow(
() -> rest.productController()
.createNewProduct(
fridgeResponse.getId(),
new ProductCreateRequest()
.quantity(10)
.type(POTATO)
)
);
assertDoesNotThrow(
() -> rest.productController()
.updateProductQuantity(productResponse.getId(), 20)
);
final var updatedProduct = assertDoesNotThrow(
() -> rest.productController()
.getProductById(productResponse.getId())
);
assertEquals(20, updatedProduct.getQuantity());
}
}
As you can see, the test is much more declarative. Moreover, the API contract become statically typed and the parameters validation proceeds during compile time!
The .gitignore
caveat and the separate test source
I told that we should put the generated classes to the .gitignore
. However, if you mark the open-api-java-client/src
directory as the unindexed by Git, then you suddenly realize that your tests do not compile in CI environment. The reason is that the process of generation the OpenAPI specification (i.e. open-api.json
file) is an individual test as well. And even if you tell the Gradle to run a single test directly, it will compile everything in the src/test
directory. In the end, tests don’t compile successfully.
Thankfully, the issue can be solved easily. Gradle provides source sets. It's a logical group that splits the code into separate modules that you can compile independently.
Firstly, let's add the gradle-testsets plugin and define a separate test source that'll contain the OpenAPITest
file. It's the one that generates the open-api.json
specification. Take a look at the code example below.
plugins {
...
id "org.unbroken-dome.test-sets" version "4.0.0"
}
...
testSets {
openApiGenerator
}
tasks.withType(Test) {
group = 'verification'
useJUnitPlatform()
testLogging {
showExceptions true
showStandardStreams = false
showCauses true
showStackTraces true
exceptionFormat "full"
events("skipped", "failed", "passed")
}
}
openApiGenerator.outputs.upToDateWhen { false }
tasks.named('openApiGenerate') {
dependsOn 'openApiGenerator'
}
The testSets
block declares a new source set called openApiGenerator
. Meaning that Gradle treats the src/openApiGenerator
directory like another test source.
The tasks.withType(Test)
declaration is also important. We need to tell Gradle that every task of Test
type (i.e. the test
task itself and the openApiGenerator
as well) should run with JUnit.
I put the upToDateWhen
option for convenience. It means that the test that generates open-api.json
file will be always run on demand and never cached.
And the last block defines that before generating the OpenAPI Java client we should update the specification in advance.
Now we just need to move the OpenAPITest
to the src/openApiGenerator
directory and also make a slight change to the openApiGenerate
task in build.gradle
. Look at the code snippet below.
openApiGenerate {
// 'test' directory should be replaced with 'openApiGenerator'
inputSpec = "$buildDir/classes/java/openApiGenerator/open-api.json".toString()
....
}
Finally, you can build the entire project with these two commands.
gradle openApiGenerate
gradle build
Conclusion
The black box testing is a crucial part of the application development process. Try it and you’ll notice that the test scenarios become much more representative. Besides, black box test are also great documentation for the API. You can even apply Spring REST Docs and generate a nice manual that’ll be useful both for the API users and QA engineers.
If you have any questions or suggestion, leave your comments down below. Thanks for reading!
Top comments (0)