Testcontainers with MySQL and Redis with an Spring Boot Kotlin Application

Microservices architecture gives us some facilities, such as independently deploy, services with a unique responsibility, working with different languages, and so on. However, it also brings us some new complexities, such as duplicated code, observability tools, distributed caches tools, and logs tools. When we think about all these resources working together, we can think "How can we test these resources?". For instance, if our service uses Redis for cache, and MySQL to save and retrieve data, how can we do integration tests? Doing these tests by yourself can be difficult, but we have some manners to create that without much effort. One option is using TestContainers. To show you the tool, I will test a Spring Boot Kotlin Application as example.

TestContainers

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.

Testcontainers make the following kinds of tests easier:

• Data access layer integration tests: use a containerized instance of a MySQL, PostgreSQL or Oracle database to test your data access layer code for complete compatibility, but without requiring complex setup on developers' machines and safe in the knowledge that your tests will always start with a known DB state. Any other database type that can be containerized can also be used.

• Application integration tests: for running your application in a short-lived test mode with dependencies, such as databases, message queues or web servers.

• UI/Acceptance tests: use containerized web browsers, compatible with Selenium, for conducting automated UI tests. Each test can get a fresh instance of the browser, with no browser state, plugin variations or automated browser upgrades to worry about. And you get a video recording of each test session, or just each session where tests failed.

• Much more! Check out the various contributed modules or create your own custom container classes using GenericContainer as a base.

Let's Code

The application that will be used has a Car domain. We can perform operations of inserting new cars, searching for available cars, and changing existing cars. The system has a controller, a service, and a repository. An overview of the architecture follows.

So, when we do a POST request to an application, the data is saved on MySQL DB.

When we do a GET request, the application retrieves the data that is saved on DB and adds the information to Redis cache.

Redis is an open-source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker. Redis' speed makes it ideal for caching database queries, complex computations, API calls, and session state.

Data in Redis

We need to guarantee that the cache was saved in Redis. For this, we can use a tool called RedisInsight to see the data. RedisInsight provides an intuitive Redis admin GUI and helps optimize your use of Redis in your applications. It supports Redis Open Source, Redis Stack, Redis Enterprise Software, Redis Enterprise Cloud, and Amazon ElastiCache. RedisInsight now incorporates a completely new tech stack based on the popular Electron and Elastic UI frameworks. And it runs cross-platform, supported on Linux, Windows, and MacOS.

As we can see, the cache's data is saved like a hash. And if we don't do POST and PUT operations to the application, this data always will be retrieved.

Testing the application

Now that we know how the application works, a controller test will be performed, where we will use MockMVC. MockMvc simulates a real request to the application, doing exactly what happens when the client calls an endpoint. This implies that at the time of running the tests, we need to have all external resources working. It is common for projects to use a real test-specific database instance to perform integration tests, however, as we saw earlier, we have a more elegant and modern way to do them, using TestContainers. As the cached service method and the repository will be called in the controller flow, it will be necessary to upload two containers, one for MySQL and the other for Redis. Let's start developing our test.

The first thing we need to do is add the TestContainers dependencies to the application's pom.xml.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.testcontainers</groupId>
      <artifactId>testcontainers-bom</artifactId>
      <version>${testcontainers.version}</version>
        <type>pom</type>
        <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>mysql</artifactId>
    <scope>test</scope>
</dependency>

We need the TestContainers junit-jupiter dependency because it provides some resources for more efficient testing. As we are using MySQL as a database, it's necessary a dependency that provides resources to create a MySQL container for tests in a simplified way.

We need to create the configuration class of the containers that will be created at test time. It will be called DatabaseContainerConfiguration and will have the following content

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
abstract class DatabaseContainerConfiguration {

    companion object {
        @Container
        private val mysqlContainer = MySQLContainer<Nothing>("mysql:latest").apply {
            withDatabaseName("testdb")
            withUsername("joao")
            withPassword("12345")
        }

        @Container
        private val redisContainer = GenericContainer<Nothing>("redis:latest").apply {
            withExposedPorts(6379)
        }

        @JvmStatic
        @DynamicPropertySource
        fun properties(registry: DynamicPropertyRegistry) {
            registry.add("spring.datasource.url", mysqlContainer::getJdbcUrl)
            registry.add("spring.datasource.password", mysqlContainer::getPassword)
            registry.add("spring.datasource.username", mysqlContainer::getUsername)

            registry.add("spring.redis.host", redisContainer::getContainerIpAddress)
            registry.add("spring.redis.port", redisContainer::getFirstMappedPort)
        }
    }
}

Now let's break down the meaning of each of the above features:

@Container: is used in conjunction with the @Testcontainers annotation to mark containers that should be managed by the Testcontainers extension.

We'll see about the @TestContainers annotation yet

MySQLContainer<Nothing>(...): Class that supports the creation of a MySQL container for testing. We passed a Nothing in the generics of MySQLContainer because of the class signature - public class MySQLContainer<SELF extends MySQLContainer<SELF>> extends JdbcDatabaseContainer<SELF> - as we can see, there is a recursive generic type, which with Java, we can ignore and just instantiate the class without generics. Using Kotlin, we don't have this option, but we have two ways to handle it.

1. Define a subclass of MySQLContainer and pass the child class to generics
2. Use Nothing, which by Kotlin doc's is:

Nothing has no instances. You can use Nothing to represent "a value that never exists": for example, if a function has the return type of Nothing, it means that it never returns.

In the MySQLContainer constructor, the version that we want to use from the MySQL image is informed.

withDatabaseName: Method of the MySQLContainer class that receives the name of the database as a parameter. This database will be created at runtime after the test container goes up.

withUsername: Method of the MySQLContainer class that receives the database username as a parameter.

withPassword: Method of the MySQLContainer class that receives the password from the database as a parameter.

GenericContainer<Nothing>(): As the name implies, it is a class that allows you to create a container in a generic way, which means that the methods contained in them can be reused for various resources, such as Redis, NoSQL Databases, among others. We are using it because Redis does not have a specific class like MySQL. As we are seeing in the code, to create a generic container, we just need to insert the image of the resource that we want to instantiate as a parameter.

withExposedPorts: Method of the GenericContainer class that receives the port that we will make available from the test container. This port will be used at runtime.

The properties function is used to dynamically add Redis and MySQL properties to the application. Before TestContainers uploads the two containers, the application will not be aware of them, so how would we carry out the integration tests? Using @DynamicPropertySource Kotlin understands that these properties will be dynamically set at runtime, so when uploading the MySQL and Redis container, the information that the application needs to know to reach them will be included.

Now we have everything we need for TestContainers to upload both containers. We just need to use the class where needed. As stated at the beginning of the section, we want to do an API test, where MockMVC will make a request to the controller and thus follow the flow to the database. So let's create this test.

@Testcontainers
@AutoConfigureMockMvc
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class CarControllerTest : DatabaseContainerConfiguration() {

    @Autowired
    private lateinit var mockMvc: MockMvc

    companion object {
        private const val URI = "/cars"
    }

    @Test
    fun `should return 200 code when call cars`() {
        mockMvc.get(URI).andExpect { status { isOk() } }
    }

As mentioned before, we can see that we have an annotation in the class declaration, which is @TestContainers. This annotation makes that, at runtime, the library identifies it and uploads the available containers through the @Container annotation, which is in our DatabaseContainerConfiguration class, which in turn was inherited by the test. This is enough for when we run the tests, the containers are instantiated.

The intention is not to talk about MockMVC, but according to the documentation, it is the main entry point for server-side Spring MVC testing support.

One last detail before running the tests is that it will be necessary to configure the migration. When the application goes up and reaches the containers, it is applied.

#src/main/resources/db/migration/V01__create_car_table_add_data.sql

create table car(
    id bigint not null auto_increment,
    name varchar(50) not null,
    model varchar(50) not null,
    primary key(id)
);

insert into car values('Golf', 'VW');

It's time to run the tests and remember, docker must be active. Running the mvn -B test command will run the tests. We can observe the containers going up at the time of the test execution, using the docker ps command in the terminal or the docker desktop.

In the test execution logs, we can see the moment when the MySQL container rises

22:17:53.206 [main] DEBUG 🐳 [mysql:latest] - Trying to create JDBC connection using com.mysql.cj.jdbc.Driver to jdbc:mysql://localhost:52550/testdb?useSSL=false&allowPublicKeyRetrieval=true with properties: {password=12345, user=joao}
22:17:53.582 [main] INFO 🐳 [mysql:latest] - Container is started (JDBC URL: jdbc:mysql://localhost:52550/testdb)
22:17:53.583 [main] INFO 🐳 [mysql:latest] - Container mysql:latest started in PT23.428294S

Just like Redis

22:17:53.583 [main] DEBUG 🐳 [redis:latest] - Starting container: redis:latest
22:17:53.583 [main] DEBUG 🐳 [redis:latest] - Trying to start container: redis:latest (attempt 1/1)
22:17:53.583 [main] DEBUG 🐳 [redis:latest] - Starting container: redis:latest
22:17:53.583 [main] INFO 🐳 [redis:latest] - Creating container for image: redis:latest

And finally the expected result

Tests Passed: 1 of 1 test - 631 ms

Conclusion

Integration tests are very important for the application, and one of the main tasks is to validate if the application's connection with external resources is being done successfully. There are N ways to do them. We can use database integration tests, in-memory databases, real instances and as presented in the post, TestContainers. TestContainers, with small settings, bring big advantages. Disposable containers, as they are only used at the time of test execution. External resource agnostic, with the GenericContainer class we can do integration testing with any resource we uploaded in docker. An excellent testing tool.

This post is in collaboration with Redis.

Application's repository: https://github.com/joao0212/car-service

References:
https://www.testcontainers.org/
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/test/web/servlet/MockMvc.html