Contract Integration Testing

Recently I wrote an article about implementing E2E testing in CI environment with Testcontainers usage. And today I want to append a small detail. We'll talk about contracts and why is it important to write integration tests for them. The code examples are in Java but you can apply the proposed solution to any programming language.

If you haven't read my piece about E2E tests, I strongly recommend you do it before going further. In this article, there will be lots of proposals and discussions based on the experience of the previous item.

E2E testing pitfalls

You see, E2E testing is incredible. It's the highest quality assurance bar one can put within the software project. At the same time, there are several nuances you should take into account:

1. E2E testing is tough. Writing meaningful and maintainable E2E tests requires significant skills.
2. Running E2E testing automatically is also not that simple. Though I proposed the solution in the previous article it cannot be called trivial.
3. The more microservices you have, the harder it is to write a new E2E test.
4. Each time the software product grows with a new microservice you have to update the environment (e.g. Testcontainers configuration) accordingly to be sure that tests are still valid.
5. If the product consists of too many microservices, it becomes almost impossible to write a solid E2E test for the whole system.

As you can see, E2E tests work until the software project becomes huge and too complicated. So, what shall we do then?

Contract integration testing

Firstly, have a look again at the system diagram below from the previous article.

The whole business scenario can be described in the following steps:

1. A user sends a message by REST API to API-Service.
2. Then API-Service transfers the message to the RabbitMQ cluster.
3. Gain-Service consumes the message, enriches it with additional information from the cache (i.e. Redis), and finally pushes the result message to the RabbitMQ again.

What contracts do we have here? Take a look at the schema below.

I highlighted the contracts with pale green ovals:

1. RabbitUserMsgPush - pushing a user's messages from API-Service to RabbitMQ.
2. RabbitUserMsgConsume - pulling a user's messages from RabbitMQ to Gain-Service.
3. RedisCacheRepo - reading and updating Redis data from the Gain-Service perspective.
4. RabbitGainMsgPush - pushing gained message from Gain-Service to RabbitMQ

Why do we need contracts at all? Though additional layers add extra complexity, contracts also help us to get rid of integration tests in API-Service and Gain-Service. Take a look at the example below.

As we already discussed, UserMsgPush is the contract. There are two implementations: RabbitUserMsgPush and InMemoryUserMsgPush. And now Liskov substitution principle comes into play. If you simplify the definition, replacing one interface implementation with another should not break the software's correctness. In this case, we don't have to involve RabbitMQ in testing API-Service! Because InMemoryUserMsgPush will be sufficient. So, here are the benefits:

1. We can cover API-Service only with unit tests.
2. Integration tests are only encapsulated within the RabbitUserMsgPush scenarios.
3. We can replace one implementation of UserMsgPush with another without touching tests in RabbitUserMsgPush.

Seems that contract testing is the silver bullet. Firstly, we reduced the number of integration tests and simplified the quality scenarios in the services. Secondly, we got rid of E2E tests completely which are tough to write, execute, and maintain. Unfortunately, contract testing also has drawbacks.

Cons of contracts

The unwanted code complexity

Microservices don't usually have much code. They often provide just a single operation. For example, API-Service accepts a message via REST API and pushes it to RabbitMQ. That's it. Therefore, extra layers, interfaces, and abstractions may seem like noise. Why do I have to operate using a facade, if I can just proceed it directly?

Although abstractions are useful, their abundance can make code more complicated than it should be.

Lack of local experiments

The primary advantage of integration testing is that it allows us to perform local experiments before submitting a pull request. For example, suppose you need to adjust some properties of the external facility your service depends on. Those can be:

1. Reducing the size of the database connection pool.
2. Changing the serialization strategy in Kafka producer.
3. Reconditioning an SQL statement to make it more efficient.

If you have integration tests within the service you're working on, you can just tune the required settings and run tests locally to verify the behaviour. But if the service relies on contracts, the task becomes intriguing. You have several approaches:

1. Change the contract in such a way that the properties you're updating are passed as parameters. So, you add additional integration tests in the contract that checks the scenario validity. But in this case, the contract exposes too many details about its implementation. That breaks the whole idea of abstraction and makes replacing one implementation with another more difficult.
2. Encapsulate the desired properties' changes as the integration scenarios within the contract. Here you don't leak implementation details to the contract's user but also you make your tests far more complicated. Because you have to verify different combinations of possible properties' values.
3. Add the required integration tests directly to the service codebase. So, you don't overcomplicate the contract, right? Indeed, but now the service depends directly on the particular contract implementation. We introduced the contracts to remove integration tests from the services completely. And here we go again.

Backward compatibility

As I wrote in the previous article, the main advantage of E2E tests is that they check the backward compatibility problems. Regrettably, contracts do not provide such a feature.

For example, look at the RabbitUserMsgPush contract implementation below.

public class RabbitUserMsgPush implements UserMsgPush {
    private final RabbitTemplate rabbitTemplate;
    private final ObjectMapper objectMapper;
    private final String targetQueue;    

    @Override 
    @SneakyThrows
    public void pushMessage(Object payload) {
        rabbitTemplate.send(
            targetQueue,
            new Message(
                objectMapper.writeValueAsString(payload)
                    .getBytes(UTF_8)
            )
        );
    }
}

In this case, the passed payload is serialized to JSON. Suppose that we decided to apply binary protocol instead (e.g. Protobuf). Take a look at the modified RabbitUserMsgPush below.

public class RabbitUserMsgPush implements UserMsgPush {
    private final RabbitTemplate rabbitTemplate;
    private final ProtobufSerializer serializer;
    private final String targetQueue;    

    @Override 
    public void pushMessage(Object payload) {
        rabbitTemplate.send(
            targetQueue,
            new Message(
                serializer.toBytesArray(payload)
            )
        );
    }
}

That is convenient. Because we can change the contract's implementations without modifying business logic at all. Nothing can go wrong, isn't it? Well, each contract defines a producer and a consumer. What if we update the producer's contract before the consumer's one? It means the consumer won't be able to deserialize the message and fail.

As you can see, we cannot break backward compatibility easily in the distributed environment. We may not be aware of previous contract versions that other clients are still using. E2E testing track those violations. But due to the contract's implementations isolation, the latter approach cannot provide such benefit.

Have a look at the proper way of applying such changes below.

public class RabbitJSONUserMsgPush implements UserMsgPush {
    private final RabbitTemplate rabbitTemplate;
    private final ObjectMapper objectMapper;
    private final String targetJsonQueue;
    private final UserMsgPush next;

    @Override 
    @SneakyThrows
    public void pushMessage(Object payload) {
        rabbitTemplate.send(
            targetJsonQueue,
            new Message(
                objectMapper.writeValueAsString(payload)
                    .getBytes(UTF_8)
            )
        );
        next.pushMessage(payload);
    }
}

public class RabbitUserMsgPush implements UserMsgPush {
    private final RabbitTemplate rabbitTemplate;
    private final ProtobufSerializer protobufSerializer;
    private final String targetProtobufQueue;
    private final UserMshPush next;    

    @Override 
    public void pushMessage(Object payload) {
        rabbitTemplate.send(
            targetProtobufQueue,
            new Message(
                serializer.toBytesArray(payload)
            )
        );
        next.pushMessage(payload);
    }
}

public class NoOpUserMsgPush implements UserMsgPush {
    @Override 
    public void pushMessage(Object payload) {
        // no operations
    }
}

Firstly, we push the payload into two queues with varying serialization strategies. The consumer won't stuck, because it continues to read the previous queue until its contract is updated. Secondly, we apply Chain-of-Responsibility design pattern to distinguish different serialization strategies. In this case, we can test them separately.

I created NoOpUserMsgPush for convenience. It's the last chain element that does not operate. So, we don't have to check whether the next element is present in each implementation.

Keeping changes backwards compatible in contracts is possible but it requires additional effort. You can also put counter metrics describing the number of particular operation invocations (i.e. pushing JSON and Protobuf messages) and display them into Grafana. That'll help you to notice when all consumers are updated and producers can safely eliminate the redundant functionality.

A single language for microservices

A contract is nothing but a regular piece of code. So, you cannot apply the contract written in Java in the microservices created with Golang. If you need to use varying technologies and languages for the different parts of your system, then implementing statically typed contracts might become a challenge. There are some solutions on the market. For example, OpenAPI and AsyncAPI specifications can generate code snippets based on the provided configurations. Anyway, the approaches have limitations.

Conclusion

Contracts are powerful. In some cases, they can make your code easier to test. On the other hand, they also bring maintainability cases that should be considered. It's up to you to decide whether to use contracts or not. But in my opinion, they cannot replace E2E testing completely.

That's all I wanted to tell you about contract integration testing. If you have any questions or suggestions, please, leave your comments down below. Thanks for reading!