Best practices for writing contract tests

Contract testing is a broad topic that includes such areas as:

• Building reliable pipeline
• Framework concepts and key functionalities knowledge
• Writing consumer contract tests in a scalable design and covering it on the provider side with the easily maintainable verifications

The first two points usually appear to be on the shoulders of a smaller group of people (Devops, platform engineers and team leads) but the latter one affects every developer who wants to write contract tests following the set of design patterns in his project. Here let's talk about this point.

Maturity level:

• You know contract testing
• You know consumer-driven contract testing
• You know API
• You care about the project and standards in your company
• You already have written your first contract
• You know how to run the provider tests and verify the consumer's contract
• You focus to get as much as possible from contract tests

Writing your best contracts

1. Focus on the actual API rather than on the code

Before writing the contract test I always recommend calling the actual endpoint to see the request and response then the goal is simple - reproduce the contract in the test. You can use any tool like Postman, curl, etc.

Call the endpoint in Postman before writing the test

Why?

• HTTP contract test is about HTTP semantics and you can't always see or can't find the required semantics that will be taken into the actual request effect in one place. For example:
  • Authorization is not visible very often
  • The base path defined in some configuration

2. Write the contract in a way it reflects the actual integration that you expect

This point comes along with the first one. Trusting a message from your colleague or your intuition and even openapi (swagger) is not enough. You should see the actual integration in front of you. If the actual endpoint doesn't exist then align the contract with your provider before the merge.

3. Be flexible in your expectations

I often find that only exact values are being used in the contract though it's not expected that API returns or expects always a constant value - it's the only case when you should use the exact value.

In all the other cases try following the robustness principle. Find more about this in this article.

The best practices for writing any part of the contract are:

Always use example value and a matcher (regex, type, date formats, etc.)

For example, this code expects that query parameter amount will be any integer but 2 as an example and an exact value in the request on the provider side :

.matchQuery("amount", ".d*", "2") 

4. Use fewer constants and common components in tests

If you like and know how to write reusable code it's not necessary these patterns must be applied to any testing code. We make it worse when we write common components in tests chasing the DRY principle. It brings unnecessary cohesion to your tests thus when you change a value in a common place you break N tests instead of one.

Try using fewer constants and common components in your tests

Making tests cohesive we:

• make the HTTP semantic less visible by looking at the pact code
• add redundant coupling: it's not easy to play with your code changing a constant because other tests will fail using that constant value

5. Use fewer production model classes in your tests code

This point is close to the 4th and the 1st. Relying on any production code or giving credit of trust to the developer who wrote that. Though that model doesn't need to reflect 1:1 the integration, there might be in place: converts, serialisation configurations, etc. You may miss this, so it's easier just to use plain JSONs and, respecting the 3d point, library DSLs.

Relying on the production code in your tests inputs we make it difficult following the TDD pattern

Another argument, respecting the 1st point, is that you think about the code instead of the actual JSON writing the test which brings you to assumptions.

6. Always verify the contract on the provider side

The contract testing with Pact is about the consumer and provider. Writing just the pact and finishing your work you can not be a curtain that you covered the integration.

Every pact must be verified on the provider side to bind the consumer and the provider

In my experience, developers that write tests separately end up in a situation where after the merge they have to do an additional job later to align/verify the contract on the provider side. It might be very exhausting work so:

Better writing fewer tests but always verifying all of them on the provider side

7. Mock as deeper as it's possible on the provider side

tl;dr

Mock the repository level or insert it into your database in the tests

Here is a great explanation video

You might get an idea to mock the controller level and this idea is bad here is why:

• All the following tests will follow the same principle because it costs a lot to restart the application for every test - no one does that, you will have the same set of beans (mocks)
• You isolate the major part of your application and you simply can't get some layer that causes an exception that you need to cover a contract

More examples:

Context: in Spring Boot applications classically we have layers: controller, repository, service etc. You can mock any layer of those by mocking a bean so that a mock returns the result of the corresponding method.

1. Less cohesive. The mocks do not really describe a state of a system (service) rather they do a small snapshot of a layer that depends on DTO classes. It makes mocks less cohesive with tests, thus it's a high chance that tests based on mocks will pass but on practice, the example data in mocks will never appear.
2. Making assumptions of application behavior. Development by mocks makes developers think about exceptions rather than the actual application behavior. Imagine you want to prepare a state that your service returns 400 when the inputs are invalid. Here as well, it makes it difficult for non-owner of the provider application to write and understand the state. The provider developer relying on a mock, e.g. exception class will prepare the state assuming that the exact exception causes the issue. Though it can be any other exception, hence when something changes the exception in the production code - the test will be not reliable.
3. Tests as examples. Pact with consumer-driven contracts is development-by-example, and if a layer is mocked, it says nothing to the observer about the actual system behavior. You can't use the test with mock as an example because you do not know how to reproduce this layer behavior.

Conclusions

• Follow TDD in your tests thinking about the API rather then the production code
• Think about reusability and how others will copy your approaches in your company

I know this article contains no code examples though I have ones in this Github repository breaking-api-bad and please not do hesitate to contact me.