A few days ago I wrote the second part of the "Designing Restful APIs using an API-First approach" post series, I really recommend you check the second post to understand how OpenAPI will help you have a MockServer without writing any code.
Today we will talk about contract testing and how to ensure that the API implementation meets the design proposed.
When we talk about APIs, we are talking about contract definition, so we must have in mind that after an API is made available, the defined contract must be supported by the API provider.
Even after the API was made available, it keeps evolving, and one of the biggest challenges during an API Life-cycle is the API evolution. Because sometimes, business requirements change so much, that the current contract doesn’t meet new requirements and a breaking change is required.
😈 Breaking Changes — The root of all evil
As I usually say APIs are contracting definitions and we can’t break a contract without any bad consequence.
When a new functionality breaks the current state of the contract, if you don’t manage it very well, you will have problems with your customers that will have their clients broken after a new release of your API.
So in order to have fast feedback during the API development, you must know if a change introduces a breaking change.
🧪 Possible Approaches
There are a couple of possible approaches that can help you achieve fast feedback about breaking changes.
Unit testing
Running a unit test against your API will let you know when the contract is broken because your test will start to fail.
It's a very easy strategy to start, and almost the applications already have a unit test suit, in the order hand, this approach is too easy to be bypassed because the developer can just fix the test to succeed and move forward.
API Versioning
Create a new version of API for each new release, keeping old releases until your clients move to the new one.
We know that API Versioning is a best practice, but if you create a new version of your API, for each release, without a shadow of a doubt this will become messy very fast, and you will have big problems keeping all those versions together.
Contract Test
Check for differences between what was designed during the API Design and what was developed, running it in the Continuous Integration pipeline will provide us fast feedback, this is a very flexible and effective approach.
What approach should I use?
If we look at the first two options, we can find clear drawbacks, either by process bypass or maintainable problems.
Comparing the Contract Test against two other approaches, we can see that developers could not bypass the process without making a change in the original OpenAPI Document, this change must be reviewed for your team and if possible API Stake Holders as well.
It's easier to be automated and provide a clear understanding of what is falling and why it is failing.
🤖 Contract Test Implementation
As described above contract test seems to be the best solution to detect breaking changes in build time, and to help us during this journal, let's use a tool called openapi-diff, this is an npm package that compares two OpenApi documents and looks for what was removed or added.
OpenAPI-diff separates changes into two groups, Breaking Changes and Smooth Changes, let's see some examples:
Breaking Changes
- Delete Path or Parameter
- Rename a Path or Parameter
- Add Required Parameter
- Modify Response Item
Smooth Changes
- Add Path or Parameter
- Add Response Item
- Add or Update Descriptions
To install openapi-diff is too easy, you just need to run one of the commands below.
// using npm
npm install openapi-diff
// using yarn
yarn add openapi-diff
Test Against What?
Probably you are asking yourself, but against what we will check by differences.
A very important step before we move forward in the next steps is to extract the swagger.json generated by your code, it will depend on the platform that you are using to write your application.
In this blog post, I will use an ASP.NET Core API that will be in the same repository that is being used for this series.
To keep it simples, I won’t show how to set up the Swashbuckle in an ASP.NET Core project, but if you want to see how to do that properly, comment below and I can write a post dedicated to this setup.
ASP.NET Core and Swashbuckle CLI
Since you have Swashbuckle configured into your ASP.NET Core project, you can install a Dotnet tool named Swashbuckle.AspNetCore.Cli, it's a very simple tool that extracts the swagger.json using the project DLL, if you want to know more about this tool, please check the Github Document.
After you install the tool as described in the official documentation, you just need to run the command below.
swagger tofile --yaml --output ./bin/swagger.yaml ./api/TodoApp.Api/bin/Debug/netcoreapp3.1/TodoApp.Api.dll v1
The command above will extract the swagger.json and convert it to a YAML file, this document looks like the OpenAPI document that we wrote during the API Design process.
OpenAPI-Diff in Action
Now that we have the OpenAPI Document that was designed and the OpenAPI Document that was generated through the code, it's time to compare those files and check the output.
Just run the command above and let see the console output.
openapi-diff ./bin/api.yaml ./bin/swagger.yaml
Without Changes
With Changes
I just made some changes in the code, add a new query string parameter to the GET /tasks endpoint, and removed the status code from the GET /tasks/{id} endpoint, we can see the output below.
As you can see above, the change in the query string parameter is not logged, but the change that removed the status code was detected as a breaking change.
Now you can just add the OpenAPI Diff to your API Pipeline to detect possible breaking changes every time that someone changes the API implementation and push it to your repository.
This strategy allows the developer to run the same script in their local machine avoiding broken pipelines, having a fail-fast validation.
🏁 Conclusion
In my opinion, avoid breaking changes is the most difficult task during the API Life Cycle, and in the majority of the time its related if the Design Session, if we take care of Design for longevity and try to understand possibles corner cases of the domain that the API aims to solve, we can reduce this necessity to break the API Contract when new business requirements appear.
In order to have breaking changes under your control, a contract test strategy may help us with fast feedback and keep us safe to break the contract and create problems for the clients of your API.
As usually every code will be updated in the Github Repository to you run to use it as a complement of this post.
Let me know what do you think about this strategy?
Do you already know it?
Comment below and let's share the experiences.
I hope you enjoy it, see you soon.
Top comments (0)