In Mettle, Pact was used to break up deployments and releases so that big bang releases were avoided. Previously testing had been done against a fixed set of versions of each of the services and if any version changed, testing would have to be re-done. I joined after this work had been done, so I had to catch up about how Pact worked.
Using consumer driven contracts is a key step towards getting to continuous delivery, where deployment to production is automatic. Continuous delivery is particularly excellent because changes going into production are small and incremental - if something is broken it’s easy to roll back to the previous version. Also, since the change was small, you know where to look for the bug - it makes the process of releasing much less risky.
These are some things I learnt about Pact, when first starting, which may help you get up to speed quicker!
It tests contracts
What’s particularly key about Pact is that it separates testing the functionality of a service (by feature tests) from the interfaces it provides or consumes. Pact is very specific about what it tests: it’s not testing authorization, authentication, standards, interactions with third party systems - it is testing the format of messages sent between services you own.
It provides fast feedback
Pacts sit in between the unit test and integration test levels of the test pyramid. It can therefore be used for really quick feedback: contract tests can be run as part of the build process, so you know immediately whether your service is compatible.
Consumers only publish contracts
It’s “consumer driven”, so it follows that consumers publish the pacts (contracts) first. Consumers don’t “verify” contracts, they only publish them. For the consumer, it’s actually quite a self-contained process.
Consumers create contracts by running an in memory server, setting up expected requests on the server and then invoking the production code to make that request.
This means that when the consumer build runs, it’s not really running against a provider, it’s doing its own thing by sending requests to a mock Pact server and checking the request is as expected. The only reason this can fail is if the code is sending a different request to what the mock server is expecting.
Providers only verify contracts
Providers only verify the contracts which have been published, they don’t publish contracts.
Providers verify pacts by running the exact request from the contract and then comparing the response they return with the expected response in the contract.
Providers pull contracts down from the broker and use them to create requests to send to the running service (in our case they run against a Spring Boot test which spins up the application in memory). This is where the contract test can fail because of a mismatch between the expected response in the contract compared to the response returned from the service.
Providers need to know when new contracts are published
To be able to get instant feedback, providers need to know when new contracts are published. Then they can run their own build to verify that they provide everything the new contract requires.
The Pact broker can show a lot of version matches in the compatibility matrix
Don’t be afraid of all the version matches shown - that’s normal, and the Pact broker is made to handle that. The reason for the number of versions is that when a provider runs a verification, it verifies against all the tags you specify and publishes each of these to the broker. So if you verify against tags master, dev, staging, prod then you may be creating 4 version matches every time you build the provider. Add that to the fact that we use a git sha as the version of the Pacticipant, and you get even more version matches. This is because each git commit will be a new version in the broker.
Including generated JWTs in your contracts will create a new contract on every consumer build
Because JWTs include time information such as expiry (exp) and issued at (iat), every time the test runs it will likely have a different JWT with a different exp and iat. Try to avoid generating JWTs and using them in your pact. Either don’t send it or use a static long-lived token. This enables Pact to do some optimisation - if the pact doesn’t change it automatically knows all the current providers fulfill it, so the provider tests don’t need to run again verify it.
Using tags it is easy to see the version of each service in each environment.
After a service is deployed to a new environment we tag its version in the Pact broker with the environment name (for consumers we tag the contract, for providers we tag the provider). This is to enable providers to verify against all the versions of the consumers we have deployed. This enables our pipeline to check in the Pact broker that a new version of a service can run against everything that is already running in dev, staging or prod.
There’s a side benefit in that you can see exactly what version of each service is deployed to which environment - pretty cool.
What to test in Pact
What to test in Pact is up to you - but try to keep it to the format of interactions rather than the functionality. For instance we decided to mock the spring service layer in pacts rather than seed the repositories with data to meet the provider states. This bypasses any logic in the services and just keep the test to exercising what the controller does which the HTTP interface.
Conclusion
These were, hopefully useful, facts about Pact!
Top comments (0)