DEV Community

Cover image for Operation Pact or: How I Learned to Stop Worrying and Love Contract Testing
Anton Yakutovich
Anton Yakutovich

Posted on

Operation Pact or: How I Learned to Stop Worrying and Love Contract Testing

#testing #devops #pactflow

I heard about Contract Tests in detail at the Ministry of Testing Meetup. Abhi Nandan made an excellent introduction talk Contract testing in Polyglot Microservices.

I liked the idea and decided to try this approach on my current project. I must say that the devil is in the details. In theory, everything is simple until you try the integration in practice on an existing project.
It is the same with algorithms: when you are watching how the developer on YouTube solves the task, everything seems straightforward. You immediately hit a snag while sitting down to solve the problem alone.

Today I want to focus on practical examples, as much as possible close to real-world conditions. I encourage you to try it on your own to gain experience and see if these approaches are worth applying to your projects. We will use the realword project on GitHub, an Exemplary Medium.com clone powered by different frameworks and tools.

I like using this project for the demo because it's much more complex than HelloWorld, where everything works most of the time. True-to-life problems give your extensive experience.

What is so unique in contract tests?

Contract tests assert that inter-application messages conform to a shared understanding documented in a contract. Without contract testing, the only way to ensure that applications will work correctly together is by using expensive and brittle integration tests.

You can hear opinions: Contract tests often make sense when you have a microservice architecture with 10-50 services.
I'm afraid I have to disagree with that. If you have at least one Provider (back-end with API) and Consumer (front-end), it already makes sense to try this testing approach.

I suggest starting exploration with bi-directional contract tests.

Bi-Directional Contract Testing is a type of static contract testing.
Teams generate a consumer contract from a mocking tool (such as Pact or Wiremock) and API providers verify a provider contract (such as an OAS) using a functional API testing tool (such as Postman). Pactflow then statically compares the contracts down to the field level to ensure they remain compatible.

Why is it better to start with BDCT? Bi-Directional Contract Testing (BDCT) allows you to use existing Provider Open API. If your current project has a back-end service with Open API specification, you are lucky to leverage it as a Provider contract.

1. Provider Contract

Let's configure publishing of OAS spec from realworld app to pactflow.

Requirements:

We want to be able to publish Open API spec to Pactflow locally and from CI/CD. To do that we will need the following: API key from Pactflow, docker and make commands installed.

Consider the Makefile:

# Makefile
PACTICIPANT ?= "realworld-openapi-spec"

## ====================
## Pactflow Provider Publishing
## ====================
PACT_CLI="docker run --rm -v ${PWD}:/app -w "/app" -e PACT_BROKER_BASE_URL -e PACT_BROKER_TOKEN pactfoundation/pact-cli"
OAS_FILE_PATH?=api/openapi.yml
OAS_FILE_CONTENT_TYPE?=application/yaml
REPORT_FILE_PATH?=api/README.md
REPORT_FILE_CONTENT_TYPE?=text/markdown
VERIFIER_TOOL?=newman

# Export all variable to sub-make if .env exists
ifneq (,$(wildcard ./.env))
    include .env
    export
endif

default:
    cat ./Makefile

ci: ci-test publish_pacts can_i_deploy

# Run the ci target from a developer machine with the environment variables
# set as if it was on CI.
# Use this for quick feedback when playing around with your workflows.
fake_ci:
    @CI=true \
    GIT_COMMIT=`git rev-parse --short HEAD` \
    GIT_BRANCH=`git rev-parse --abbrev-ref HEAD` \
    make ci

## =====================
## Build/test tasks
## =====================

ci-test:
    @echo "\n========== STAGE: CI Tests ==========\n"

## =====================
## Pact tasks
## =====================

publish_pacts:
    @echo "\n========== STAGE: publish provider contract (spec + results) - success ==========\n"
    PACTICIPANT=${PACTICIPANT} \
    "${PACT_CLI}" pactflow publish-provider-contract \
    /app/${OAS_FILE_PATH} \
    --provider ${PACTICIPANT} \
    --provider-app-version ${GIT_COMMIT} \
    --branch ${GIT_BRANCH} \
    --content-type ${OAS_FILE_CONTENT_TYPE} \
    --verification-exit-code=0 \
    --verification-results /app/${REPORT_FILE_PATH} \
    --verification-results-content-type ${REPORT_FILE_CONTENT_TYPE} \
    --verifier ${VERIFIER_TOOL}

deploy: deploy_app record_deployment

can_i_deploy:
    @echo "\n========== STAGE: can-i-deploy? 🌉 ==========\n"
    "${PACT_CLI}" broker can-i-deploy --pacticipant ${PACTICIPANT} --version ${GIT_COMMIT} --to-environment test

deploy_app:
    @echo "\n========== STAGE: deploy ==========\n"
    @echo "Deploying to test"

record_deployment:
    "${PACT_CLI}" broker record-deployment --pacticipant ${PACTICIPANT} --version ${GIT_COMMIT} --environment test

## =====================
## Misc
## =====================

.env:
    cp -n .env.example .env || true
Enter fullscreen mode Exit fullscreen mode

Run make .env to create .env file. You will need to update two variables:

  • PACT_BROKER_BASE_URL
  • PACT_BROKER_TOKEN

[Optional] We may try to publish the contract from the local machine using make fake-ci. This target under the hood executes the following stages:

  1. ci-test runs functional tests (skipped in our case)
  2. publish_pacts uses pactflow docker image to publish current OAS to Pactflow broker. GIT_COMMIT and GIT_BRANCH environment variables are used to set the version tag and the branch on the broker side.
  3. can_i_deploy verifies that the published contract is compatible with dependent consumers. Hence we are safe to deploy.

And the last piece. Let's create a GitHub workflow to publish OAS on each push to the repository:

# .github/workflows/pact.yml

name: Pact CI

on:
  pull_request:
  push:
    branches:
      - main
    paths-ignore:
      - '*.md'

concurrency:
  # For pull requests, cancel all currently-running jobs for this workflow
  # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

env:
  PACT_BROKER_BASE_URL: https://drakulavich.pactflow.io
  PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
  GIT_COMMIT: ${{ github.sha }}

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3

    - name: Inject slug/short variables
      uses: rlespinasse/github-slug-action@v4

    - name: Tests
      run: GIT_BRANCH=${GITHUB_REF_NAME_SLUG} make ci

  deploy:
    runs-on: ubuntu-latest
    needs:
      - verify
    if: ${{ github.ref == 'refs/heads/main' }}
    steps:
      - uses: actions/checkout@v3

      - name: Inject slug/short variables
        uses: rlespinasse/github-slug-action@v4

      - name: 🚀 Record deployment on Pactflow
        run: GIT_BRANCH=${GITHUB_REF_NAME_SLUG} make deploy
Enter fullscreen mode Exit fullscreen mode

GitHub will use the same Makefile. Please, have a look at the Deploy job from the workflow above. We are executing make deploy to explicitly mark the deployed version for the test environment on the Pactflow broker.

We will provide the environment variable to access Pactflow differently. Don't forget to add the secret PACT_BROKER_TOKEN in Settings — Secrets — New repository secret.

PACT_BROKER_TOKEN repository secret

Ok. We are ready to commit the files! If everything is fine, the pipeline should be green.

Pipeline to publish OAS to Pactflow

Open your Pactflow workspace and check out the Provider Contract. You should be able to open Open API Swagger spec.

Open API Swagger on Pactflow

So, we finished with the first part. We have CI/CD for Provider Contract. Let's move on to the second part.

2. Consumer contract

We will use ts-redux-react-realworld-example-app project which implements the front-end in TS for realword app using React/Redux.

If you want to follow the same approach, I suggest you to check pact-workshop-js for better understanding.

1) We'll start by installing @pact-foundation/pact dependency:

npm install --save-dev @pact-foundation/pact
Enter fullscreen mode Exit fullscreen mode

2) Create directory src/services/consumer/
3) Create src/services/consumer/apiPactProvider.ts
Here we'll describe the pact specification format and name of our pacticipants (participants).

// src/services/consumer/apiPactProvider.ts

import path from 'path';
import { PactV3, MatchersV3, SpecificationVersion } from '@pact-foundation/pact';

export const { eachLike, like } = MatchersV3;

export const provider = new PactV3({
  consumer: 'ts-redux-react-realworld-example-app',
  provider: 'realworld-openapi-spec',
  logLevel: 'warn',
  dir: path.resolve(process.cwd(), 'pacts'),
  spec: SpecificationVersion.SPECIFICATION_VERSION_V2,
});
Enter fullscreen mode Exit fullscreen mode

4) Create src/services/consumer/api.pact.spec.ts

Let's start with something tiny and achievable. We would like to check /tags endpoint.

// src/services/consumer/api.pact.spec.ts

describe('API Pact tests', () => {
  describe('getting all tags', () => {
    test('tags exist', async () => {
      // set up Pact interactions

      // Execute provider interaction
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

From pact-workshop-js you will learn that each Pact test contains two parts:

  1. Setup Pact interactions.
  2. Execute the interaction using the API client.

The first part might look like:

// src/services/consumer/api.pact.spec.ts
// ...
      const tagsResponse = {
        tags: ['reactjs', 'angularjs'],
      };

      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'tags exist' }],
        uponReceiving: 'get all tags',
        withRequest: {
          method: 'GET',
          path: '/tags',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(tagsResponse),
        },
      });
// ...
Enter fullscreen mode Exit fullscreen mode

We created a stub for the response tagsResponse. Then we added interaction to the provider object from src/services/consumer/apiPactProvider.ts. The essential parts relate to the expected request and how the provider will reply.

In the second part we will add the test execution:

// src/services/consumer/api.pact.spec.ts
// ...

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const tags = await getTags();

        expect(tags).toStrictEqual(tagsResponse);
      });
// ...
Enter fullscreen mode Exit fullscreen mode

Ultimately we need to change the URL for the API client and execute the request after it. In our case we have to alter axios baseURL, because all API interactions explicitly use axios. For example, getTags() implementation:

export async function getTags(): Promise<{ tags: string[] }> {
  return guard(object({ tags: array(string) }))((await axios.get('tags')).data);
}
Enter fullscreen mode Exit fullscreen mode

We want to verify that response is the same as a prepared stub. After collecting all parts inside the test file we will get the following:

// src/services/consumer/api.pact.spec.ts

import axios from 'axios';
import { provider, like } from './apiPactProvider';
import { getTags } from '../conduit';

describe('API Pact tests', () => {
  describe('getting all tags', () => {
    test('tags exist', async () => {
      const tagsResponse = {
        tags: ['reactjs', 'angularjs'],
      };

      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'tags exist' }],
        uponReceiving: 'get all tags',
        withRequest: {
          method: 'GET',
          path: '/tags',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(tagsResponse),
        },
      });

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const tags = await getTags();

        expect(tags).toStrictEqual(tagsResponse);
      });
    });
  });
});
Enter fullscreen mode Exit fullscreen mode

Let's run make fake-ci to check how is going:

Computer says yes \o/ 

CONSUMER                             | C.VERSION          | PROVIDER               | P.VERSION  | SUCCESS? | RESULT#
-------------------------------------|--------------------|------------------------|------------|----------|--------
ts-redux-react-realworld-example-app | d981433+1663657371 | realworld-openapi-spec | 95dbd23... | true     | 1
Enter fullscreen mode Exit fullscreen mode

Hooray! It's a huge step towards contract testing.
Now let's try to test another endpoint GET /user. We will start with the same ideas: create a stub, prepare interactions and execute API client call:

// src/services/consumer/api.pact.spec.ts
// ...

  describe('getting current user', () => {
    test('user exists', async () => {
      const tUser: User = {
        email: 'jake@jake.jake',
        token: 'jwt.token.here',
        username: 'jake',
        bio: 'I work at statefarm',
        image: 'https://i.stack.imgur.com/xHWG8.jpg',
      };

      const userResponse = {
        user: tUser,
      };
      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'user has logged in' }],
        uponReceiving: 'get user',
        withRequest: {
          method: 'GET',
          path: '/user',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(userResponse),
        },
      });

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const user = await getUser();

        expect(user).toStrictEqual(tUser);
      });
    });
  });

// ...
Enter fullscreen mode Exit fullscreen mode

If you run make fake-ci, you will get an error:

Computer says no ¯_(ツ)_/¯
Enter fullscreen mode Exit fullscreen mode

Follow the generated report link and you will see the details:

Request Authorization header is missing but is required by the spec file
Enter fullscreen mode Exit fullscreen mode

Error on Pactflow side

We missed Authorization header that is required to check logged in user. Let's add the stub (1), headers to the expected request (2) and axios defaults (3):

// src/services/consumer/api.pact.spec.ts

      // (1)
      const authToken = {
        Authorization: 'Token xxxxxx.yyyyyyy.zzzzzz',
      };

// ...

        uponReceiving: 'get user',
        withRequest: {
          method: 'GET',
          path: '/user',
          headers: authToken, // (2)
        },
// ...
      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;
        axios.defaults.headers.Authorization = authToken.Authorization; // (3)
Enter fullscreen mode Exit fullscreen mode

After next make fake-ci attempt you will get the new error:

FAIL src/services/consumer/api.pact.spec.ts
  ● Console

    console.error
      Error: Cross origin http://localhost forbidden
Enter fullscreen mode Exit fullscreen mode

I found the fix for it on StackOverflow. We need to add the following line on the top of the test file:

axios.defaults.adapter = require('axios/lib/adapters/http');
Enter fullscreen mode Exit fullscreen mode

Next make fake-ci should be successful.

Pactflow broker UI

Summary

We have learned how to configure Provider and Consumer for bi-directional contract testing. We used RealWorld project to practice integration with Pactflow broker. And finally, we wrote a couple of consumer tests for ts-redux-react-realworld-example-app. We also learned how to debug the tests using make fake-ci command. All of the examples are available on GitHub. Feel free to try and post your questions if you face the issues.

Materials

Top comments (0)

Read next

jphi_baconnais profile image

✨ Discovering GitLab Duo 🤖

Jean-Phi Baconnais -

ericbsantana profile image

Creating a free-tier AWS RDS PostgreSQL instance using Terraform

Eric Santana -

chigozieco profile image

Configure Postfix to Send Email with Gmail's SMTP From the Terminal

ChigozieCO -

dm8ry profile image

How to check the distribution of timestamps in a PostgreSQL table?

Dmitry Romanoff -