Snapshot Testing in Python

Have you ever found yourself writing long, tedious tests that take in a large data structure, and test the output of a similar, validated data structure? Whether it's converting a dictionary to a named tuple or serializing an object, if you've ever tried testing a data model or a GraphQL model for an API, you've probably experienced this.

Writing tests and replicating the same response we expect can be a tedious and monotonous task. Snapshot testing can be a great way to ease the pain of testing these data structures. As our APIs evolve, we need to know when our changes introduce any breaking changes, and a snapshot test will tell us exactly that.

What is Snapshot testing?

Put simply, A snapshot is a single state of your output, saved in a file.

A typical snapshot test case for a mobile app renders a UI component, takes a screenshot, then compares it to a reference image stored alongside the test. The test will fail if the two images do not match: either the change is unexpected, or the screenshot needs to be updated to the new version of the UI component.

A similar approach can be taken when it comes to testing your APIs or data structures. Instead of rendering the graphical UI, you can use a test renderer, such as the SnapshotTest library, to quickly generate a serializable value for your API response.

Snapshots lets us write all these tests in a breeze, as it automatically creates the snapshots for us the first time the test is executed. Then, every subsequent time the tests are run, it will compare the output with the file snapshot. If they are different, our tests will fail, immediately alerting us of the change.

How to use the SnapshotTest library

First, we install the library:

pip install snapshottest

I'll provide code snippets below for both the python built-in unittest library and pytest, but you can probably use this with any testing framework you choose.

We'll begin with a simple namedtuple that we'd like to test.

'''Our Clown Code'''

from collections import namedtuple

Clown = namedtuple('Clown', 'name', 'nose_type'))

With unittest:

from snapshottest import TestCase

class TestClown(TestCase):

    def test_clown(self):
      clown = Clown('bozo', 'red and round')                                  
      self.assertMatchSnapshot(clown)  

# >> python -m unittest

With pytest:

import pytest

def test_clown(snapshot):
      clown = Clown('bozo', 'red and round')                                  
      snapshot.assert_match(clown) 

# >> pytest

This will create a snapshot directory and a snapshot file the first time the test is executed.

>> pytest
=================== SnapshotTest summary ===================
1 snapshot passed.
1 snapshot written in 1 test suites.

Next time we run our tests, it will compare the results of our test with the snapshot. Since nothing changed, we get the following:

>> pytest
=================== SnapshotTest summary ===================

1 snapshot passed.

If our data structure or output changes, the snapshot tests will fail. We can then compare the changes and determine whether they are valid. To update the snapshot we can run:

pytest --snapshot-update

Obviously the above is a super simple test case, and we're not really testing any logic. But this same pattern can be applied for testing API requests.

import requests

def test_my_api(snapshot):
    response = request.get('my_api.com')
    snapshot.assert_match(response)

Or for testing a GraphQL API

from graphene.test import Client

def test_hey(snapshot):
    client = Client(my_schema)
    response = client.execute('''{ hey }''')
    snapshot.assert_match(response)

Snapshot testing may not be a replacement for unit or integration tests, but it's a great way to quickly and effectively determine whether the state of your API or data serialization has changed.