Test organization in Python

For quite some time now Python has been my hacking and proof-of-concept language of choice.

As such, testing in Python never really was an important issue for me because I wouldn't use the code in production or even revisit it most of the time.

Recently, I have been writing a small flask app in python to calibrate and continuously read from an HX711 load cell with a Raspberry Pi Zero W.

Since I'm using this code as part of my bachelor thesis I consider it to be 'production' code and most importantly I want it to be easy to refactor and extend.

So for the first time I had to think about testing in Python.

The setup

First things first, my project without tests looks like this:

As you can see, it is a very simple setup with 2 modules backend and service and an entry point server.py.

The Goal

For easy testing I want to be able to define tests for each module and be able to run them separately, as well as an easy way of running all tests of all modules.

The solution

For testing I will use pythons unittest module. The smallest unit for testing with unittest is the TestCase which can have 1 or more methods of the form def test_*():.

To begin, I tried to separate the different responsibilities of my modules into TestCase classes, eg.:

    import unittest

    class TestCalibrationChecks(unittest.TestCase):

        """
        methods that test whether all checks on the calibration state of the scale work
        """

    class TestCalibrationProcess(unittest.TestCase):

        """
        methods that test calibration of the scale
        """

After writing test cases for the modules my project looked like this:

Running a TestCase from the project root looks like this: python -m unittest backend.test.calibration_test.TestCalibrationChecks

Now I wanted to be able to run all tests in each module so I looked around the docs and came to unittest's TestSuite.

A TestSuite is a collection of TestCases and/or TestSuites and can be used to group tests together.

So in the root of my modules I wrote a file tests.py to combine all TestCases from that module's test folder into a single suite.
For the backend module it looked like this:

    import unittest

    from backend.test import TestCalibrationChecks
    from backend.test import TestCalibrationProcess
    from backend.test import TestSystemHealthChecks

    def test_scale_suite():
        scale_test_suite = unittest.TestSuite([
            unittest.TestLoader().loadTestsFromTestCase(TestCalibrationChecks),
            unittest.TestLoader().loadTestsFromTestCase(TestCalibrationProcess),
            unittest.TestLoader().loadTestsFromTestCase(TestSystemHealthChecks)
        ])
        result = unittest.TestResult()
        runner = unittest.TextTestRunner()
        print(runner.run(scale_test_suite))

    if __name__ == '__main__':
        test_scale_suite()

After writing a file like that for each module my project looked like this:

I was now able to run tests for each module from the project root with a command like this: python -m unittest backend.tests.

But what about running all tests from all modules with a single command?

With a tiny refactor of backend/tests.py and service/tests.py I was able to write a tests.py in the root directory and combine the backend and service test suites to a larger TestSuite

First I changed the tests.py in each module so that the TestSuite isn't created in the function but is an exported variable:

    import unittest

    from backend.test import TestCalibrationChecks
    from backend.test import TestCalibrationProcess
    from backend.test import TestSystemHealthChecks

    scale_test_suite = unittest.TestSuite([
        unittest.TestLoader().loadTestsFromTestCase(TestCalibrationChecks),
        unittest.TestLoader().loadTestsFromTestCase(TestCalibrationProcess),
        unittest.TestLoader().loadTestsFromTestCase(TestSystemHealthChecks)
    ])

    def test_scale_suite():
        result = unittest.TestResult()
        runner = unittest.TextTestRunner()
        print(runner.run(scale_test_suite))

    if __name__ == '__main__':
        test_scale_suite()

Now I was able to define tests.py in the project root like this:

    import unittest

    from backend import tests as backend_tests
    from service import tests as service_tests

    complete_test_suite = unittest.TestSuite([
        backend_tests.scale_test_suite,
        service_tests.service_test_suite
    ])


    def run_all_suites():
        result = unittest.TestResult()
        runner = unittest.TextTestRunner()
        print(runner.run(complete_test_suite))


    if __name__ == '__main__':
        run_all_suites()

The finished project now looks like this:

And from the project root I am able to run

• all tests python -m tests
• backend tests python -m unittest backend.tests
• service tests python -m unittest service.tests

Mission Accomplished

Notes

• I am aware there is a function unittest.TestLoader().discover() which will automatically discover all TestCases in a file tree. I chose not to use it because I like to explicitly declare what to run so I can see what to expect and avoid some tests not running because the discoverer missed them for some reason.
• I am not saying this is the right way to organize your tests in Python, it just seems to work very well for me so far. Happy to hear your comments on this :)