Introduction to testing with Python for beginners

Introduction

You started programming with Python, write code and solve problems. On this article we'll discuss how you can write tests for your code, you can run them quickly to make sure everything works instead of running your app over and over to make sure your changes work.

Our program and making sure it works

We will use a very simple example that will help you get the idea of the problem you may face and the solution tests can bring.
The idea is for the example to be super simple (dumb even) so you can focus on the new concept instead of on the example code iteself.

Let's say we have this program where we ask the user for two numbers and we multiply them.

# file: main.py
from mymath import multiply

print("Mutliply two numbers")
a = input("first number: ")
b = input("second number: ")
print("result:", multiply(int(a), int(b)))

Again, this is a simple program, in the real world you could be asking for data relevant to any problem and do complex calculations with them.

And say we have this implementation for multiplication:

# file: mymath.py
def multiply(a, b):
    # sum "a" "b" times, 2*3 -> 2+2+2
    result = 0
    for _ in range(b):
        result += a

    return result

Let's run the app on the command line:

$ python3 main.py
Mutliply two numbers
first number: 2
second number: 3
result: 6

$ python3 main.py
Mutliply two numbers
first number: 5
second number: 6
result: 30

So far so good, let's try a negative number:

$ python3 main.py
Mutliply two numbers
first number: -4
second number: 4
result: -16

Still works, let's try again:

$ python3 main.py
Mutliply two numbers
first number: 4
second number: -4
result: 0

Alright, we found a bug!

Testing our program with another program

When your program is larger and there are many more options than just one math operation, it will take you much more program runs to actually find some cases that are not working.

And more importantly, you can test all the things in a blink every time you make changes on your app without having to worry if your change broke something that you didn't notice.

Let's write a little program that do this testing for us:

# file: mymath_tests_v1.py
from mymath import multiply

if (multiply(2, 2) != 4):
    raise Exception("result error")

if (multiply(2, 4) != 8):
    raise Exception("result error")

if (multiply(2, -2) != -4):
    raise Exception("result error")

Let's run it:

$ python3 mymath_test_v1.py
Traceback (most recent call last):
  File "mymath_test_v1.py", line 10, in <module>
      raise Exception("result error")
      Exception: result error

Now, we are checking the same thing that we would check running the app, but with code. And as added value, we are documenting the use cases we support on our app, and the ones that are more relevant when we want to make sure that the program works.

This is basically what we can call a "test", or "unit test". Code that checks that make sure that your code works. There are other types of tests, for other purposes, but that's out of the scope of this article.

Improving our tests

The approach we just explored was only a crude implementation of tests using just conditionals, let's take it a step further.

Instead of manually writing conditionals and raising exceptions, we'll make use of the assert keyword.

It basically works like this:

$ python3
>>> assert(2 == 2)  # no error
>>> assert(2 == 3)  # throws exception
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

Here's how we can use it for our tests:

# file: mymath_tests_v2.py
from mymath import multiply

assert(multiply(2, 2) == 4)
assert(multiply(2, 4) == 8)
assert(multiply(5, 6) == 30)

$ python3 mymath_tests_v2.py
Traceback (most recent call last):
  File "mymath_tests_v2.py", line 5, in <module>
    assert(multiply(2, -2) == -4)
AssertionError

As you may have noticed, not only our code is shorter but when an assertion fails, there's information on the exception about why it failed and instead of just getting the line where the problem appeared we get the expression we run, and that makes figuring out the problem much easier.

Improving our tests, using a framework

Testing can be improved even further, there are frameworks (sometimes called libraries, modules or packages) that provide some extra tools for us to make writing and running tests even easier, as well as providing much nicer results for tests that pass and fail.

Python comes with its own framework to run tests, it's called unittest. See its documentation

Here's how we would write our test using it:

# file: test_mymath_simple.py
import unittest
from mymath import multiply

class MyMathTest(unittest.TestCase):
    def test_basics(self):
        self.assertEqual(multiply(2, 2), 4)
        self.assertEqual(multiply(2, 4), 8)
        self.assertEqual(multiply(2, -2), -4)

If you're not familiar with class you can ignore the details and instead of writing MyMathTest you can write whatever you want, and all the functions have to start with test_.

You have many self.assert prefixed helper functions to test different things, we can stick with equality comparison for now.

Let's run these tests:

$ python3 -m unittest -v test_mymath_simple.py
test_basics (test_mymath_simple.MyMathTest) ... FAIL

======================================================================
FAIL: test_basics (test_mymath_simple.MyMathTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ivan/data/Devel/blog/test_mymath_simple.py", line 9, in test_basics
    self.assertEqual(multiply(2, -2), -4)
AssertionError: 0 != -4

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

We are getting more output (partly because of the -v flag) and some of it is valuable: for example 0 != -4, that's exactly the condition that failed. Results are more explicit for us, so it's much easier to figure out the problem.

Remember, here's easy because of our really small example, but on larger programs gets much harder.

Using unittests like in the real world

# file: test_mymath.py
import unittest
from mymath import multiply

class MyMathTest(unittest.TestCase):
    def test_small(self):
        self.assertEqual(multiply(2, 2), 4)
        self.assertEqual(multiply(2, 4), 8)
        self.assertEqual(multiply(10, 10), 100)

    def test_zero(self):
        self.assertEqual(multiply(2, 0), 0)
        self.assertEqual(multiply(0, 2), 0)

    def test_negative(self):
        self.assertEqual(multiply(-2, 2), -4)
        self.assertEqual(multiply(-2, -2), 4)
        self.assertEqual(multiply(2, -2), -4)

On this example, we have different test "groups", in which we make sure different aspects of our program behave as it should.
Those are run separately and we'll get information about which of them work and which of them fail.

Let's run our new test "suite".

$ python3 -m unittest -v test_mymath.py
test_negative (test_mymath.MyMathTest) ... FAIL
test_small (test_mymath.MyMathTest) ... ok
test_zero (test_mymath.MyMathTest) ... ok

======================================================================
FAIL: test_negative (test_mymath.MyMathTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ivan/data/Devel/blog/test_mymath.py", line 17, in test_negative
    self.assertEqual(multiply(-2, -2), 4)
AssertionError: 0 != 4

----------------------------------------------------------------------
Ran 4 tests in 0.001s

FAILED (failures=1)

You can have many test files, and as long as they start with test_ you can run python3 -m unittest -v and it'll run all your test, wherever they are.

Closing thoughts

Tests allow you to verify your program quickly, frequently and consistently. Using them you can have more confidence on your app and on the changes you make to it.

There's much more to learn about testing, but hopefully this introduction will be enough for you to get started testing your code.

Thre are several third party libraries that provide very nice functionalities on top of what you get from the standard library.
Once you're comfortable writing tests, you can give some of them a try and see how do you like them.