Asserting Exceptions with Pytest

#python #tdd #learning #tutorial

First time I had someone review my pull requests, she was pretty strict on tests. I couldn't merge if the tests were failing, of course. But I also couldn't merge if coverage had decreased by even 1%. TDD was still new to me so maintaining coverage was a challenge since I was only testing the bare minimum I could. I had to find out how to make my tests more robust and ensure as much of my code was tested as possible. One area that I wasn't really sure how to test was the custom exceptions I had written. Here's an example:



# login.py

def check_email_format(email):
    """check that the entered email format is correct"""
    pass

def test_email_exception():
    """test that exception is raised for invalid emails"""
    pass

This is probably something you want to do if you're implementing a system with email authentication. The example is oversimplified, but it serves the purpose of this post well.

To test for raised exceptions, pytest offers a handy method: pytest.raises. Let's see how to use it in our example:



import re
import pytest

def check_email_format(email):
    """check that the entered email format is correct"""
    if not re.match(r"(^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$)", email):
        raise Exception("Invalid email format")
    else:
        return "Email format is ok"

def test_email_exception():
    """test that exception is raised for invalid emails"""
    with pytest.raises(Exception):
        assert check_email_format("good@email.com")

The check_email_format method takes in an email and checks that it matches the regex pattern given. If it does, it returns "Email format is ok", otherwise, an exception is raised.

Using pytest.raises in a with block as a context manager, we can check that an exception is actually raised if an invalid email is given. Running the tests on the code as it is above should fail:



collected 1 item                                                                                                                                                                                       
login.py F                [100%]

==================== FAILURES ========================

    def test_email_exception():
        """test that exception is raised for invalid emails"""
        with pytest.raises(Exception):
>           assert check_email_format("good@email.com")
E           Failed: DID NOT RAISE <class 'Exception'>

login.py:16: Failed

Notice it says Failed: DID NOT RAISE <class 'Exception'>. If an exception is not raised, the test fails. I found this to be pretty awesome. We passed in a valid email format (according to our standards here) so the test works as expected. Now we can make it pass.



import re
import pytest

def check_email_format(email):
    """check that the entered email format is correct"""
    if not re.match(r"(^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$)", email):
        raise Exception("Invalid email format")
    else:
        return "Email format is ok"

def test_email_exception():
    """test that exception is raised for invalid emails"""
    with pytest.raises(Exception):
        assert check_email_format("bademail.com") # invalid email format to raise exception

Run your test: pytest login.py:



collected 1 item                         

login.py .              [100%]

====================== 1 passed in 0.05 seconds ======================

You can also add an extra check for the exception message:



import re
import pytest

def check_email_format(email):
    """check that the entered email format is correct"""
    if not re.match(r"(^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$)", email):
        raise Exception("Invalid email format")
    else:
        return "Email format is ok"

def test_email_exception():
    """test that exception is raised for invalid emails"""
    with pytest.raises(Exception) as e:
        assert check_email_format("bademail.com")
    assert str(e.value) == "Invalid email format"

Top comments (7)

rhymes • Jan 29 '19 • Edited

I know it's just an example but I think I should mention it for beginners: there are not many cases when raising Exception is the best strategy. The reason being that Exception is the class from which most error classes inherit from.

>>> Exception.__mro__
(<class 'Exception'>, <class 'BaseException'>, <class 'object'>)
>>> ValueError.__mro__
(<class 'ValueError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>)

It would make it harder for the caller to properly handle different types of exceptions.

In your case ValueError (or a custom exception) is probably more appropriate:

Raised when an operation or function receives an argument that has the right type but an inappropriate value

A bonus tip: pytest.raises accepts an argument that you might find useful: match. You could rewrite:

 with pytest.raises(Exception) as e:
        assert check_email_format("bademail.com")
    assert str(e) == "Invalid email format"

with pytest.raises(Exception, match="Invalid email format"):
        assert check_email_format("bademail.com")

match is used to check the error message with a regular expression.

Hope this helps!

Toshio Kuratomi • Aug 9 '19

I just wanted to correct a common mistake in this comment since it was one of the first results from my google search. message is actually used for setting the message that pytest.rasies will display on failure. It's not about a comparison to the exception's message. match should always be used for that purpose. docs.pytest.org/en/latest/referenc...

(This problem catches almost everyone at one point or another. Which is the reason that pytest has chosen to deprecate the message parameter ;-)

rhymes • Aug 9 '19

thank you Toshio! I'll update the comment!

What did you search on Google to get here?

Kelvin Wangonya • Jan 29 '19

Yeah, it helps! Thanks for the input. It was just an example, of course in an ideal situation things would be different. I should have mentioned that.

Thanks again!