DEV Community

Kelvin Wangonya
Kelvin Wangonya

Posted on

Asserting Exceptions with Pytest

#python #tdd #learning #tutorial

First time I had someone review my pull requests, she was pretty strict on tests. I couldn't merge if the tests were failing, of course. But I also couldn't merge if coverage had decreased by even 1%. TDD was still new to me so maintaining coverage was a challenge since I was only testing the bare minimum I could. I had to find out how to make my tests more robust and ensure as much of my code was tested as possible. One area that I wasn't really sure how to test was the custom exceptions I had written. Here's an example:

# login.py

def check_email_format(email):
    """check that the entered email format is correct"""
    pass

def test_email_exception():
    """test that exception is raised for invalid emails"""
    pass
Enter fullscreen mode Exit fullscreen mode

This is probably something you want to do if you're implementing a system with email authentication. The example is oversimplified, but it serves the purpose of this post well.

To test for raised exceptions, pytest offers a handy method: pytest.raises. Let's see how to use it in our example:

import re
import pytest

def check_email_format(email):
    """check that the entered email format is correct"""
    if not re.match(r"(^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$)", email):
        raise Exception("Invalid email format")
    else:
        return "Email format is ok"

def test_email_exception():
    """test that exception is raised for invalid emails"""
    with pytest.raises(Exception):
        assert check_email_format("good@email.com")
Enter fullscreen mode Exit fullscreen mode

The check_email_format method takes in an email and checks that it matches the regex pattern given. If it does, it returns "Email format is ok", otherwise, an exception is raised.

Using pytest.raises in a with block as a context manager, we can check that an exception is actually raised if an invalid email is given. Running the tests on the code as it is above should fail:

collected 1 item                                                                                                                                                                                       
login.py F                [100%]

==================== FAILURES ========================

    def test_email_exception():
        """test that exception is raised for invalid emails"""
        with pytest.raises(Exception):
>           assert check_email_format("good@email.com")
E           Failed: DID NOT RAISE <class 'Exception'>

login.py:16: Failed
Enter fullscreen mode Exit fullscreen mode

Notice it says Failed: DID NOT RAISE <class 'Exception'>. If an exception is not raised, the test fails. I found this to be pretty awesome. We passed in a valid email format (according to our standards here) so the test works as expected. Now we can make it pass.

import re
import pytest

def check_email_format(email):
    """check that the entered email format is correct"""
    if not re.match(r"(^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$)", email):
        raise Exception("Invalid email format")
    else:
        return "Email format is ok"

def test_email_exception():
    """test that exception is raised for invalid emails"""
    with pytest.raises(Exception):
        assert check_email_format("bademail.com") # invalid email format to raise exception
Enter fullscreen mode Exit fullscreen mode

Run your test: pytest login.py:

collected 1 item                         

login.py .              [100%]

====================== 1 passed in 0.05 seconds ======================
Enter fullscreen mode Exit fullscreen mode

You can also add an extra check for the exception message:

import re
import pytest

def check_email_format(email):
    """check that the entered email format is correct"""
    if not re.match(r"(^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$)", email):
        raise Exception("Invalid email format")
    else:
        return "Email format is ok"

def test_email_exception():
    """test that exception is raised for invalid emails"""
    with pytest.raises(Exception) as e:
        assert check_email_format("bademail.com")
    assert str(e.value) == "Invalid email format"
Enter fullscreen mode Exit fullscreen mode

gif

Top comments (7)

Collapse
 
rhymes profile image
rhymes • • Edited

I know it's just an example but I think I should mention it for beginners: there are not many cases when raising Exception is the best strategy. The reason being that Exception is the class from which most error classes inherit from.

>>> Exception.__mro__
(<class 'Exception'>, <class 'BaseException'>, <class 'object'>)
>>> ValueError.__mro__
(<class 'ValueError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>)
Enter fullscreen mode Exit fullscreen mode

It would make it harder for the caller to properly handle different types of exceptions.

In your case ValueError (or a custom exception) is probably more appropriate:

Raised when an operation or function receives an argument that has the right type but an inappropriate value

A bonus tip: pytest.raises accepts an argument that you might find useful: match. You could rewrite:

 with pytest.raises(Exception) as e:
        assert check_email_format("bademail.com")
    assert str(e) == "Invalid email format"
Enter fullscreen mode Exit fullscreen mode

as

with pytest.raises(Exception, match="Invalid email format"):
        assert check_email_format("bademail.com")
Enter fullscreen mode Exit fullscreen mode

match is used to check the error message with a regular expression.

Hope this helps!

Collapse
 
abadger profile image
Toshio Kuratomi •

I just wanted to correct a common mistake in this comment since it was one of the first results from my google search. message is actually used for setting the message that pytest.rasies will display on failure. It's not about a comparison to the exception's message. match should always be used for that purpose. docs.pytest.org/en/latest/referenc...

(This problem catches almost everyone at one point or another. Which is the reason that pytest has chosen to deprecate the message parameter ;-)

Collapse
 
rhymes profile image
rhymes •

thank you Toshio! I'll update the comment!

What did you search on Google to get here?

Collapse
 
wangonya profile image
Kelvin Wangonya •

Yeah, it helps! Thanks for the input. It was just an example, of course in an ideal situation things would be different. I should have mentioned that.

Thanks again!

Collapse
 
hhmahmood profile image
hhmahmood •

Hi have tried the same solution but still having the problem.

def get_param(param)

if param is None:
raise ValueError('param is not set')

def test_param():
with pytest.raises(ValueError) as e:
get_param()

The problem is that when function does not raise exception, test_param() gets fail with the following error.

Failed: DID NOT RAISE
It works as expected when get_param(param) function throws exception.

Thanks in advance :-)

Collapse
 
wangonya profile image
Kelvin Wangonya •

That's the expected behaviour. The test is checking that an exception was raised, so if that doesn't happen, the tests fails.

Collapse
 
aderchox profile image
aderchox • • Edited

What does this mean?
"But I also couldn't merge if coverage had decreased by even 1%.".
Would you please explain it a bit more?

Read next

allanpablo profile image

Resilience and Failure Management in DevOps

Allan Pablo -

dev3l profile image

PyStructures

Justin L Beall -

furkangozukara profile image

Finding Best Training Hyper Parameters / Configuration Is Neither Cheap Nor Easy

Furkan Gözükara -

wewake-dev profile image

Solving Coding Problems using the Inclusion-Exclusion Principle and its variations

Wewake -