Focusing on high code coverage can be a trap

Code coverage is a good negative indicator but a bad positive one.

In the following lines I will try to explain why this affirmative is true. You don't need to stop care about code coverage of your project, but understand how to read code coverage is important.

Code coverage or test coverage is a coverage metric that shows the ratio of the number of code lines executed by at least one test and the total number of lines in the code base.

Lets suppose we want to know if a number is even, we can do it with the following code in Python:

def is_even(num):
    if (num % 2) == 0:
        return True
    return False

The code above works well, and now we want to test it, we can do it with the following code:

def test_is_even():
    assert is_even(4) is True

It's easy to calculate the code coverage, our assert is just testing the True case, this gives us 3/4 = 0.75 = 75% code coverage.

Name           Stmts   Miss  Cover
----------------------------------
main.py            4      1    75%

But, what if we refactor this code and eliminate the if statement?

def is_even(num):
    return (num % 2) == 0

Does the code coverage change? Yes, it does. Now we have executed all lines of code and achieved 100% code coverage:

Name           Stmts   Miss  Cover
----------------------------------
main.py            2      0   100%

But did we improved the tests with this refactoring? No, we still don't test the False outcome, we just compacted the code, the more compact your code is, the higher the test coverage metric becomes.

Now let's do an assertion-free testing, which means write a test that doesn't have any assertion:

def test_is_even():
    result = is_even(4)

What do you think happened to out code coverage? We still have 100% code coverage.

Name           Stmts   Miss  Cover
----------------------------------
main.py            2      0   100%

This test show us 100% code coverage, and it is useless because it doesn’t check anything, it just enforce that the lines of code are being executed.

Aiming at a particular coverage number is not the best way to work with tests, measuring the code coverage is just the first step on the way to
have a quality test suite.

I think these examples are enough to show you why code coverage is a good negative indicator but a bad positive one. If you have 10% code coverage you know that you are not testing your codebase, but have 100% code coverage doesn't mean you have good tests.

Make sure to test every case and don't stop to create tests when you achieve the 100%, the examples above can show you that you still have tests to write when even when you have 100% code coverage.

References:

• Unit Testing Principles, Practices, and Patterns