DEV Community

Cover image for The Myth of Code Coverage

The Myth of Code Coverage

Matt Eland on November 09, 2019

In this article I’ll discuss what Code Coverage is and its usefulness and limitations. I’ll advocate for a risk-aware approach to software quality ...
Collapse
 
luigledr profile image
Luiz Alberto

One way to avoid some of the pitfalls you'd presented here is using mutation tests technique, it's a very cost way of guarantee more code quality to our projects, but done correctly, it could be very effective.

Collapse
 
integerman profile image
Matt Eland

This is a huge point. I love it. Yes, you can automate some of these tests, particularly with enums. I picked a few readily available examples from my own hobby code to discuss.

Fuzz testing, test data generators, and looping over all possible values won't find everything, but it's something to strongly consider.

Collapse
 
aminmansuri profile image
hidden_dude

I think of code coverage like I think of tests:

The fact that I have a ton of tests that pass doesn't mean that I don't have bugs. But if a test doesn't pass, then I know I do have a bug.

Likewise, the fact that I have high coverage doesn't mean I have good tests. But if I have very low coverage, then I know I don't have enough tests.

Collapse
 
aleksikauppila profile image
Aleksi Kauppila

Even if some of the tests don’t verify anything valuable, it’s good to have some test exercise it before it goes to production.

So if i’m seeing a number like 60% code coverage. I know that 40% is either never executed or we just hope it doesn’t break our app.

Collapse
 
integerman profile image
Matt Eland

I'm glad we agree!

Collapse
 
integerman profile image
Matt Eland

Definitely agree, with the caveat that you have some other plan for verifying your code if you're not using a test.