DEV Community

Cover image for Signs That Your Unit Tests Are Not Helpful
megazear7
megazear7

Posted on

Signs That Your Unit Tests Are Not Helpful

Here are two metrics that you can look at with your unit tests:

  1. Percentage of test failures that are only a problem with the unit test, not a problem with the code running in a real environment. This often happens when you are mocking interfaces.
  2. Percentage of times that a bug fix or otherwise non breaking change requires an update to a unit test.

The goal would be to have both these percentages at 0%. This would indicate that every test failure represents a real issue in your code and that only feature changes require updates to the unit tests.

As these percentages approach 100% the less helpful your unit tests become. However what percentage should act as the cut for when your unit tests are a net positive verse when they are a net negative? We can make two assumptions:

  1. Every test failure that reflects a real code issue exactly out weighs every false positive.
  2. Every unit test update that comes from a non feature change is out weighed by the benefits of having a unit test that comes from a feature change.

With these two assumptions in place then 50% for each of the metrics is the cut off for when your unit tests become a net negative. These are broad assumptions that might not always be true, but would need determined on a project by project basis based upon the technology that is in place. However it does provide a framework for thinking about the utility of unit testing.

Check out my blog for more of my musings upon technology and various other topics.

Discussion (0)