Good tests, Bad tests

This article originally appeared on my blog

Unit testing, as with any practice, can be taken to the absurd. I have already written about it.

The truth is that writing good tests is as hard as writing good production code. It takes practice and it requires careful thought, but so many times I have seen how tests are treated as second class citizens, which usually means that we do not reap the whole benefits of a good test suite, ending up with suites that are more of a burden than an asset.

A good test suite:

1. Gives you confidence.
2. Finds bugs.
3. Is cheap to maintain.
4. Document the system.
5. Robustness.
6. Does not have bugs.
7. Is run.

Confidence

After making a change, is running your test enough to commit the code? And to deploy to production?

My experience tells me that unit tests are not enough and some kind of full stack, BDD and integration tests are needed to raise the level of confidence.

Note that if two tests give you confidence in the same area/functionality, one of them is redundant.

Also, if you are really confident about your tests, but you are still shipping bugs to production, your confidence is misplaced.

Find bugs

If you have a test suite that never goes red, your tests are giving you no value as they are not catching any bugs, which is their main reason for being.

Cheap to maintain

If when you go to refactor you code, you find yourself expending any significant amount of time either fixing your tests or rewriting them, your test suite is not paying off. I have found this is especially true if your unit is the class and if you are using a lot of mock objects - in most cases you end up with overspecified software.

Document

Tests should tell us what the system does and ideally why it does it.

A minimum set of tests should also tell us how it does it, but with the expectation that those tests will be thrown away when we make a sizable refactoring.

No bugs

The best way to know that a test is testing what it is supposed to test, i.e. that the test has no bugs, is to see the test change from red to green when you make a change in your production code.

This is why you should write the test before the production code.

Also consider mutation testing to find the blind spots on your suite.

Robust

If your test suite has a flaky test, one that randomly goes red, it will undermine the whole test suite as people stop caring about it.

It is better to remove the flaky tests than to keep them around.

Run

Maybe this sounds a little bit silly, but I have seen and written a lot of tests that did not invite to be run. These included:

• Commented out or @Ignored tests.
• Very slow tests.
• Tests that require some manual step like installing a DB, an application container or a message broker.
• Tests that cannot be run or debugged from your IDE.

In summary, a good test suite will be the pillar that allow us to evolve and change our systems.

Tests must support change, not inhibit it.

At the end of the day, as Dave Thomas and Andy Hunt wisely point out, we are always changing an existing sytem.

All programming is maintenance programming, because you are rarely writing original code.

It's only the first 10 minutes that the code's original, when you type it in the first time. That's it.