Multiple asserts are bad and that's the hill I'll die on

Photo by Jørgen Håland on Unsplash

Context

When writing an automated test, sometimes multiple things happen and sometimes we want to check (i.e. assert) all of them and sometimes we want to do that in one test.

Like this:

Don't do it!

Reasons:

1. Readability is lower: the assertion is the center of the test (thus should always be written first but that's another article) and having more than one disorients the reader
2. The test framework will stop on first failure, meaning the failure report will be partial - you'll never know which output was incorrect and which correct, except the first incorrect one (and all the correct ones before it, provided you know by heart which ones went before)
3. The test name is either weirdly composed of multiple parts which makes it long and hard to follow (like in the Gist) or describes just part of the scenario, i.e. it is lying

How to do it differently?

Method A: Assertion Structure

In a simple-enough case, when all the asserted data is in-fact closely related (domain-wise), we can lump the actual data in one structure, the expected data in another (of the same type) and assert the structures' equality.

Like this:

For simple matters, arrays (or any collection in other languages, e.g. Lists in Java/Kotlin) will suffice.

For more complex stuff and much greater readability use custom data objects:

The increased readability is apparent in code, but much more so in the test failure report!

Method B: Multiple Tests

When the data isn't that much related to each other domain-wise, you should have multiple tests with one assert each:

The duplication of arrange (given) and act (when) steps can be solved in a number of ways, but can also be left duplicated - these are tests after all and DRY holds much weaker here: readability is much more important.

In the end...

When you see a test with multiple asserts, or are starting to make one, ask yourself:

What is the point of this test?

Then try to see which of the above methods should be applied.

But what if we use...

...a soft assertion library (which executes all multiple asserts and the failure report displays all results)?

That works, but I dislike those because:

1. They usually require the developer to finally call .assertAllOfUs() in the end and this can be forgotten which results in the most evil thing there is: A TEST THAT NEVER FAILS 💣
2. I have to learn how yet another library and it's API work 🙄

Next-morning Addition

A thing one remembers when walking one's dog in the morning: AssertK has the "soft" assertions solved with their multiple assertions feature, which avoids the bad point 1. but still kind of leaves point 2. Also, allows people to complicate test code, especially when using multiple assertions within a multiple assertion. 🤯

Also the lib is for Kotlin only ... but everyone should be writing Kotlin anyway. 😛