How Manual Testing is Still Important Even with Automated Tests

So far, we’ve looked at three of the Testing Pyramid’s four layers.

Tests in the first two layers are coupled closely with production code. With unit tests, we check the functionality of individual components and processes. Integration tests verify that chains of these units can work together to produce expected results. In both cases, we replace some or all dependencies with test-substitutes. In doing so, we concentrate on the subject under test; external factors such as logic from other modules can’t influence the results.

In end-to-end tests we take a subtly different approach. We write tests in ways similar to how end-users might use our product. The test code becomes independent from any internal workings, and this ensures that our tests don’t accidentally use APIs that normally aren’t accessible to third parties. In addition to verifying our logic, end-to-end tests also help to detect problems in the release process: if there are issues in packaging or deployment, our tests would most likely fail.

The number of components involved in individual tests typically increases as we move along the scale from unit to end-to-end tests. The inclusion of more systems means we can check things work at different levels, but also potentially leads to longer running times. As such, it’s often beneficial to have more unit than end-to-end tests. While unit tests are less encompassing, they take little time to run and are highly targeted. With enough unit tests, it’s possible to cover many scenarios quickly and find smaller problems before they become bigger ones.

The Final Part of the Pyramid

The final section of the Testing Pyramid involves manual testing. This is an important part of the process, even when we have automated tests. In many businesses, it’s rare for product requirements to come from software development teams; manual testing gives everyone involved a chance to experience the software as an end-user would.

QA teams are often very good at finding ways that we, as software authors, might not have imagined the product to be used. If you’ve not seen this video before, it offers a funny take on this concept.

Designing a product often requires testing ideas and refining them. Product teams can check both that their ideas work, and that they’ve been translated accurately between the design and implementation stages. Miscommunications during the development cycle are possible, and manual testing offers an opportunity to verify that everyone’s expectations were aligned.

In addition to the benefits of having someone else try the software, manual testing is important for several reasons:

• Tests can be difficult or impossible to write when inflexible legacy code or certain third-party libraries are involved. If mockable APIs aren’t available, writing a test could involve adding disproportionate amounts of complexity to production code. In these cases, it might be worth keeping it simple and testing it manually, especially if the code is not changed often.

• Null-reference exceptions are easy to miss. When configuring unit/integration test substitutes, we can forget that some properties might be null at runtime.

• Parallel computation and concurrency issues are extremely difficult to detect via automated tests.

• Edge cases are easy to miss because of their unconventional nature.

• It’s virtually impossible to exhaustively list every potential use case.

• There typically isn’t enough time to write a test for every use case.

It helps to have a plan when testing software manually. We can use this to map out important things that need testing in advance, potentially including scenarios that are difficult to test using only code. Following the plan helps to avoid accidentally leaving things untested. If any bugs are found during testing, it’s good practice to write automated tests for them where possible. This helps to prevent regressions from occurring in future.

Summary

Manual testing is the final piece of the Testing Pyramid, and it has advantages even when automated tests are available. It’s an important part of the process, as it lets you experience the product as an end-user. When working in a team, it’s helpful to get others involved: QA engineers often test in ways that you as a software developer might not think of, and Product teams can verify and refine their ideas.

But manual testing is useful even if you’re working alone. There typically isn’t enough time to write an automated test for every single use case, even if you could list them all. Un-mockable APIs and concurrency issues are challenging, and edge cases are easy to miss simply because of their nature. If you come across any of these, it’s a good idea to put them in a test plan along with any other scenarios that need testing so that nothing is forgotten.

But the good news is that manual testing is just one aspect of the process. You have a whole suite to automated tests to help you with other scenarios, and to complement your efforts. And if you find a bug in manual testing that lends itself to automated testing, you can write a test for it. That way, you shouldn’t have to worry about it again.

Thanks for reading!

Software development evolves quickly. But we can all help each other learn. By sharing, we all grow together.

I write stories, tips, and insights as an Angular/C# developer.

If you’d like to never miss an issue, you’re invited to subscribe for free for more articles like this, delivered straight to your inbox (link goes to Substack).