I know what are you thinking, "why can anyone say that we don't need test automation?" and you're right! But, we need to also consider the context, the most of articles that I have read about "why you should automate your test" always talk about the benefits like reduce effort on repetitive tests, greater consistency to our testing approaches or easier access to info (results) about testing. The thing is, we are not always working on a well defined processes, for example we are a typical company that have our testers and had heard about the Automated Testing process and benefits so, the company, want to apply this to their daily activities. Here is when we should review if we should or not add the automated test activity to our business.
A common error that we get when we want to add the automated testing approach to our system is thinking that this will be the key solution to improve the Quality of our product, and that is not the case, the automated test are expensive to maintain, and will not cover the whole testing process for the products, usually, the automated tests cover only the happy path on a system, let's say that we have a product that sell clothes online, you will create an automated test that start from login, to adding clothes to your cart and will end with placing an order (in a lower environment of course, we don't want to add fake order in production and mess up our analytics). Now, what happen if, in the login page, the user type the wrong credentials, what if the clothe size is out of stock, what if the user set a wrong address for the delivery, all of this scenarios should not being automated, I mean you can do it, but if we decide to change the flow or, even, we make a small change on the login button selector, we must update all of our test so they can start working properly again.
Another thing to consider when you want to add automated test in a product, is that we usually tend to underestimate the time, cost and effort on the tool that we want to use for automating (Selenium, Cypress, TestCafe, etc.), this means that it cost a lot to add the tool and we will start seeing the benefits of it in several months, it will not give us value since the beginning of the implementation.
To finish this, we should keep in mind that Automating doesn't mean that we will replace other testing processes like Unit Testing, Integration Testing or Manual Testing. This will help us to improve some of the testing activities, more oriented (in my personal opinion) on the pipeline flow to confirm that the happy path works and we are good to deploy on prod without any breaking change. What do you think about it? You can leave your opinion in the comment section :D
Top comments (2)
If your automated tests only cover the "happy path", then they're not very good tests.
I agree that you can't rely solely on automated tests. And it seems you understand that the purpose of automated testing is to free up QA resources to work on more advanced testing scenarios. However, automated testing of certain types of code makes more sense because it is tedious and prone to human error. Once you have validated a spec for an automated test, it will run exactly the same way every single time. I've yet to meet a QA engineer who's a robot :)
As you say, unit tests are prime example of what is to be automated. But how far can it go? I'd argue that it should go as far as it possibly can. It's possible to validate all the invariants within the code automatically, and you should. This is perhaps the biggest source of bugs that I can think of: an object in an invalid state that gets persisted, and so on down the line.
I'd also argue for automated tests as much as possible because then developers submit better code. Just as even the most methodical QA engineers may miss something, so much more so do developers. How much time is wasted in QA because you have to send something back nearly right away? How many times does this repeat? If developers are forced to write good tests (or better yet, someone writes good tests for them), then they will submit better work, and QA time is better spent.
Well-written applications should have very little application logic in the presentation layer. And using modern architectures like the so-called "Onion" architecture, you can end up with a highly testable, rich domain model that encapsulates the majority of the business logic. Testing this is both fast and easy. So, put as much in your core classes as you can without taking on dependencies, and write automated tests for full code coverage of this model. Now your manual QA focus can be on the actual application itself, and not the irritating little bugs that affect so many things.
Hello Brian! Yeah you’re right, this post is oriented to e2e automated testing that they’re really expensive to maintain and develop, I totally agree with you, the automation for unit and integration tests should be in a high percentage on each project of course depending on the team QA approach, I really appreciate your comments and clarification!