Test Automation Tips and Recommendations

I have been doing test automation for quite some time now. From ground-up projects to established companies, regardless of the tools used, there seem to be common challenges in projects with test automation.

Here are some recommended practices that I followed along the way:

1. Know the long-term goals of your test automation and let others know about them too.

Test automation can either be underappreciated or have high expectations from project management. Set the expectation that not everything is automatable. Emphasize that test automation is not an all-in-one solution, but it will definitely help us:

• reduce manual efforts on testing in different levels (ST, SIT, end-to-end, regression, etc.)
• provide faster feedback on developers' work
• do faster releases without compromising the quality of the product.

The earlier we report defects, the earlier they will be fixed, and we'll have a higher probability of meeting the deadline. With test automation, we can "test early, test often."

2. Align the Project, the People and your Automation Tools

If you are in a position to decide what tools to use and make your team use them, consider:

• project budget
• capacity of existing team
• skill set of the team
• willingness of people to automate

When choosing the tools, consider:

• if the company is willing to buy licenses or annual fees availability of support (especially if you choose an open source tool. Hope that it won't get abandoned!)
• how fast people can adapt to it (some test automation tools don't need heavy knowledge on programming languages)
• should meet company standards (i.e. for banking projects, you can only use "approved" tools that meet the security standards).

Thinking ahead will save you from big headaches on rework in the future.

3. Understand your company's business

Know what product or value your company gives to its consumers and how they use your application. This helps you determine business-critical scenarios and highlights areas where you need to test heavily.

One of the test principles is "exhaustive testing is impossible". Rather than attempting to test exhaustively, test case prioritization and risk-based testing should be used to focus test efforts. If you want to know about other test principles, visit the ISTQB Official Syllabus, Chapter 1.3

4. Refine your Test Cases

As your product grows, the test case count grows too.
Set a time to revisit existing tests.

• Get feedback from POs and developers to make sure you are covering enough.
• Prioritize ones that bring business value first.
• Remove outdated test cases.
• Avoid duplicates.
• Use the test techniques on ISTQB Official Syllabus Chapter 4

5. Plan your time for test automation

Often times, only development work is considered in sprint planning. Voice out that automation work also has to be considered in your sprint capacity. While development is in progress, you can automate in parallel. If details are not yet final, you can at least prepare the "backbone" of your automated tests.

6. UI Locators:: It is best when you have proper identifiers

Have an agreement with your dev team to always add an id, name or any custom unique identifier in the UI elements. Sometimes developers forget it, and if they don't put ids, it will eat up your time to automate as you need to create UI locators (with xpath/css locators, etc.). If your UI is frequently updated, you have to keep updating your locators too. Always save your time.

7. Practice coding standards

Code review is not just for developers. Testers and automation engineers need it too. We review our code (test scripts) for:

• accuracy
  Check the flow of the test (input-process-output) to see if you are expecting the correct values. Watch out for logical and typographical errors too.

• maintainability
  As much as possible, avoid hard coding. Short-term, you can deliver fast. Long-term, it will take your time to maintain it. Imagine if you hardcoded for 100 test cases and suddenly a product update came and you had to visit those 100 test cases again. What more if you had to update so many files related to it?

• reusability
  A common challenge for new test automation projects is that they are often hard-coded and can only run in one environment. That's sad.
  It is easy to just hard code it, but if you think long-term, hard coding is not very productive. As you mature in test automation, you will appreciate "parameterized" projects.

When you parameterize, you can easily switch your environments with minimal changes to your script.
i.e. you have the ENV parameter. You can simply change ENV=dev to ENV=test to switch environments, fetch environment-specific data and execute your test without updating so many files.

When you parameterize, you make it flexible for future updates.
i.e. Your application has a search page. If your automation is hard coded to do "SearchByFilterA" and a new filter is added, most juniors will just want to add "SearchByFilterB", "SearchByFilterC" etc.
It is often recommended to just make one keyword "Search (FilterValue)". One keyword to maintain, multiple scenarios supported.

Adopt good coding habits; it will help you and your team in the long term.

8. Aim for daily runs or frequent runs

If your scripts are done and merged, make the most of it.
No need to run everything but at least test the business critical ones. Most common automation tools already have "tags" so you can select which specific ones you want to run.

For Jenkins and Gitlab (and others), there is also an option where you can set your scheduled runs. Setting this up will be worth it.

9. Aim to Support Multiple environments or multi-tenancy (if applicable)

For large and complex projects, this is essential. If your test automation project has a bad architecture from the beginning, then you might not be able to reach this point. I remember re-architecting a project from the ground again because they forgot to consider that their application would have to support multiple tenants. This is why I mentioned it on point 3: Understand your company's business. If you know your business well, you can future-proof your test automation for it.

So what did I do in that project to support multiple environments or multi-tenancy? I had to break down the existing project into a more organized repository with more configurable pieces. Test data is sorted and made independent for each environment. Functions and keywords are parameterized. I will share more about this in a different post and link it here once available.

10. Aim for Automated test results

Your automation is now working and running regularly. Let's make it more independent and reliable.

Depending on your tool, you can automatically send test results to TestRail, qTest or wherever you record test results. You can also publish your test results to a Slack channel. Setup for this is specific to the tools you use, here are some Google search terms that can point you to the right place: "Test Integration" "Slack plugin" "Webhook"

Investing time to set this up is worth it. For me, if results are regularly published automatically, then it is one sign that a project has a 'matured' or 'established' test automation.

Good to have: Parallel Runs (for even faster results)

Some tools support parallel runs. If you are after fast feedback, you can try to tailor your project to support this. I personally did it for a Java-based test automation project in the past. I haven't tried it for Python-based ones yet.

---

Most people think that test automation is just about writing tests, but actually, there's a lot of work under the hood. I hope you find value in what I shared here, and I'm wishing you all the best in your test automation projects. If I missed anything, feel free to share your recommendations too.

Happy testing!