I was asked a few months ago on potential ways in reducing test automation run time so I opened my Notion editor ✍🏻 and started compiling a list based on my experience. I've been in companies where tests were running for HOURS! (imagine the long feedback loop and re-work required if something goes wrong! 😱).
This might be handy to Quality Engineers looking for a test automation strategy. Let's start with the quick wins 💨.
✅ Use explicit waits instead of implicit waits - This is a well-known anti-pattern in the test automation community, putting a default waiting time attracts longer run-time and flakiness. Explicit waits such as waiting for an expected element to be visible should be used instead.
✅ Avoid usage of xpath (if possible) - (I understand it is a bit controversial as automation engineers rely on this regularly but hear me out) but xpath identification is generally slower than other direct attributes, so using id or name (if unique) should be the top choices. It is also harder for maintenance as it is not easily visible on the DevTools when troubleshooting issues.
✅ Include unique tag / custom attributes on selectors - Instead of traversing the whole DOM, it is usually easier to have a unique identifier for a specific element such as adding data attribute such as data-test-id on the actual source code.
✅ Run via headless mode - Headless are much faster than real / headed where the main difference is that the screen / browser is hidden. This can be achieved by CLI options (e.g. npx cypress run --headless - most frameworks would have this option).
✅ Use API / libraries to quickly generate test data - Instead of creating test data via the UI, it is significantly faster via API or libraries. Plug-ins such as faker or running API's can be included in the test scripts before any UI functionality is performed.
✅ Bypass the UI if not testing those functionalities - Similar to the above, for example, if we are not testing the login functionality, then there should be multiple ways to bypass this. Although, we can still have a separate UI test for login, the other tests (after login) can be checked without going through the long UI login process.
✅ Set a scheduled run of tests on a higher frequency - This won't reduce the run-time but would be beneficial on continuous health check. A sub-set / smoke tests can be run multiple times in a day or a longer test (e.g. regression) can be run daily overnight via the CI/CD tool.
✅ Parallelisation + Deterministic tests - Parallelisation allows higher test coverage in a shorter run-time. By making the tests "deterministic" (independent of any other tests, data, environment, etc.), it is easier to de-couple tests and run them in parallel. This can be easily configured as most CI/CD tools are capable of running parallel tests.
At the same time, let's take a step back and wear our holistic hat. The following may require longer effort and big mindset shift 🌟.
✅ Choose the tooling & language - Modern automation tools such as Webdriverio, Playwright & Cypress have smarter ways to make the tests run faster and are scripted via NodeJS / Javascript which allows asynchronous calls so it does not need to wait sequentially for each commands.
✅ Clean up the database / environment if possible - If there is a way to easily spin up an environment with minimal dataset or a script can clean up the database, then this will speed up the run-time. Assistance from environment owners and building up automated jobs can help fast-track these activities. Note: If you are worried about performance difference with a Production environment, then it's best practice to use a dedicated Production-like environment for Performance testing.
✅ Built-in documentation and having developers contribute to the code - In the modern testing, even the developers are involved in the test automation. As they would usually know good coding patterns, they can help improve the speed of the tests. Pair programming / testing and continuous collaboration would help build a scalable and robust test automation framework. Include in-line comments help resources to understand the scripts better.
✅ Reduce by increasing unit tests as compared to e2e - Following the test pyramid principles, there should be a significantly higher coverage of unit and integration tests as compared to e2e tests. Both unit and integration tests run significantly faster too as compared to tests that involve the UI. It is ideal to focus on high-value E2E tests that are critical to the customers and stakeholders. Duplicate tests that have been covered in the earlier level of tests should be considered for streamlining the test suite.
Again, this is just on my experience and there might be some better ways so just pick the bits and pieces that fit your need 👋.
Top comments (2)
MAAAAAAAAAAAATE. Noice.
I love all of this and bonus points for mentioning Notion.
I'm often advocating our customers decouple things like their UI from their functional tests, or add unique selectors, and having it all in a list is chef's kiss. I think the biggest challenge for teams where the QA folks aren't empowered to influence design is they miss out on all of these advantages.
It's also weird being a former Ruby developer where a lot of the test admin stuff was taken for granted, and now parts of the JS and SDET community are having to re-discover them.
Thanks and 100% agree. The way Devs should be able to contribute to the test scripts should be the same as the QA's being empowered to contribute to the codebase (e.g. adding unique selectors, etc.). It all boils down to collaboration. It's hard to break the stigma of devs and QA's pushing back and forth with who's going to do what.