Don’t prioritize speed for e2e testing but consider smarter test schedules

End-to-end(e2e) testing such as by Cypress and Playwright is considered slow compared to unit/integrated testings such as by Jest.

In this post, however, I would like to argue that a few devOps practices can dodge this slow performance issue in the devOps cycle and the developers can take fully advantage of e2e based quality assurance.

Why is the slow performance considered an issue, by the way?

Testing performance is usually considered significant because developers want to include it in the CI/CD cycle, which should get completed as fast as possible.

It usually takes more than 30 secs to execute one e2e testing for one web page. And thus, it would take more than 5 mins when e2e testing covers more than 10 pages.

Common CI/CD pipeline

1. Make a pull request on a feature branch
2. Build/compile the application
3. Run tests
4. Merge the new branch to the main branch
5. Deploy the new application to servers

Due to the popularity of the above CI/CD pattern, front-end developers struggle to include e2e testing in Step 2. However, it means that the testing tightly binds with its development & deployment flow.

Problems

1. Every new push to the remote repository runs entire new tests. Too much waste of computing resources.
2. New development/refactoring/configuration easily breaks some of the tests. And we want to fix broken tests at a different time.

Run entire tests only in specific situations

Include static test (eslint/prettier) and unit test (Jest) in Step 2, but not entire e2e tests.

Of course, it is necessary to run prepared e2e tests before releasing the application to the production environment. But Git tags usually come into play for controlling a production release these days. In fact, the Github action supports tag based actions. Thus, we can separate the CI/CD phase and the production release phase.

Run e2e tests for only affected parts

We can search modified files by git commands on a feature branch and run e2e tests to these files only on push to the remote repository.

So, for example, if a developer modified pageA.js, pageB.js and pageC.js, the CI/CD pipeline runs pageA.jstest.js, pageB.jstest.js and pageC.jstest.js

git ls-files | -I % /bin/bash -c '<test-command> %test.js;'

Or you can manually run tests regarding the affected pages and upload the test results to the pull request description as a part of the internal regulations.

Run all tests in parallel

Most e2e automation tools such as Cypress and Playwright support running tests in parallel.

Each test should not be too lengthly because parallel processing mechanism is file-based. If the test file length is more than 200 lines, it would be wise to split the test file into two.