I hope that most of us use unit tests in day to day development because it saves a lot of time for us to make something new instead of repeat the same mistakes again. In this article, I'll talk about our approach to deal with visual tests
All visual tests* based on the same approach.
- You should run the server with your application.
- You should write tests which run by NodeJS
- As a glue between our application and tests, we use Puppeteer or Playwright
- Expectation code is clean and straightforward
const image = await page.screenshot();
- Test code and code under tests are too far from each other In the test code, we usually call the driver to open some page where located code under test. In the best cases, such pages are somehow automatically generated. This led to notable switch between code under test and test itself that lead to frustration
// Average visual regression test code
const page = await browser.newPage(); // create a tab/page
await page.goto('https://localhost:3000'); // navigate to URL where located the code which we want to test
- Complicated setup and teardown. To run tests, we usually should set up two servers. One for code under test and another for tests itself
- Usually such tests are slow due to complex setup and communication between test and code
All these problems and consequences prevent to write visual tests with pleasure. But visual problems haven't gone anywhere
I always dreamed to have something similar to unit test experience. Where you can easily add or remove test. Play with code under tests by tweaking different options and observe the result. Focus on specific test and enjoy fast feedback loop between change and result
render(<UserProfile/>) // code under test
// test code
const image = await page.screenshot(); // Take a screenshot
expect(image).toMatchImageSnapshot(); // Test with reference
In the example above, our test code and what we want to test placed near each other. So by removing test, we're also removing code under test. We can experiment with test and code under test. Because it looks and behave as a regular unit test
To make it possible, we take Puppeteer or Playwright as launcher for our tests where we expose
screenshot functionality and possibility to compare screenshot with reference image on the disk for our test code through expose function API.
We liked matcher functionality provided by jest-image-snapshot, so we just took this solution and adopted for jasmine and Karma.
The result is the boom of visual unit tests in our product. Because now write visual unit test as easy as write plain unit test
We love this approach because it already brings benefits of visual tests. And it does not require much effort. We pack it as a NPM package for karma, but what we love is the idea of having test code and code under test together, so you can look at our approach and maybe bring a more powerful tool. The repository itself contains tests, so you can open it via Gitpod or GitHub Codespace and play with it without additional setup.
- - When we're building these tool there no cypress which such functionality but now as I know it provide something similar, and this is cool. Maybe in the future we would migrate to it, but for now we are happy with our current approach
An innate problem of visual test is a drawing font on different platforms. For us, we decided that in visual tests we want to test only visual part and text/font is more logical part, so we just create a custom font for visual tests where all symbols just black squares which works on all required platforms for us. So the developer can easily write a visual test on macOS and make a reference, and it would work for Linux.
Currently, we use only Puppeteer/Chrome to simplify the process of installation, reduce headache with different browsers and improve developer experience to encourage writing visual unit tests. It works for us because it already brings benefits. But solution support use of Playwright(Chromium/Edge, Safari, Firefox) so in the future if all would work fine and robust we are planing to run our tests on different browsers