Playwright is an excellent tool for verifying that your app’s most important user flows work as expected. But did you know that Playwright can also help you verify that those flows look as expected too? This process is called visual regression testing, or visual testing for short. And it’s super helpful for verifying all kinds of UI details: responsive design shifts, cross-browser/device differences, localization, dynamic content, and more.
In this post, we’re going to walk through how to run visual regression tests using Playwright. We’ll add visual testing to an example E2E test and walk through the typical workflow when changes occur. For this exercise, we’ll assume that you have Playwright configured and running at least one E2E test already.
How does visual testing work?
Playwright’s visual testing revolves around a process of screenshot comparison. When the test begins, Playwright takes a screenshot of whatever is being tested and stores this as the baseline. Then, whenever you change your code, Playwright takes another screenshot and compares it against the baseline.
If the changes look correct, you can accept them and then set the new baseline. Otherwise, you can continue updating your code and accept it when you’re happy with the results.
Create screenshots
To begin visual testing in Playwright, let’s start with an example E2E test which opens a dialog and then clicks the close button within it:
// tests/dashboard.spec.ts
import { test, expect } from "@playwright/test";
test("Dashboard", async ({ page }) => {
await page.goto("/dashboard/acme");
await expect(page).toHaveTitle(/Acme Dashboard/);
const expandButton = await page.locator(
".main .card:nth-child(0) .btn-expand"
);
await expandButton.click();
const dialog = await page.locator(".dialog");
const closeButton = await dialog.locator(".btn-close");
await closeButton.click();
});
Playwright provides the page.screenshot API to take screenshots of the page you’re testing. To use it, add a line that calls the function at the part of the flow where a screenshot should be taken:
// tests/dashboard.spec.ts
import { test, expect } from "@playwright/test";
test("Dashboard", async ({ page }) => {
await page.goto("/dashboard/acme");
await expect(page).toHaveTitle(/Acme Dashboard/);
const expandButton = await page.locator(
".main .card:nth-child(0) .btn-expand"
);
await expandButton.click();
const dialog = await page.locator(".dialog");
// 👇 Take a screenshot once the dialog is located
page.screenshot({ path: 'latencyExpanded.png' });
const closeButton = await dialog.locator(".btn-close");
await closeButton.click();
});
page.screenshot has a couple of options that may be useful to you. You can capture the full page height (instead of just the viewport height) and you can configure where the screenshot is saved:
await page.screenshot({
fullPage: true, // Capture full page height
path: 'screenshot.png', // Provide save location
});
You can also screenshot a particular element instead of the entire page:
await page.locator('.dialog').screenshot(...);
Once you’ve updated your test, run your test command to save a screenshot:
yarn playwright test
In our example above, the screenshot will save in the root of the project:
Open the screenshot to confirm that it looks correct:
Now, commit both your code change and the newly-created screenshot. Then push your commit, making the screenshot your baseline.
Run visual tests
Next, we’ll use that baseline screenshot to verify a change.
First, create a new branch and make a code change. For this example, we’ll change our primary chart color:
- fontFamily: 'system-ui, san-serif',
+ fontFamily: 'Inter, system-ui, san-serif',
After making the change, run your test command again to save a new screenshot. Once more, open the screenshot to confirm it looks correct.
Commit your changes and make a PR for your new branch.
Now, when we review that PR, we can compare the previous baseline screenshot and the modified one:
Automate tests in CI
In the workflow so far, we’ve created the screenshots on a local machine and then committed them to your repo. This can cause problems if you're working with other developers on the same E2E tests, as device differences like installed fonts could cause unnecessary image differences.
To mitigate this, take your screenshots within your continuous integration (CI) environment, and let your CI job handle the screenshot and committing process for you. Then, you can review the results as part of the PR review, as before.
Here’s an example using GitHub Actions:
# .github/workflows/e2e.yml
name: E2E tests
on: push
permissions:
contents: write
jobs:
E2E:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-node@v3
with:
node-version: '16'
- uses: actions/checkout@v3
- name: Install dependencies
run: yarn
- name: Install Playwright Browsers
run: yarn playwright install --with-deps
- name: Run tests
run: yarn playwright test
- name: Update E2E screenshots
run: |
git config --global user.name 'Your Name'
git config --global user.email 'you@example.com'
git commit -am "Update screenshots"
git push
The screenshots are still part of your repo (which can take up a lot of space if you have many tests), but now they’re created in a shared, consistent environment, eliminating any device differences.
Debugging an issue can be difficult in this flow. A static screenshot often doesn’t provide enough information, and debugging the real page requires navigating through a staging environment and recreating any interactions that took place.
Reviewing and collaborating with non-developers can also be tricky, because it all happens within the PR experience, which many are unfamiliar with.
Debuggable snapshots and collaborative review
You can take this workflow even further when you integrate Playwright with Chromatic, a cloud-based visual testing platform.
Here’s how to use it in our example above:
// tests/dashboard.spec.ts
// ➖ Remove this line
// import { test, expect } from '@playwright/test';
// ➕ Add this line
import { test, expect, takeArchive } from "@chromaui/test-archiver";
test("Dashboard", async ({ page }) => {
await page.goto("/dashboard/acme");
await expect(page).toHaveTitle(/Acme Dashboard/);
const expandButton = await page.locator(
".main .card:nth-child(0) .btn-expand"
);
await expandButton.click();
const dialog = await page.locator(".dialog");
// 👇 Take a screenshot once the dialog is located
// ➖ Remove this line
// page.screenshot();
// ➕ Add this line
await takeArchive(page, testInfo);
const closeButton = await dialog.locator(".btn-close");
await closeButton.click();
});
And then the CI job (using GitHub Actions here) would be:
# .github/workflows/e2e.yml
name: E2E tests
on: push
jobs:
E2E:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Install dependencies
run: yarn
- name: Install Playwright Browsers
run: yarn playwright install --with-deps
# 👇 Run your E2E tests *before* running Chromatic for your E2E test archives
- name: Run tests
run: yarn playwright test
# 👇 Run Chromatic for your E2E test archives
- name: Publish E2E Archives to Chromatic
uses: chromaui/action@v1
with:
projectToken: ${{ secrets.CHROMATIC_ARCHIVE_PROJECT_TOKEN }}
buildScriptName: build-archive-storybook
Now, instead of creating a static screenshot, running your test will create a fully debuggable snapshot of the page that you can open in the browser. Those snapshots are created in a dedicated, specialized environment—making the process fast and consistent—and saved in Chromatic’s database instead of cluttering your project’s repo. Once in Chromatic, snapshots are easily shared with all of your project's stakeholders and contributors. And there’s a web app purpose-built for reviewing visual tests, which integrates with your git provider.
Right now, Chromatic's Playwright E2E testing integration is available to try in early access, with free usage during the beta period.
Top comments (2)
Thanks for sharing! Super cool to see Playwright enable this 🚀
❤️ this. Great walkthrough, folks