Visual regression testing with Backstop JS

What is Visual regression testing

A visual regression test checks what the user will see after any code changes have been executed by comparing screenshots taken before and after code changes.

Why Backstop JS

Consider that, we have a web app with 50+ scenarios/screens and it is supported in multiple devices like mobiles, tablet and desktops. Every time, UX change is done, it is time consuming to test all scenarios in all supported devices.

Backstop JS helps front end developers to automate the visual regression testing.

• Provides an easy way to compare current change against Reference(old screenshot)
• Supports multiple viewports
• Multiple scenario based configurations for testing
• Reduce lot of effort in manual testing

Let’s dive into configuring backstop js with basic react app

npx create-react-app visual-regression-testing
cd visual-regression-testing
npm start

This will open basic react app in http://localhost:3000/

Replace the content in App.js with some heading in h1 tag

function App() {
  return (
    <div className="App">
      <div className="login-form">
       <h1>Backstop visual regression testing...</h1>
       <p>By Madasamy Ponraj</p>
      </div>
    </div>
  );
}

In root folder, install backstop js using below command.

npm i backstopjs

Execute below command to generate backstop configuration files.

backstop init

Open backstop.json and change the url under scenarios to http://localhost:3000/

Run backstop test to generate initial reference files.

backstop test

You will be seeing Unexpected error image inside backstop_data/bitmaps_reference folder. This is happening as our app is not running. We will resolve this using start-server-test dependency.

npm i --save-dev start-server-and-test

Add below scripts to package.json scripts section

"healthcheck": "npm run backstop:test",
"visual-test": "backstop test",
"backstop:approve": "backstop approve",
"backstop:test": "start-server-and-test start http://localhost:3000 visual-test"

Run below command. Two tabs will be opened one with preview and another one with reference screenshots.

npm run backstop:test

To approve these initial reference screenshot.

npm run backstop:approve

If you execute npm run backstop:test once again, you can see that command executed with out any failures, as references are approved. Lets add one more p tag below our h1 tag inside our App.js file to see visual comparison.

<p>By Madasamy Ponraj</p>

Run npm run backstop:test now. You can see the report generated like below. If the changes are expected, we can approve it or we can investigate the issues.

If you want to add backstop testing as part of health check along with unit tests and lint, we can use healthcheck script.

Please download the codebase from here visual-regression-testing, if you want to try.