End-to-end testing Chrome Extensions (using Playwright)

It’s frustrating for users when an extension regularly crashes, freezes, or fails to function as expected. Negative reviews, product abandonment, and ultimately, revenue losses are all situations that developers want to avoid. Not to mention, the process of fixing unstable extensions can be time-consuming and costly. So, it’s vital to ensure that your extension remains stable and reliable after changes have been made. In this article, we’ll explain how to accomplish this via end-to-end testing.

To ensure your extension works properly, it’s essential to test it as closely as possible to how users will interact with it. To do this, we’ll write end-to-end tests using a tool called Playwright.

Here are a few reasons why it’s the right tool for this task:

• It supports multiple programming languages like JavaScript, TypeScript, and Python.
• It has the ability to intercept and deflect network requests, which is a powerful feature.
• It does a great job of automating tests on Chromium, Firefox, and Webkit browsers (but for the sake of brevity, we will only cover Chrome extension testing).

Getting set up

The first step in the process is to set up your testing environment. So, to add Playwright to your project run:

npm init playwright@latest

or see more installation options.

The installer will ask you:

• For a location to put tests
• To add a GitHub Actions workflow
• To install browsers for running tests
• After installation, you’ll see example specs and a Playwright config.

Since we’ll only be testing Chrome extensions here, we can remove Firefox and Webkit in Playwright config:

...
projects: [{
  name: "chromium",     
  use: {            
    ...devices["Desktop Chrome"],   
  },    
}],
...

Once you’ve set up Playwright, you’ll be able to launch example tests using the command npx playwright test.

Add extension to Playwright configuration

To test your extension, you’ll need to create a browser context with the extension enabled. This can be done using a helper function that launches a persistent context in Chrome. Here’s an example of what the code might look like:

import { chromium } from "@playwright/test";
import path from "path";

...

export const createBrowserContext = async () => {
  // assuming your extension is built to the 'public' directory 
  const pathToExtension = path.join(__dirname, './public')  
  const userDataDir = '/tmp/test-user-data-dir' 
  const browserContext = await chromium.launchPersistentContext(        
    userDataDir,    
    {   
      headless: false,  
      args: [`--disable-extensions-except=${pathToExtension}`], 
      ignoreDefaultArgs: ['--disable-component-extensions-with-background-pages'],  
    }
)}

Note: By default, Chrome’s headless mode in Playwright does not support Chrome extensions. To overcome this limitation, you can use a workaround like this:

const browserContext = await chromium.launchPersistentContext(
  userDataDir,  
  {     
    headless: false,    
    args: [     
      `--disable-extensions-except=${pathToExtension}`,     
      '--headless=chromium' 
    ],  
    ignoreDefaultArgs: ['--disable-component-extensions-with-background-pages'],
  }
)

For more detail and alternatives around testing Chrome extensions, check out this guide in Playwright docs.

Writing tests

Now, it’s time to write the test. Here’s an example of what the finished product might look like:

import { test, expect } from '@playwright/test'

test.describe('Test some behaviour of page and extension', () => {  
  test('Look for Earth', async () => {      
    const browserContext = await createBrowserContext() 
    const page = await browserContext.newPage() 
    await page.goto('https://www.google.com/')

    const searchCombobox = await page.getByRole('combobox') 
    expect(searchCombobox).toBeVisible()    

    await searchCombobox.fill('planet earth wiki')  
    await page.keyboard.press('Enter');
    await page.getByRole('link', { name: /Earth - Wikipedia/ }).click() 
    await page.waitForURL('https://en.wikipedia.org/wiki/Earth')

    expect(page.getByText('Earth')).toBeVisible()

    browserContext.close()
  })
})

The above code snippet demonstrates a very basic example. So you will probably need to make adjustments (describing the behavior specific to your extension and what it does with web pages). Once you’ve formulated the test, you can run it using the command npx playwright test path/to/file.

If you need to debug your tests, you can use the --debug flag to walk through the test one step at a time.

npx playwright test path/to/file --debug

Running tests in debug mode has its benefits since you can:

• Use Chrome DevTools
• Manually go through steps at your own pace
• Stop execution whenever you want

Here’s what that might look like.

Add Playwright to GitHub CI

If you generated a GitHub workflow during Playwright installation, you can just push it as is and CI will launch Playwright tests. The only caveat is that if you run tests in headful mode, you should do so with XServer running. To do that, change the line launching tests to this:

xvfb-run npx playwright test

Pro tips

• Use the “Record” button in debug mode to effortlessly generate the skeleton of your test. Start by clicking “Record” and then click all the elements you need on the page. This will improve your productivity in writing tests, since you will get all the selectors you need upfront, and will need to make fewer corrections. Check our quick demo here.
• As much as possible, use selectors that resemble how users will interact with your code (e.g. getByRole is better than getByTestId since users can’t see testId, but they can see button or input and its labels). Check out this doc for a good example of selectors’ priority in docs of react-testing-library.
• Browse Playwright documentation to gain a more thorough understanding of the tool’s capabilities. We’ve only covered the basics here.

Wrapping up

End-to-end testing is the best way to prevent bugs and instabilities from showing up in your extensions. Taking a methodical approach to writing and debugging tests will help you minimize mistakes and improve the reliability of your solution. Meanwhile, robust tools like Playwright take the hassle out of test automation and test environment setup. With the above knowledge, you can be confident that your extensions will continue to provide a smooth user experience even after changes are made.

Thanks for reading! 🎉 I'll leave the original article here if you want to watch the demo videos on how to conduct end-to-end tests on Chrome extensions using Playwright.