Developer Weekly Blog #4 - Playwright

Introduction

This week I learned about Playwright, this is a tool to do different types of testing like: automating testing or visual testing on your web applications. What I like about this library is that it needs almost 0 configuration and you already have a lot of tools ready out of the box to create your tests, like a tool that helps you to create your own tests called codegen, when you initiate your project automatically create a pipeline that is easy to use in your pipeline, it allows you to create visual tests for your app and you’re able to run the tests in various browsers. And using this tool I realized that this was an ideal solution for a problem that I had back in the past and I want to share this experience with you.

I was writing tests on React Testing Library, a tool to create unit tests for components. At some point I abused this tool, using it in moments where I shouldn’t, creating long tests, those tests started to become flaky tests (tests that sometimes succeed or failed with no changes on the code.) Why were they failing may ask yourself? Because due to how it works this tool, it emulates a browser and sometimes as we were refreshing so many things a lot of times happened that doesn’t reach to the point where the changes were reflected on the emulated browser and the test failed.

It would be nice back then to have the possibility to write some tests in Playwright because they would be more effective and accurate than my long and ineffective tests written in Testing Library. But this is not the only reason that I would recommend to use it on your projects, also is easy to use, has a lot of ways to debug your tests, it has a tool that helps you to create your own tests called codegen, is easy to deploy, you can create visual testing and you’re able to use it in various browsers. Now that I list you some of the reasons why I love this tool it’s time to see how it works.

Why is important to include automating testing

Automating testing is so important in your workflow because this allows you to make changes with the confidence that this doesn’t break anything extremely important on your app. Have you ever made a change and feel the stress and anxiety of something breaking? So where does this insecurity come from? I think more than impostor syndrome as an answer, is that it is not always possible to check all the scenarios where your change is going to be reflected. And it is impossible that your app works fine in every case, but you’re able to make your app work fine most of the time. But how to ensure nothing breaks?

You possibly can test every time you make a change but a good way to ensure repetitiveness is through automated testing. And to better understand the benefits of doing it I refer to the story of Google Web Server that illustrates how automated testing improves productivity  confidence on changes and  the reduction of numerous bugs. Using this practice you ensure the changes don't break some important behavior, reduce stress, reduce uncertainty, keep application flexible and easy to change overtime and increase confidence on your team. When the test succeeds you deploy with confidence.

So I want to share information about this tool that allows you to write this kind of test in an easy way with low configuration.

Getting started with Playwright

To install Playwright you need to have Node 16+ installed, and get Playwright through the command $npx playwright install this will get the browsers to run your tests (Chromium, Mozilla and Safari). After that you’re able to use the commands that Playwright includes to run your tests and complementary tools to create your tests. If you want to add the configuration to an existing project use $npm init plawright this creates some files in your project but the most important is playwright.config.ts that allows you to change the default configuration and indicates that Playwright is being used in this project. From there it is just a matter of creating your tests, to do it create files with the extension .spec.ts on the test folder and run them with $npx playwright test. And see the results with $npx playwright show-report. With those steps you’re ready to create more tests in your existing project. Let's see what it is like to create a test.

The first thing you need to take into account is that you're using a real browser. If you're familiar with other UI testing libraries, you'll find a lot of similarities between them. However, since you're using a browser, there are some functions like changing accessing a different URL, something not available in other UI testing libraries like React Testing Library. With that in mind, how does a test in Playwright looks? A test in Playwright is mostly composed of 3 main instructions: goto(), locator(), and expect() throughout a test. So let's define the purpose of each one of these:

• goto() is a function used to navigate to a website, similar to typing a URL in your browser. This instruction is used to access the page that you will be probing.
• locator() is used to find elements within a website. Playwright uses the same syntax as CSS selectors when using the locator method. This means that you can find elements using HTML element types, classes, or IDs to identify them. There are other ways to locate elements, but they are more related to accessibility attributes rather than HTML properties.
• expect() is used to compare the actual results of a test with the expected results. This function allows you to check if elements satisfy certain conditions, such as being visible, checked (for checkboxes), or having specific properties. There are many possibilities for comparing the results with the expected results. I will explain some of the most common methods to make these comparisons.

So I want to show a brief example that shows a short a test to add products in a shopping cart on a website called automation exercise, designed to train yourself to make automated testing.

import { test, expect } from "@playwright/test";

test("Shopping cart test", async ({ page }) => {
  // Enter to automationexercise.com webpage
  await page.goto("https://automationexercise.com/");

  // Locate link to access the product 
  await page.locator(".features_items .choose a").first().click();

  // Click to skip ads 
  await page.mouse.click(0, 0);

  // Add 3 products
  await page.locator("#quantity").fill("3");

  // Add product to the cart
  await page.locator(".product-information button").click();

  // Click on the button to close the modal and keep navigating 
  await page.locator(".modal button").click();

  // Expect the modal is closed once is clicked 
  await expect(page.locator(".modal")).not.toBeVisible();
});

This is the step-by-step process of what I'm doing here:

1. Enter the website automation exercise.
2. Go to the section that includes the products and find the first link located there.
3. Add an instruction to skip the ads that sometimes appear on the website and click in the top-left corner to close a modal that contains the advertisement.
4. Once on the product detail screen, fill out the form to add 3 products.
5. Click the button to add the products to the basket. When the products are added, a modal is displayed.
6. Close the modal by clicking the button to do so.
7. Close the modal using the button inside, and finally, check if the modal is not visible.

If you notice that I was on two screens while doing this test, do I need to explicitly mention it? No, because Playwright does a lot of auto-waiting for me. For example, when moving to a new URL or waiting for the modal to close. Those are the tiny details that makes the developer experience awesome using it.

How to wait for something to happens

But if you want to wait for something to happen because sometimes you need to wait for something to continue your test, like wait for some element to appear/disappear on the screen. You have the option to do it explicitly using waitFor() to make a pause until some element satisfies a condition. I think this is a very repetitive situation when using a testing framework for the UI. In my first experiments I have to use it to wait that an elements appear on the screen and I wanted to share with you an example about how I use it.

This is a test that need to wait for an element to be visible to continue to test a functionality.

test("Search for something that has at least one result", async ({
  page,
}) => {
    // Enter to Playwright page
    await page.goto("https://playwright.dev/");

    // Find the search button
  await page.getByRole("button", { name: "Search " }).click();

    // Locate the search box to find a doc
  const searchBox = page.getByPlaceholder("Search docs");

    // Click on the search box
  await searchBox.click();

  // Type havetext on the searchbox
  await page.getByPlaceholder("Search docs").fill("havetext");

  // Wait for one result to appear on the screen
  await page
    .locator(".DocSearch-Dropdown-Container section")
    .nth(1)
    .waitFor({ state: "visible" });

  // Count how many results does it get
  const numberOfResults = await page
    .locator(".DocSearch-Dropdown-Container section")
    .count();

    // Expect at least to have one result 
        // looking at the results section
  await expect(numberOfResults).toBeGreaterThan(0);
});

The step-by-step process is something like this:

1. Go to the Playwright website.
2. Find the search button.
3. Locate the input with the placeholder "search docs".
4. Click on that input.
5. Fill the input with the text "havetext".
6. Wait for the results to be visible because we need to wait until the result is sent back to us.
7. Count the number of results.
8. Make sure that at least one result exists.

Now that we are familiar with the most common tests you will find, let's talk about the tools that come with Playwright.

Tools

One of the things I love the most about Playwright is the tooling that comes with it. There are a lot of different ways that Playwright facilitates the job of creating or fixing tests. Let's begin with the main tool that I found very useful to get started with: codegen, a tool that opens a browser and starts to create code based on the actions you perform on the website, such as clicking on an element, typing in a form, or selecting text on the page. Is it possible to use the codegen result as the tests you will use? Sometimes yes, but I think for the vast majority of the tests, you will need to make some changes to improve things like the locator of an element, use a REGEX to ignore case when searching for text on your website, or make any other improvements to increase flexibility in your tests. Now let’s talk about debugging tools.

To debug in Playwright, there are plenty of options to see the steps it's executing. For example, if you want to observe how the tests execute in the browser, you can do so by adding the --headed flag when running the test. However, the test might be fast, and you may not perceive anything. If you prefer a slower execution, you can use an option called slowMo to prevent the test from executing too quickly. Additionally, using the --debug flag opens another window that allows you to pause, proceed to the next step, or run the test until the end, giving you control over the test's actions. These tools make your life easier when you need to fix errors, and these are not the only options available. Another option is called "trace."

When you run your test with the flag --trace on, this will record every activity that the browser does. After the tests are done, the failed tests will generate an interactive record that allows you to see how the element was located and how the DOM looks at that moment. It's similar to when you open a page with DevTools, which allows you to interact with the page and view the details. To see the trace, you need to use show-report to see the test results and be able to open the trace. The report is only available through the option show-report. You cannot serve it through other static servers or open it as plain HTML, especially when there are files that you need to view, like the traces so if you want to see traces results use that option.

Now we are going to review the functions available to write tests and let’s begging with the selectors.

Selectors

There are many ways to filter elements inside a page, and now I want you to become familiar with some of the ways to find elements inside a page. You can find elements by HTML attributes or the purpose on the page, such as accessibility role, label text, placeholder (if it is an input), text inside a page, alt text of an image, title attribute, or test ID. To find elements using any of these criteria, use some of the functions provided by Playwright.

• getByRole() to locate elements based on how people that uses assistive technologies perceive the page, so is useful to review which accessible roles exists to help you to understand which type of elements are you able to locate you can find more about roles in the following link: WAI-ARIA Roles MDN Web Docs.
• getByLabel() to locate mainly form related elements, the label is the inscription for any type of input like text , checkboxes, radio buttons, selector and many others.
• getByPlaceholder() is related to form elements as well, but it is created to find elements based on a text that indicates how to fill the input or what the input is for.
• getByText() to locate any text inside of the page.
• getByAltText() helps you locate an image using the descriptive text used by people who use assistive technologies to understand the content of those images.
• getByTitle() to locate elements that use title tag
• getByTestId() is a special attribute used in HTML called test-id, which is intended for testing frameworks to easily locate elements on the page.

If you want to get an extensive list of all the selectors and get more information, go to the official documentation on locators to learn more. Since there are many ways to locate elements on the page, there are also many ways to compare final results with expected results, so let’s review some of them.

Assertions

There are many functions to compare results, so I want to talk about the most common ones that you will use to check if some element of the page satisfies a certain condition. Let's see some of them:

• toBeTruthy() checks if an element is not false, null, undefined, or 0. It is used to verify if a condition is met and is one of the simplest ways to compare if an element fulfills a certain requirement. The opposite is toBeFalsy(), which checks if something does not meet a condition.
• toBe() checks if the value is the same. It is one of the most basic ways to check if something has an expected value. For example, when performing a search, you expect to have a certain number of elements.
• toBeDefined() checks if an element is not undefined. It is useful for verifying the existence of an element on a web page. For example, you can use it to check if a button is present on the web page.
• toBeVisible() checks if an element is visible on the web page. For example, if you have an element that appears or disappears depending on user interactions, you can use this function to check if the element is visible on the page.
• toHaveText() check if some element have some text. Let’s say a button and you need to verify if the call to action have the right text, you can use this function to verify it.
• toBeDisabled() helps you verify if an element is disabled. It is useful to check if buttons, inputs, and any other kind of interactive element are not enabled for the user to interact with.
• toHaveURL() checks if a link has the correct URL. When you need to verify that a link has the correct redirection, you will use this.
• toHaveAttribute() allows you to select an attribute and compare if it contains a certain value. For example, if you want to check if an image has a specific width. And not only that, you can virtually compare any attribute that was not previously included.

Sometimes, we actually need to verify that something does not satisfy a condition. To cover those cases, we have the .not attribute available. It allows us to check if something does not match a condition. For example, if you want to verify that some text has changed, you can use the .not attribute to check if the text is no longer on the page.

I will probably continue to describe every way to make an assertion, but you can check every assertion that exists in the test assertions documentation and you'll get more information there. If you consider that you need to use toHaveAttribute() to compare something, please first read if there exists a method for that because it will make your intention clearer.

Now we talk about one of the most impressive features, that is doing visual testing with Playwright.

Visual testing

Have you ever imagined having the chance to visually compare how a page looks before and after a change, automatically, and know if any differences exist? Well, this is now possible with Playwright. You can create a screenshot that automatically compares the appearance before and after running tests. If any differences are found, the test will fail. The best part is that these tests generate a diff file that shows the specific differences between the previous and new versions, allowing you to easily spot the changes.

To create this kind of test, all you need to do is use the assertion toHaveScreenshot(). Once you use it, it creates an image that will be used as a reference for how the page is supposed to look, and it compares the results with that reference every time the test is executed. If you feel this is too strict and want to allow a little freedom, you can define a tolerance for the difference between screenshots using the option maxDiffPixels. This allows you to change the maximum tolerated difference in pixels between two images.

So doing a visual testing is that simple as write a test like this:

test('example test', async ({ page }) => {
  await page.goto('https://playwright.dev');
  await expect(page).toHaveScreenshot({ maxDiffPixels: 100 });
});

And you got it you have your visual test online.

Final thoughts

I found Playwright to be a tool that is really easy to use. It has a lot of capabilities out of the box. If you're familiar with other tools for UI testing, this will be familiar to you. Remember that automated tests are a way to increase confidence in your app in a scalable way. You can add a few tests on Playwright to make sure that the most important parts of your app are working fine. If you have an e-commerce test where you can log in, add products to your shopping cart, and complete the purchase, it will cover one of the most common and important flows in e-commerce applications.

Take advantage of this tool and test it because it will increase your productivity overall and catch bugs early on the development.