How to Accelerate Selenide Tests with Playwright

Playwright + Selenide = unite and rule

Hello all. I’m Serhii Bryt. I’m Senior Test Automation Engineer at Provectus. Here in Provectus we use different kind of tools for automation in different areas: web UI, RestApi, Big Data, AI, ML etc. This article is about experiments with combining two different tools for web UI automation.

Ask yourself what bothers you the most about automation tests.

Some will answer that it’s the time for development, some will complain about debugging. But to me, the most bothersome thing is the time it takes to run tests.

Imagine that you want to use Selenium to automate the process of filling out a form that consists of 20 fields. The time of running the test will be short, but if your next step is to automate the same form with 120 fields, the time required will increase by at least several times.

This happens because of the nature of Selenium. In this article, I will try to explain how to overcome the limitations of Selenium and resolve the test running problem without having to migrate to another tool.

But first, let’s take a look at some of the most popular UI automation tools.

UI Automation Tools

Selenium – A great tool for web automation, Selenium allows you to manipulate the browser and its content. The tool comes with a big community and supports all known browsers: Chrome, Edge, Firefox, Safari, etc. It has implementations in all popular languages: Java, JavaScript, Python, C#, etc.

Selenide – A wrapper for Selenium, Selenide consists of many other tools, including Selenium, WebdriverManager, and BrowserUp Proxy. Selenide is well-organized and comes with good documentation and a large community. It enables you to simplify complex tasks, for example: file download, autostart and autostop of the browser, using a proxy, etc. Though it officially supports only Java, there is also an unofficial port in Python called Selene.

Cypress.io – A great tool with a big community, Cypress.io is perfect for e2e tests and supports JavaScript/TypeScript. It is not compatible with Selenium. Make sure to run your tests in a native environment, to get all the benefits of this tool.

CodeceptJS – A wrapper for different tools in one interface. Language: JavaScript/TypeScript. Frameworks that are wrapped: Playwright, Webdriver.io, Puppeteer, Protractor, etc. The service also comes with a large community and is easy to install and use for e2e testing.

Puppeteer – A web testing tool based on CDP (Chrome devtools protocol), Puppeteer also works perfectly with Chromium-based browsers and has limited support from Firefox. Puppeteer has a supportive community that works hard on documentation. The tool has unofficial ports for Python, Java, etc. When it comes to tests, it uses JavaScript/TypeScript.

Playwright – The hero of this article, Playwright is a new, exciting solution for web (and other) test automation. It Is supported by Microsoft and has great documentation and a strong community. The service comes with various implementations for Java, JS, .Net, and Python. The syntax is similar to Puppeteer. Playwright is a fork of Puppeteer, but can work with Chrome, Firefox, and Webkit-based browsers with a single API.
A Little Bit of Theory on UI Automation Tools

Selenium

Selenium uses the Json Wired protocol to work with browsers. You actually need only three things to work with it:

1. Webdriver. This is a binary file produced by browser vendors: Chromium (Google) driver, Gecko (Mozilla) driver, and Webkit driver (Apple Safari).
2. Selenium framework. A webdriver protocol implementation for any language (Java, Python, C#, JavaScript/TypeScript etc.) + Test runner (Junit, TestNF, etc.)
3. Browser.

In general, Selenium has a client-server architecture.

Selenide

The Selenide project is built on top of Selenium. But it consists of many other tools such as: Webdriver manager, BrowserUp proxy, PdfBox, etc. It works similarly to Selenium, but has more out-of-the-box features.

Playwright

The project is young but promising, and comes with powerful functionality.

At the beginning, it was just a fork of another famous tool — Puppeteer. But the main disadvantage of Puppeteer is that it works mostly with Chrome using CDP.

The Playwright team decided to change this and created another framework, with one interface for interacting with different browsers and with syntax, as in Puppeteer. But the main idea of the project, and I think a good one, is to develop automation tests for different browsers, regardless of the OS being used. To do this, they created many instances of browser engines that users could run with or without head on any platform.

I cannot help mentioning that Playwright does not have a server level like Selenium. It is just a webdriver binary. Also, this framework provides many out-of-the-box features, like recording video, recording traffic and tracing, making restApi requests, and so on.

And the most important advantage of PlayWright is that it has framework implementations for such popular languages as Java, Python, C#, and JavaScript/TypeScript.

The Problem of Test Run Time

Let's go back to the problem mentioned at the beginning of this article and try to resolve it.
As an example, I have chosen the portal DataTables example - Form inputs. If you select “Show 100 entries,” you will be able to interact with about 170 inputs.

Regular Selenium (Selenide) tests showed not bad results. The average time is 24 seconds to fill in all inputs.

When using Selenide for these tests, I also tried to use the Selenide configuration flag:

Configuration.fastSetValue = true

So now, Selenide will use Java script to set up values for the inputs wherever possible.

Average time: 11 seconds.
Very good result. Time is decreased by 2x.

Then, I tried to use Playwright.

Average time: 3 seconds!!!!
This is a great result!

But how can I combine these two tools to save even more time?

The answer is CDP.

“The Chrome DevTools Protocol allows for tools to instrument, inspect, debug and profile Chromium, Chrome and other Blink-based browsers. Many existing projects currently use the protocol. The Chrome DevTools uses this protocol and the team maintains its API.”
— Chrome DevTools Protocol (Wiki)

Good support of CDP began when webdriver 4.x was released.

So let's try to connect the already started browser to Playwright, and try to interact with web content. It should look something like this:

Playwright playwright = Playwright.create();
Browser browser = playwright.chromium().connectOverCDP(
        ((RemoteWebDriver) webdriver().object())
                .getCapabilities()
                .getCapability("se:cdp")
                .toString());

"se:cdp" is created automatically when Selenide creates a new browser session. This is the websocket connection address used by the CDP protocol.

The results (in seconds) of one of the test runs is:

The results are great.

As we can see, Selenide using JS for filling text is much slower than Playwright connected to the Selenide session (PW + Selenide).

You can find this example in my GitLab repository.

Bonus Features of Playwright

Video Recording

For Selenide, I used this solution to record a video to demonstrate how tests are run. It records the desktop, but Playwright has a feature that can do this too. If you use PW only, you simply point to the folder where you want to save your video.

Playwright playwright = Playwright.create();
Browser chrome = playwright
      .chromium()
      .launch(new BrowserType.LaunchOptions().setHeadless(true).setChannel("chrome"));
BrowserContext video = chrome.newContext(new Browser.NewContextOptions()
      .setRecordVideoDir(Paths.get("video"));

But this is impossible to do when you try to connect to an existing Selenide session. So, we can do one trick. We run a PW session with recording first, and then connect Selenide to the PW session using a debug port.

Example:

Playwright playwright = Playwright.create();
Browser chrome = playwright.chromium().launch(new BrowserType.LaunchOptions()
        .setHeadless(true)
        .setChannel("chrome")
        .setArgs(List.of("--remote-debugging-port=9222")));
BrowserContext video = chrome.newContext(new Browser.NewContextOptions()
        .setRecordVideoDir(Paths.get("video")));
......
Configuration.browser = MyWDProvider.class.getName();

Where MyWDProvider.class is:

public class MyWDProvider implements WebDriverProvider {
    @Override
    public WebDriver createDriver(@Nonnull Capabilities capabilities) {
        WebDriverManager.chromedriver().setup();
        ChromeOptions chromeOptions = new ChromeOptions();
        chromeOptions.setExperimentalOption("debuggerAddress", "127.0.0.1:9222");
        chromeOptions.merge(capabilities);
        return new ChromeDriver(chromeOptions);
    }
}

Record HAR

From time to time, you need to store traffic while tests are running for later analysis. For Selenide (Selenium), we can use a BrowserMob proxy or a BrowserUp proxy. We can connect one of them to a newly created Selenium session, check traffic, store it in an array, and at the end of the test, we can save it to an HAR file. After that, we can analyze this file using any HAR viewer.

As with the previous feature, you simply point to the folder where you want to save the resulting HAR file.

Playwright playwright = Playwright.create();
Browser chrome = playwright.chromium().launch(new BrowserType.LaunchOptions().setHeadless(true).setChannel("chrome"));
BrowserContext video = chrome.newContext(new Browser.NewContextOptions()
            .setRecordHarPath(Paths.get("har/har.har")));

Record Tracing

Sometimes we need to record all actions with traffic and load timings, so we need tracing. Playwright has a solution for this. You just need to add a couple of rows of code, like this:

....
video.tracing().start(new Tracing.StartOptions().setScreenshots(true).setSnapshots(true));
....
video.tracing().stop(new Tracing.StopOptions().setPath(Paths.get("trace/trace.zip")));

More about tracing and trace viewer

After the recording, you can use online trace viewer or local:

mvn exec:java -e -Dexec.mainClass=com.microsoft.playwright.CLI -Dexec.args="show-trace trace.zip"

That is all that I wanted to share about combining two different tools.

This combination of two UI automation tools has helped me to solve the test runtime problem. And I got more than just a decrease in time; I got the possibility to take full screenshots of failed tests, to save traffic and tracing, to make quick requests to the server, to play around with CDP, and much more.

Do not be afraid to experiment. And good luck!

NOTE. You can find all the source code here.

Image by Kjpargeter from freepik