It's Hack Week again at SUSE. 🥳 An annual tradition where we all work on passion projects for a whole week. Some of us make music, others use the time to experiment with the latest technologies and start new Open Source projects.
My project this time was to see if there's a better way for us to do automated browser testing of our Mojolicious web applications. For a long time Selenium has been the de facto standard for browser automation, but these days there are more modern alternatives available, such as Playwright. But how good is Playwright really? Spoiler: It's very good. But keep reading to find out why and at what cost.
What is Playwright?
Playwright, just like Selenium before it, is a framework for browser automation. You can use it for all sorts of scripted interactions with websites, such as buying those Nvidia GPUs from online retailers faster than everyone else 😉, but it is most commonly used in test suites of web applications.
The code is being developed by Microsoft as an Open Source project with Apache 2.0 license and distributed as an NPM package. So all you need is Node, and you can install it with a one-liner.
$ npm i playwright
There are bindings for other languages, but to get the most out of Playwright you do want to be using JavaScript. Now when it comes to browser support, where Selenium would give you the choice to pick any WebDriver compatible browser as backend, Playwright will download custom builds of Chromium, Firefox and WebKit for you. And that's all you get.
They are doing it for pretty good reasons though. The browser binaries tend to work flawlessly on all supported platforms, which currently include Windows, macOS and Linux (x86). And when you are used to the sluggishness of Selenium, it almost seems magical how fast and reliable Playwright runs.
This is because where Selenium sticks to open protocols, Playwright will use every trick in the book for better performance. Including custom patches for those browsers, extending their DevTools protocols, and then using those protocols to control the browsers. I'm not a huge fan of the approach, but it's hard to argue with the results.
Short term there are huge benefits, but having to maintain these browser patches indefinitely, if they don't get merged upstream, might hamper the longevity of the project.
Using Playwright
import assert from 'assert/strict';
import { chromium } from 'playwright';
(async () => {
const browser = await chromium.launch({ headless: false, slowMo: 50 });
const context = await browser.newContext();
const page = await context.newPage();
await page.goto('https://mojolicious.org/');
await page.click('text=Documentation');
await page.click('text=Tutorial');
assert.equal(page.url(), 'https://docs.mojolicious.org/Mojolicious/Guides/Tutorial');
await page.screenshot({ path: 'tutorial.png' });
await context.close();
await browser.close();
})();
If you've ever done web development before, the API will be very intuitive, and it was clearly designed with async/await in mind, which i'm a huge fan of. You can have multiple isolated browser contexts, with their own cookies etc., and each context can have multiple pages.
Every interaction, such as page.click(), will automatically wait for the element to become visible, with a timeout that defaults to 30 seconds. This is a huge step up from Selenium, where you have to build this logic yourself, and will get it wrong in many many entertaining ways. 😅
You can emulate devices such as iPhones, use geolocation, change timezones, choose between headless and headful mode for all browsers, and have the option to take screenshots or make video recordings at any time.
One of the latest features to be added was the GUI recorder, which opens a Chromium window, and then records all user interactions while generating JavaScript code as you go. I was a bit sceptical about this at first, but it can significantly speed up test development, since you don't have to think too much about CSS selectors anymore. Even if you just end up using parts of the generated code.
Playwright and Perl
Running Playwright against live websites is very straight forward. But for automated testing of web applications you also want your test scripts to start and stop the web server for you. And this is where things get a little bit tricky if your web application happens to be written in a language other than JavaScript.
use Mojolicious::Lite -signatures;
get '/' => {template => 'index'};
app->start;
__DATA__
@@ index.html.ep
<!DOCTYPE html>
<html>
<body>Hello World!</body>
</html>
What i needed to run my Perl app was a JavaScript superdaemon with support for socket activation. Unfortunately i've not been able to find a module for the job on NPM, and had to resort to writing my own. And now the Mojolicious organisation is not just on CPAN, but also on NPM. 😇
import assert from 'assert/strict';
import ServerStarter from '@mojolicious/server-starter';
import { chromium } from 'playwright';
(async () => {
const server = await ServerStarter.newServer();
await server.launch('perl', ['test.pl', 'daemon', '-l', 'http://*?fd=3']);
const browser = await chromium.launch();
const context = await browser.newContext();
const page = await context.newPage();
const url = server.url();
await page.goto(url);
const body = await page.innerText('body');
assert.equal(body, 'Hello World!');
await context.close();
await browser.close();
await server.close();
})();
You might have noticed the odd listen location http://*?fd=3. That's a Mojolicious feature we've originally developed for systemd deployment with .socket files. The superdaemon, in that case systemd, would bind the listen socket very early during system startup, and then pass it to the service the .socket file belongs to as file descriptor 3. This has many advantages, such as services being started as unprivileged users able to use privileged ports.
Anyway, our use case here is slightly different, but the same mechanism can be used. And by having the superdaemon activate the socket we can avoid multiple race conditions. The socket will be active before the web application process has even been spawned, meaning that page.goto() can never get a connection refused error. Instead it will just be waiting for its connection to be accepted. And important for very large scale testing, with many tests running in parallel on the same machine, we can use random ports assigned to us by the operating system. Avoiding the possibility of conflicts as a result of bad timing.
Combining Everything
And for my final trick i will be using the excellent Node-Tap, allowing our JavaScript tests to use the Test Anything Protocol, which happens to be the standard used in the Perl world for testing.
#!/usr/bin/env node
import t from 'tap';
import ServerStarter from '@mojolicious/server-starter';
import { chromium } from 'playwright';
t.test('Test the Hello World app', async t => {
const server = await ServerStarter.newServer();
await server.launch('perl', ['test.pl', 'daemon', '-l', 'http://*?fd=3']);
const browser = await chromium.launch();
const context = await browser.newContext();
const page = await context.newPage();
const url = server.url();
await page.goto(url);
const body = await page.innerText('body');
t.equal(body, 'Hello World!');
await context.close();
await browser.close();
await server.close();
});
You might have noticed the shebang line #!/usr/bin/env node. That's another little Perl trick. When the Perl interpreter encounters a shebang line that's not perl it will re-exec the script. In this case with node, and as a side effect we can use standard Perl testing tools like prove to run our JavaScript tests right next to normal Perl tests.
$ prove t/*.t t/*.js
t/just_a_perl_test.t ... ok
t/test.js .. ok
All tests successful.
Files=3, Tests=4, 2 wallclock secs ( 0.03 usr 0.01 sys + 2.42 cusr 0.62 csys = 3.08 CPU)
Result: PASS
In fact, you could even run multiple of these tests in parallel with prove -j 9 t/*.js to scale up effortlessly. Playwright can handle parallel runs and will perform incredibly well in headless mode.
One More Thing
And if you've made it this far i've got one more thing for you. In the mojo-playwright repo on GitHub you can find a WebSocket chat application and mixed JavaScript/Perl tests that you can use for experimenting. It also contains solutions for how to set up test fixtures with wrapper scripts and how to run them in GitHub Actions. Have fun!
Top comments (3)
Awesome work, and very useful remarks, congrats Sebastian!
Combination with Mojolicious and JavaScript!
Impressive! 👍