loading...
Cover image for Playwright for E2E testing?

Playwright for E2E testing?

nickytonline profile image Nick Taylor (he/him) ・1 min read

I'm a fan of Cypress for end to end testing (E2E), but I'm wondering about the Playwright project from Microsoft.

GitHub logo microsoft / playwright

Node.js library to automate Chromium, Firefox and WebKit with a single API

🎭 Playwright

npm version Join Slack Chromium version Firefox version WebKit version

Documentation | API reference

Playwright is a Node.js library to automate Chromium, Firefox and WebKit with a single API. Playwright is built to enable cross-browser web automation that is ever-green, capable, reliable and fast.

Linux macOS Windows
Chromium 86.0.4238.0
WebKit 14.0
Firefox 80.0b8

Headless execution is supported for all the browsers on all platforms. Check out system requirements for details.

Usage

npm i -D playwright

This installs Playwright and browser binaries for Chromium, Firefox and WebKit. Once installed, you can require Playwright in a Node.js script and automate web browser interactions.

Capabilities

Playwright is built to automate the broad and growing set of web browser capabilities used by Single Page Apps and Progressive Web Apps.

  • Scenarios that span multiple page, domains and iframes
  • Auto-wait for elements to be ready…

It's essentially cross-browser Puppeteer from what I've understood and there are some new goodies in the 1.4 release.

I'm going to take it for a spin, but curious if others on DEV have used it.

Fire beside desk of IT crowd employee

Drop your comments, posts or whatever you got in the discussion below.

Discussion

pic
Editor guide
Collapse
jessekphillips profile image
Jesse Phillips

I'm not familiar with Cypress but it looks like it brings tools together to provide a complete solution. Playwright/Puppeteer only provide communication to the browser.

Is Cypress working for you? What issues do you have with it?

My experience lies with C# and selenium. This past couple weeks I've been diving into Puppeteer (Playwright is a fork). We are utilizing it with jest and it is not anywhere near as stable or fast as our selenium solution. This isn't about the tools though, we need to put in the work to achieve the same standards.

Also, we aren't writing end to end tests. At some point I may identify a place this in need of E2E, but I look to avoid that and have appropriate mocks/stubs in place.

So don't look at changing tools, look at changing test strategy.

Collapse
nickytonline profile image
Nick Taylor (he/him) Author

Cypress works well.

Where I'm currently at, DEV/Forem, we're still looking into an E2E tool. We currently use Capybara with Rspec, but it is mainly integration tests from what I've seen.

I'm just looking at other tools still as it is an open source project. Here's the issue I created a couple of years ago. Funnily enough as I read through the issue again, I can see I mentioned Playwright when I started at DEV/Forem.

Consider Adding End to End (E2E) Testing to the Deployment Pipeline #382

@nickytonline commented on Thu Jul 19 2018

Story - Consider Adding End to End (E2E) Testing to the Deployment Pipeline

@benhalpern @maestromac @Zhao-Andy @jessleenyc et all.

Just wanted to open a discussion here about E2E testing.

There's a lot going on on the site. The backend has lots of unit tests, the new front-end stuff has unit tests and snapshot testing, but there is no E2E testing in place.

When I worked at McAfee, it was essential to have E2E testing in place because we were working on a password browser extension. We had to be certain the UI functioned all the time. In the case of a web site, it's a good idea as well, but a lot of times it's omitted due to lack of resources/time. Not good excuses, but that's the reality I've lived through at other jobs. There's also a burden of maintaining them.

Having said that, they help you spot regressions fairly quickly and it allows you to test full workflows, e.g. on-boarding. In terms or resources, I think we'll have those once the code is fully open sourced. I have a feeling a few people will want to contribute to the code base. 😉

I think it's a win win honestly. The site will be more stable because of the E2E tests and there will be so many tests to do that it will be an easy way for new contributors to get some PRs in (provided we onboard properly).

If we decide to go ahead with this, we'll need to decide on an E2E testing framework. I'm personally leaning towards Cypress, but let's make this part of the discussion.


@benhalpern commented on Fri Jul 20 2018

Yeah let's do this. I haven't seen any reason to not go with Cypress. It's definitely been on my list of things to try and haven't seen anything better.

Rails has its own concept of E2E testing which we make some use of but it's been such a pain in the ass I've never had the patience to be good at this part. I'll leave this topic open for anyone to argue that we should stick into this world as opposed to Cypress.

Either way, yeah I'm in agreement with the approach to lead to stability, as long as we don't get too heavy-handed to the point where we need to change 10 files for every change. Let's try to walk the line the right way.


@nickytonline commented on Sun Jul 22 2018

Also, it currently only supports Chrome, but FireFox support is on it's way.


@nickytonline commented on Thu Aug 16 2018

@maestromac , can you migrate this issue over? I'll put up my PR in my branch later tonight or tomorrow. Thanks.

Collapse
jessekphillips profile image
Jesse Phillips

This is likely due to my misunderstanding of E2E. I generally find E2E implementations tend to focus only on one end, which is fine when you already have all the integration testing that validate those ends.

But even then the E2E testing is executed in the wrong place. E2E is a validation of the system setup. All of the efforts goes into verifying lower environments. Being test environments they are "broken" to test a situation. So integration tests fail and everyone is trying to find that one person who can put it back together.

But with a good suite of contract tests in place you focus on the real issue, components aren't connected together correctly. Anyway I ramble.

Collapse
griffadev profile image
George Griffiths

Check out CodeceptJS codecept.io/
It's an abstraction on most of the e2e tools out there, including Playwright, you choose your driver e.g. Playwright/Puppeteer but have a consistent simple API for test cases

Collapse
nickytonline profile image
Nick Taylor (he/him) Author

Cool stuff! Thanks for sharing.

Collapse
rafi993 profile image
Rafi

Isn't Playwright much closer to Puppeteer than cypress?

Collapse
arjunattam profile image
Arjun Attam

Arjun from the Playwright team here. Yes, Playwright - as it stands today - is an automation API and one can use it with testing frameworks to enable e2e testing. To improve this experience, we are investing in e2e test tooling around Playwright. For example, playwright-runner is an upcoming first-party test runner integration. Docs coming soon for early adopters!

Collapse
nickytonline profile image
Nick Taylor (he/him) Author

Awesome! Thanks so much for chiming in Arjun.

Hackerman from Kung Fury putting on a Nintendo Power glove

Collapse
rafi993 profile image
Rafi

Much suited for more general things like scraping and automating browser instead of testing.

Collapse
nickytonline profile image
Nick Taylor (he/him) Author

Yeah, I mention that in the post. On its own it does not do any test assertions, but you can integrate it with jest, mocha etc. To provide e2e testing.

Collapse
rafi993 profile image
Rafi

Awesome. I would love to hear more about this.

Collapse
vip383 profile image
vinay_patil94

We should have separate frameworks for hi and API testing,makes it cleaner