I'm a fan of Cypress for end to end testing (E2E), but I'm wondering about the Playwright project from Microsoft.
microsoft
/
playwright
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
🎠Playwright
Documentation | API reference
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API. Playwright is built to enable cross-browser web automation that is ever-green, capable, reliable and fast.
| Linux | macOS | Windows | |
|---|---|---|---|
| Chromium 123.0.6312.4 | ✅ | ✅ | ✅ |
| WebKit 17.4 | ✅ | ✅ | ✅ |
| Firefox 122.0 | ✅ | ✅ | ✅ |
Headless execution is supported for all browsers on all platforms. Check out system requirements for details.
Looking for Playwright for Python, .NET, or Java?
Installation
Playwright has its own test runner for end-to-end tests, we call it Playwright Test.
Using init command
The easiest way to get started with Playwright Test is to run the init command.
# Run from your project's root directory
npm init playwright@latest
# Or create a new project
npm init playwright@latest new-project
This will create a configuration file, optionally…
It's essentially cross-browser Puppeteer from what I've understood and there are some new goodies in the 1.4 release.
I'm going to take it for a spin, but curious if others on DEV have used it.
Drop your comments, posts or whatever you got in the discussion below.
Top comments (12)
I'm not familiar with Cypress but it looks like it brings tools together to provide a complete solution. Playwright/Puppeteer only provide communication to the browser.
Is Cypress working for you? What issues do you have with it?
My experience lies with C# and selenium. This past couple weeks I've been diving into Puppeteer (Playwright is a fork). We are utilizing it with jest and it is not anywhere near as stable or fast as our selenium solution. This isn't about the tools though, we need to put in the work to achieve the same standards.
Also, we aren't writing end to end tests. At some point I may identify a place this in need of E2E, but I look to avoid that and have appropriate mocks/stubs in place.
So don't look at changing tools, look at changing test strategy.
Cypress works well.
Where I'm currently at, DEV/Forem, we're still looking into an E2E tool. We currently use Capybara with Rspec, but it is mainly integration tests from what I've seen.
I'm just looking at other tools still as it is an open source project. Here's the issue I created a couple of years ago. Funnily enough as I read through the issue again, I can see I mentioned Playwright when I started at DEV/Forem.
@nickytonline commented on Thu Jul 19 2018
Story - Consider Adding End to End (E2E) Testing to the Deployment Pipeline
@benhalpern @maestromac @Zhao-Andy @jessleenyc et all.
Just wanted to open a discussion here about E2E testing.
There's a lot going on on the site. The backend has lots of unit tests, the new front-end stuff has unit tests and snapshot testing, but there is no E2E testing in place.
When I worked at McAfee, it was essential to have E2E testing in place because we were working on a password browser extension. We had to be certain the UI functioned all the time. In the case of a web site, it's a good idea as well, but a lot of times it's omitted due to lack of resources/time. Not good excuses, but that's the reality I've lived through at other jobs. There's also a burden of maintaining them.
Having said that, they help you spot regressions fairly quickly and it allows you to test full workflows, e.g. on-boarding. In terms or resources, I think we'll have those once the code is fully open sourced. I have a feeling a few people will want to contribute to the code base.😉
I think it's a win win honestly. The site will be more stable because of the E2E tests and there will be so many tests to do that it will be an easy way for new contributors to get some PRs in (provided we onboard properly).
If we decide to go ahead with this, we'll need to decide on an E2E testing framework. I'm personally leaning towards Cypress, but let's make this part of the discussion.
@benhalpern commented on Fri Jul 20 2018
Yeah let's do this. I haven't seen any reason to not go with Cypress. It's definitely been on my list of things to try and haven't seen anything better.
Rails has its own concept of E2E testing which we make some use of but it's been such a pain in the ass I've never had the patience to be good at this part. I'll leave this topic open for anyone to argue that we should stick into this world as opposed to Cypress.
Either way, yeah I'm in agreement with the approach to lead to stability, as long as we don't get too heavy-handed to the point where we need to change 10 files for every change. Let's try to walk the line the right way.
@nickytonline commented on Sun Jul 22 2018
Also, it currently only supports Chrome, but FireFox support is on it's way.
@nickytonline commented on Thu Aug 16 2018
@maestromac , can you migrate this issue over? I'll put up my PR in my branch later tonight or tomorrow. Thanks.
This is likely due to my misunderstanding of E2E. I generally find E2E implementations tend to focus only on one end, which is fine when you already have all the integration testing that validate those ends.
But even then the E2E testing is executed in the wrong place. E2E is a validation of the system setup. All of the efforts goes into verifying lower environments. Being test environments they are "broken" to test a situation. So integration tests fail and everyone is trying to find that one person who can put it back together.
But with a good suite of contract tests in place you focus on the real issue, components aren't connected together correctly. Anyway I ramble.
Eliminating Environments
Jesse Phillips ・ Sep 7 ・ 3 min read
Check out CodeceptJS codecept.io/
It's an abstraction on most of the e2e tools out there, including Playwright, you choose your
drivere.g. Playwright/Puppeteer but have a consistent simple API for test casesCool stuff! Thanks for sharing.
Isn't Playwright much closer to Puppeteer than cypress?
Arjun from the Playwright team here. Yes, Playwright - as it stands today - is an automation API and one can use it with testing frameworks to enable e2e testing. To improve this experience, we are investing in e2e test tooling around Playwright. For example, playwright-runner is an upcoming first-party test runner integration. Docs coming soon for early adopters!
Awesome! Thanks so much for chiming in Arjun.
Much suited for more general things like scraping and automating browser instead of testing.
Yeah, I mention that in the post. On its own it does not do any test assertions, but you can integrate it with jest, mocha etc. To provide e2e testing.
Awesome. I would love to hear more about this.
We should have separate frameworks for hi and API testing,makes it cleaner