DEV Community

Cover image for  Puppeteer vs Selenium
Klaus for Endtest

Posted on

Puppeteer vs Selenium

Puppeteer and Selenium are both popular, powerful and widely used solutions for automating web applications.

But which one should YOU use?

The answer depends on your specific needs.

Choosing the wrong option could delay or worse, sabotage your automated testing project.

I'll share my own opinion at the end of this article.

Here are some PROS and CONS for both of them.

Puppeteer

PROS
• Faster than Selenium
• Easy to install
• Written and maintained by the Google Chrome devs

CONS
• Works only with Chrome
• The only supported language is Node.js

Selenium

PROS
• Works with most browsers
• Multi-language support
• Huge community of users

CONS
• Difficult to run on all browsers
• A bit slower than Puppeteer

To sum it up, Puppeteer is faster than Selenium, but it works only with Chrome, while Selenium works with Chrome, Firefox, Safari, Internet Explorer and Edge.

Now, for my opinion: both solutions are outdated and built with the narrow mindset of developers who refuse to see beyond their code.

It may sound lazy, but I believe that a solution for automated testing should be like a washing machine: it should give me enough flexibility to choose different inputs and settings, but it shouldn't make me pump the water with one hand and spin the drum with the other.

What both solutions are missing:

• Video Recording functionality
• Turnkey Machine Learning
• Integrated Email Testing option
• Screenshot Comparison feature
• Automatic Smart Waiting
• Structured results

Other issues with Selenium and Puppeteer:

• Difficult to upload files in a test
• Difficult to deal with iframes
• Difficult to integrate with your CI/CD system
• Difficult to configure Element Load Timeout

If you're looking for a better alternative, try Endtest.
It has Puppeteer's speed, it supports Selenium's browsers AND it has all the features that they're missing.

Top comments (21)

Collapse
 
schnuckyschuster profile image
SchnuckySchuster

@klaus ,

your post sounds quite biased.
1) Every now and then (for the last 20 years) a new tool pops up that promises easy to use test specification without any coding knowledge needed. None of those tools have lived up to their promises.
2) You should compare your tool with Serenity BDD, CodeceptJS or any other high level test suite. Selenium and Puppeteer are merely the drivers that speak with the browser
3) Given that more and more teams learn to work agile, testers will be able to code and coders will be able to test. That renders a dedicated tool for people that can not code useless
4) I can code. I have experimented with a few point and klick test tools. Your approach might be of some use for tiny test suites which have no relevance in the real world. Once a test suite becomes a bit bigger there is no alternative to code tests.

Collapse
 
razgandeanu profile image
Klaus

Thank you for taking the time to write that comment.

CodeceptJS and Serenity BDD are just some libraries.
Selenium is also just a library.

The open source and zero license cost nature of such libraries is attractive when compared to the license or subscription costs of commercial automation solutions.

But there is more to it than just that when you look at the cost of implementing an automation solution.

For example, Selenium is open source and free, however building and using an automation solution on this ‘free’ software comes with some real costs:

1) The open source tools/libraries provide a starting point but you still have to write code for test creation, management, execution, handling errors and reporting. Note that this is over and above the test logic that you want to automate.

2) If you are looking to test integrations with back-end systems and external systems via API, you have to build in support for that into your framework.

3) If you are looking to implement a continuous testing solution, you must add to the above effort, the effort of building integration with DevOps or CI/CD tools.

4) Also to be considered are the costs of building and managing an infrastructure to run these tests. These costs are likely to go up as you scale your infrastructure in order to support increased test volumes or to increase the pace of testing.

Collapse
 
entrptaher profile image
Md Abu Taher

To be honest, "EndTest" got nothing to do with the comparison between "Puppeteer vs Selenium". You are missing a big part of Puppeteer and Selenium.

EndTest is focused on testing in mind. You should compare testing tools when mentioning EndTest, not browser automation tools. Your tool is not a browser automation tool, directly.

  • Selenium automates browsers.
  • Puppeteer is a Node library which provides a high-level API to control Chrome.
  • Endless: Codeless Automated Testing.

Nope, EndLess is completely out of context here.

Collapse
 
spekulatius1984 profile image
Peter Thaleikis 🍪 • Edited

Hey, your link above isn't working for me - SSL error. Just FYI

Collapse
 
razgandeanu profile image
Klaus

I don't know which link you're talking about, all of them work.

Collapse
 
spekulatius1984 profile image
Peter Thaleikis 🍪 • Edited

oh, okay. I've retried it and it worked. I ended on the www version (which didn't work). Anyways. Interesting idea

Thread Thread
 
razgandeanu profile image
Klaus

Interesting.
The www version redirects me to the non-www version.
What browser are you using?

Thread Thread
 
spekulatius1984 profile image
Peter Thaleikis 🍪

Firefox 68. I've just tested it and it doesn't redirect www to non-www for me.

Collapse
 
slyubinsky profile image
Sapir Lyubinsky

Thank you Klaus, Great article! I liked that you briefly explained the pros & cons of Selenium and Puppeteer. Also, I liked the reference to what was missing from both solutions. I agree that Puppeteer is faster than Selenium only with Chrome, while Selenium works with Chrome, Firefox, Safari, Internet Explorer and Edge (Don't forget also Mozilla). See this table that explains very nicely the differences you have indicated(See pic) from blog.testproject.io/2020/02/20/sel...

Collapse
 
xclvkjvsd profile image
Alex

There is not that much point with no-code autotests since even very extensive autotests only require high level script coding. The challenges are not primarily programming techniques / advanced coding.

Collapse
 
bennypowers profile image
Benny Powers 🇮🇱🇨🇦
Collapse
 
razgandeanu profile image
Klaus

It says:
BEWARE: Experimental. Just for preview. Installation and usage will change.

We can talk about it when it's production ready.

Collapse
 
bennypowers profile image
Benny Powers 🇮🇱🇨🇦

Still, it got featured on Chrome's YouTube changelog, so that may indicate some measure of stability

Collapse
 
scriptmunkee profile image
Ken Simeon

It's an interesting implementation of a Keyword/Action driven automated testing framework. I've built similar using Java, Selenium, SauceLab and a few add-on libraries for screenshot comparisons. I could see the pull your platform would have on some QA organizations or Directors of Engineering.

Since you've put your product out here I'm assuming you're cool if I ask questions about your platform.

1) I've reviewed your documentation and what I didn't see was how your test framework handles executing the same tests against different environments or URLs. Current software dev shops have multiple environments that they want to run their automated test against (Dev, QA, Stage, ect). How is this handled in your platform using variables?

2) It would be nice to see a demo of how you can have 1 test or suite execute against against multiple browsers without having to duplicate tests for each browser.

3) Do you have any comprehensive examples of your structured results reporting? Your documentation is lacking in this area and as we all know, upper management folks like pretty pictures. ;-)

4) Does your platform offer a way to call external APIs or programs? When I build my automation for a GUI, I like to pre-populate my tests with data before the UI executions and validations without having to drive the UI all the time. Is this possible?

5) When you integrate your platform with Jenkins, Github or others, are the test results pushed back to the calling platform?

6) With your platform retaining all of test actions, video recordings, screenshots and basically the keys to a software teams kingdom with your constant analyzing the sites/apps, what is your security model to protect your customers data?

Thanks for letting me prob into your platform.

Cheers!
Ken

Collapse
 
razgandeanu profile image
Klaus

Hi Ken,

I really appreciate the fact that you tried Endtest.

My opinion is that you could have obtained some of those answers simply by digging more into the platform.

But here are the answers:

  1. You can easily run tests on different environments (Dev, QA, Stage, etc) with Endtest.
    Those environments are obviously going to have different URLs, different usernames, different passwords, etc.
    You can use the "Set Variable" action to create or generate different sets of variables and you can use IF Statements to see which environment you're on and to use the corresponding usernames and passwords.
    You can even define variables directly in the API call for starting a test execution, which makes it even easier.
    There are multiple ways in which you can handle different environments, it depends on what works best for you.

  2. You can run the same test suite on any operating system, browser or mobile device.
    All you have to do is select the desired configuration in the "Run Test Suite" modal or in the API call.
    You don't have to duplicate or modify a test suite in order to run it on different configurations.

  3. You can export the test results directly from the Results page. In that XLSX report, you have all the numbers, all the details, everything that can be extracted from a test execution.
    Since it's in XLSX format, you can easily create your own reports in Microsoft Excel by simply selecting the relevant columns.

  4. Yes, you can use the "Execute JavaScript" step to call any API you want or you can use the "Connect to Database" and "Execute SQL Query" actions to write that data directly in your database.

  5. Absolutely, the integration with services like Jenkins and Github is done through webhooks, you just add the API calls for starting a test execution and fetching the results in the Webhooks sections from those services.
    Additionally, Endtest provides it's own Webhook functionality, this allows you to automatically send the results to a URL after each test execution.

  6. All the traffic on the Endtest platform is encrypted.
    The test data from the execution machines is deleted right after the test is finished.
    The Machine Learning functionality from Endtest does not constantly analyse the sites and the apps, it only analyses them while the tests are being executed.
    There are multiple Security Standards that are being followed, including the famous ISO/IEC 27001.

Collapse
 
scriptmunkee profile image
Ken Simeon

Klaus,

Thanks for the quick response. It was very helpful.

Collapse
 
cescquintero profile image
Francisco Quintero 🇨🇴

Hi!

Just wanted to say that you have a typo in selenium cons:

  • A bit slower than Selenium

should be A bit slower than Puppeteer ?

Collapse
 
razgandeanu profile image
Klaus

Thank you for pointing that out. Fixed it.

Collapse
 
gabssnake profile image
Gabriel Vivas

Your "article" should start with a conflict-of-interest disclosure...

Collapse
 
kaustubhmanglu1 profile image
Kaustubh Manglurkar

As Puppeteer now supports Firefox in addition to the Chrome browser, you guys might want to take a look at this excellent learning resource on Puppeteer written by Dario Kondratiuk - amazon.com/Testing-Puppeteer-end-e...