Disclaimer: I work at Frontend Robot, so I'm a little biased here. Where I express personal considerations I will try to make it as clear as possible that they are my own views.
The best thing that happened to the software industry
If we take a look at the evolution of web development in the last few years, I can only think to one aspect that really changed the face of the industry and that's Developer Experience (DX). Based on the slogan "developers are people too", the DX movement is bringing the ergonomics and the concepts of User Experience (UX) into the developer's world.
Developer experience spans tools, APIs, SDKs, documentation and workflows; essentially, most aspects of a developer's job. In general, if a developer uses or interacts with a product of some kind, then, DX is involved.
Personally, the first time that I became DX-aware, was in 2015 when I started using React Hot Loader after watching a talk by Dan Abramov and his subsequent interview with Kent C. Dodds where they explicitly talk about "Developer Experience and Tools". For those not familiar with the tool, React Hot Loader allows developers to see changes made to a React application live, without the need to reload the page. It works like magic. React Hot Loader became for me the perfect example of what a good Developer Experience means; less frustration, more productivity and putting back the word fun into software development.
Tester Experience (TX), encompasses the same ergonomics principles that characterize UX and DX, but applied to software testers. Luckily, in the last few years, the industry has made giant steps in the area of TX as well. Similarly to UX and DX, Tester Experience is not just about the ease of use of a product, but also about the emotional impact that it has on the person.
From open-source tools to cloud-based services, we have witnessed a surge of web testing tools aimed at radically improving the testing experience, making it simpler, more rewarding and even fun.
The one that follows is a list of the top 5 testing tools born in the last few years, which shine for providing a great Tester Experience. Eeach one has its pros and cons, so just choose the one that works best for you.
1. Frontend Robot
Frontend Robot is a cloud platform which will get you up and running with automated testing in just a couple of minutes (literally). Frontend Robot was born two years ago from my frustration with setting up and writing automated tests for the web. Just to write a simple test, I needed to choose a framework, set it up, and have a build machine to run it on. On top of that, writing tests was a laborious matter with a constant back and forth between the code editor and the browser to check the results and create element selectors. That wasn't fun at all, I just wanted to create and run tests!
Now, with Frontend Robot it's just a matter of opening the browser, creating a test, and adding actions and assertions while there is a live test session, which immediately shows the results on screen. Paste the smart hook URL in your deployment script and have the test running at every deploy. It's as simple as that.
Frontend Robot also provides a nice set of goodies that will help with writing tests and debugging, such as:
- DOM Snapshots
- Smart CSS selector picking
- Time travel
- Variables
Frontend Robot was designed from the ground up to provide the best possible Tester Experience, studied in every detail to remove any obstacles or complications to make writing tests less frustrating and fun again. If you are mainly after simplicity and an enjoyable testing experience, then Frontend Robot is the right tool for you.
2. Ghost Inspector
Born in 2014, Ghost Inspector was created by Justin Klemm from similar motivations to those that pushed me to create Frontend Robot. I definitely consider Ghost Inspector one of the main source of inspirations for Frontend Robot. If you are curious about the story behind Ghost Inspector, you can take a look at Justin's blogpost Why Did We Build Ghost Inspector? where he highlights the main pain points that Ghost Inspector tries to solve.
I consider Ghost Inspector one of the most seasoned and stable tools in this list. If you look at their documentation, it's clear that they worked closely with their customers over the years to build a tool that can cover almost every major need in test automation. From screenshots comparison to file uploads and testing emails, they have almost everything you need to write a complete test. The only drawback in my opinion is a UI that looks outdated and a Tester Experience which can be improved.
3. Cypress
While the other tools in this list bet on codeless test authoring, Cypress focused mainly on reinventing the traditional code-based testing framework. Cypress built a test runner from scratch, allowing to provide some interesting features out-of-the-box, such as:
- Spies, stubs and clocks
- Network traffic control
- Time travel and DOM snapshots
- Real-time reload
Being code-based, plus having a custom runner built on Chromium, allow developers and testers to write very advanced and complex test routines, which would be impossible to create with the other tools in this list.
Despite being code-based, Cypress strives to provide the best Developers Experience in the industry. In fact, their API is crafted for maximum efficiency and their documentation very detailed and easy to read. They even have a dedicated Developer Experience team!
The Cypress team has big plans for the future, however being still a young product, it has some limitations. For example, as of today it only support Chromium-based browsers. Plus, it's not a 100% cloud-based solution, something that can be a good news for some but a bad news for others. In fact, Cypress has an open-source runner that everybody can use for free, and a paid dashboard service to collect and organize test results. However, it's still responsibility of the developers/tester to setup the tests to run in a separate CI (Continuous Integration) provider.
4. Testim
Testim launched in 2014 and it's currently backed by $20M in funding, which makes it one the most well funded products in this list. Testim is a hybrid tool which allows to both record tests using their Chrome extension or using code written with your favourite IDE. Testim is marketed as an AI-powered testing tool, featuring self-healing selectors which should require less maintenance. But to be honest with you, besides this feature, I'm not sure how much other AI is in the rest of the product. Said that, Testim looks like a solid, featureful product, which strives to provide a simplified and more enjoyable testing experience.
One interesting feature is the ability to integrate with third party browser grids, such as Saucelabs and Browserstack allowing you to run your tests on a variety of desktop and mobile browsers.
For some, the only real negative side of Testim may be its price (at the time of writing starting at $450/month) which will make it off limits for many individual developers or small companies.
5. Mabl
Mabl is the most well funded tool in this list with a staggering $30M invested in it. Founded by two ex-Google employees, Mabl defines itself as a "unified DevTestOps platform". In other words, Mabl is a cloud-based codeless tool that simplifies the entire testing experience, from setting up tests to running them in their Google Cloud based grid. Their test recorder, called the mabl trainer, is a Chrome plugin which allows to transform browser actions into test steps.
Mabl is the only product in this list to natively support cross-browser testing on their own cloud, with Chrome, Firefox, IE, and Safari being the available browsers. Like Testim, also Mabl has some AI embedded in it, allowing self-healing tests which adapt to UI changes, resulting in more robust tests that break less often.
Mabl doesn't provide public pricing information, so my guess is that it's mainly targeted at medium to big businesses.
Conclusions
The products we've just analyzed are, at the end of the day, very similar in their goal. They all aim to improve the Tester Experience making the tester's job simpler, more productive and more enjoyable. However each one of them is targeted at a specific group of testers/developers. Frontend Robot and Ghost Inspector are the products to choose if you don't want any fuss or extra complications with writing your tests. Cypress can be a good choice if you still want to write code for your tests and need more powerful features and control. Testim and Mabl are mainly targeted at larger companies, who need more support and enterprise features, at the cost of a more expensive subscription.
There are many other products that I've left out from this list, and that doesn't mean they are less good. However, there is an important player that I've left out just because it's still very young and in active development, and this is the new Selenium IDE, which recently took the place of the old, Selenium IDE. The team is working on a new Electron-based editor, which promises to bring some interesting features.
As always, if you have any questions or feedback you can send me an email at mario@frontendrobot.com.
Top comments (0)