Cypress vs.....

We switched to Cypress from Angular's Protractor and have absolutely loved it. It's modern, easy to use, and seems much more stable and less prone to random failures. (just in our experience of course) While there are some trade-offs, we personally didn't see those as deal breakers for our use case, and every testing framework you use is going to be that way.

We do use Cypress in my workplace and we're pretty happy with it. Even though you can still find some limitations compared to other more mature tools, its benefits widely compensate the small losses you may find.

I agree with @shiling, that you have decide what you're trying to accomplish overall. There is nothing wrong with RSpec and your current use to validate your base Rails infrastructure [ yes, I took a quick peak ].

I can only guess that you're looking to expand your functional testing coverage with UI based testing [ not truly E2E testing ]. Its definitely helpful for when you're changing foundational underlining code that shouldn't impact the user experience or UI components. However, as its been pointed out the intentions of the UI testing needs to be defined.

Now I can't give you insight into Cypress vs TestCafe, but I can give you my thoughts or insights for adopting GUI automation from a QE perspective.

• If the UI test creation audience is the developers and not a QE, then keep the test framework & language close to the daily used language(s) (easier adoption)

• Be definitive about separating automated test focus [unit tests, API tests & functional UI tests].

• UI test should validate/assert behavior and/or data interactions.

• Treat UI tests like a unit test. They should run independently. They should not rely on another tests success or failure.

• When UI tests are independent, all necessary data to complete a test must be seeded, created or found in the test environment [ key for a successful CI/CD/CT environment ].

• True UI E2E tests should only be built in special cases and run during integration or before a release.

For me it's Test cafe because of it being browser agnostic. You can have a look at surveyjs github on how test cafe is being used

Capybara with Ruby 😍
Cucumber as test framework.

Make sure to check out Cypress Testing Library for some useful commands.

The @testing-library family of packages helps you test UI components in a user-centric way.

The more your tests resemble the way your software is used, the more confidence they can give you.

🧐What's wrong with your current experience with rspec?

I could really go on about the pros and cons of different testing tools, but without knowing what's currently not working out for your team makes it hard to recommend something suitable for your requirements and preferences - because there's always a trade off when you pick a tool. 🙃

Before switching tools, I usually think care about what exactly do I want in my new tool. I try to resist jumping on to a bandwagon simple because it's popular. I knew what I disliked about Angular - it was hard to teach to junior devs - and picked Vue instead of the more popular React back in 2016. And recently, it turns out I didn't really like Elastic + Kibana for analytics because I want more flexibility with structuring data, so I'm trying out MongoDB Altas instead.

Figure out the following first:

Requirements:

• What browsers do you want to test for now? Cypress and Puppeteer only supports Chrome.

Commitment level:

• How much time does your team want to commit to creating and maintaining the tests? Selenium requires a lot of commitment to learn to write and setup and maintain the infrastructure.
• Do you have time to commit to setting up pipelines to ensure tests are automatically ran? Tests that are run manually will certainly be forgotten - it's important to make sure they run automatically. Nearly all FOSS testing tools will require you to DIY integration with CI/CD. Most paid tools will offer built-in Scheduling and Monitoring capabilities.

Preference:

• Does your team prefer BDD-style?
• Does your team want the tests to live beside UI code in Ruby on Rails? Cabybara + RSpec is the popular choice. If your team is comfortable with Javascript, there's a lot of variety to pick from.

Other factors to consider:

• How fast do you want the tests to run? Cypress is pretty damn fast, so is Puppeteer. I believe both of them utilises the Chrome DevTools Protocol instead of WebDriver protocol which makes it fast, but you'll lose some accuracy in terms of simulating actual interactions instead of syntactic interactions.
• Do you want screenshots for every test and every step? Most tools require you to manually add steps to capture screenshots.
• Do you want visual testing?

While @picocreator and I are the creators of UI-licious ourselves, we won't recommend it if it doesn't suit your style. :) It's designed to be easy for non-technical folks, whilst flexible for Javascript programmers, and supports all browsers - but some folks don't like Javascript, others want tests to run blazing fast, and plenty of folks want it free. ¯_(ツ)_/¯

While facing a similar situation and trying several tools, I ended up making github.com/angrykoala/wendigo with the intention of making frontend+e2e tests as simple as possible

It is basically a wrapper of Puppeteer (with all the goodness and bad of Puppeteer), it's syntax is similar to what you would find with cypress but way less opinionated over how to write your tests and what other tools you may use (which is the main downside I found with Cypress). Not sure if it may help you but you may want to give it a look as an alternative

I gave a Talk about Cypress in Berlin, Germany. Please have a look. It has slides but not the code samples of demo.

Cypress e2e testing : My talk at Node.JS Meetup Berlin in Germany 💥

Shahjada Talukdar ・ Oct 17 ・ 1 min read

#node #cypress #javascript #testing

I like cypress.
The code is pure js and looks very native to me.
Developer experience is cool and so easy to use.

I guess the question is what is the gain? Convenience?

Doesn't a Selenium-based driver ensure you can test cross-browser and since this is an open-source platform where we need to be inclusive to browser and platform choice that we'd choose Selenium which is going to give us that coverage?

Is Capyabara + Cucumber really that bad? How slow would it really be? I have a project with 40 features and it takes 1-2mins to run.

I would say before introducing and E2E tool into the codebase we get specs moved to where Rails developers expect them to be. I have little problem working with the codebase quirks but let's ease it for juniors.

My main issue with cypress is that it only supports Chrome. For me this is a deal breaker.

If you need to test iframes, selenium is the best option to work with 👍🏼 I have tried using Cypress, Katalon Studio, etc. but I got stuck because they weren't compatible with iframes in some way or another. Maybe an option would be to use Puppeteer but it only uses Chrome.

I used Cypress on a project and loved it, but I have a hard time selling it to stakeholders because it isn't cross browser.

Chrome is rarely the browser with weird bugs so testing there is considered happy path. :/