DEV Community

Suzanne Aitchison for The DEV Team

Posted on

What Cypress E2E testing has taught us about our code

A little over a year ago Forem introduced End to End testing to the app using Cypress. Since that time we've definitely evolved our understanding of Cypress, but we've also learned a lot about our app too. So much so, that I wanted to share some of the things Cypress testing has taught us 😄

Some background

We use Cypress alongside cypress-rails and Cypress Testing Library.

Testing Library gives us a range of enhanced queries that help us focus on testing in a way that closely mirrors how users interact with the app. Instead of targeting page elements with e.g. CSS selectors or IDs, we lean towards finding elements by their role and accessible name.

When I talk about testing with Cypress at Forem, I mean specifically testing with Cypress and Cypress Testing Library 😄

A greater awareness of accessibility

I've made no secret of personally being a huge fan of Testing Library for the impact it can have on accessibility testing and awareness. It therefore isn't a surprise to me that we have learned a lot about the accessibility of our app since introducing E2E tests with Cypress.

Uniqueness of elements

One of the challenges we've had with testing some older views in Cypress is discovering that the same element/name combination is present multiple times.

A great example of this is the "Follow" buttons throughout the app. We tried to target a Follow button in test with cy.findByRole('button', { name: 'Follow' } and found it incredibly difficult to target the one we wanted.

This immediately flagged itself up as an accessibility issue - users of screen readers would hear every follow button announced as "Follow"... but, follow who? 😅 Just as we were struggling to target the right button in test, a screen reader user would struggle to target the button of the user they wanted to follow.

Big shout out to community member @keshavbiswa for making improvements to the follow buttons accessible names after we discovered this!

The power of landmarks

We take advantage of Cypress' scoping abilities by writing code like:

cy.findByRole("main").within(() => {
  // Everything in here is scoped to the "main" element only
  cy.findByRole('heading', { name: "Dashboard" });
});
Enter fullscreen mode Exit fullscreen mode

In the example above, the within helper allows us to scope the test to all the HTML inside the main landmark. This can be really handy and powerful! For example, when we wanted to test the main feed navigation, we could do:

cy.findByRole('navigation', { name: 'View posts by' }).within(() => {
  cy.findByRole('link', { name: 'Relevant' }).as('relevant');
  cy.get('@relevant').should('have.attr', 'aria-current', 'page');
});
Enter fullscreen mode Exit fullscreen mode

We were able to go directly to the feed navigation landmark with ease. This is cool for testing, but it's also been great for building awareness and understanding of how important landmarks and their labels can be for screen reader users. Just like we could skip directly to the "View posts by" navigation in Cypress, screen reader users can skip directly there with their screen reader too! Great news for anyone that wants to get straight to the latest posts 😄

Headings, headings, headings

Headings play an important role in accessibility (I have an older post that digs into this a bit), and they've become really important to our tests too.

One struggle we had with Cypress was that sometimes when we followed a link to a new page, the new page wouldn't have loaded yet before Cypress tried to click some button/check for some behaviour.

For example, we had some test code:

cy.findByRole("main").findByRole("link", { name: "Test article" }).click();
// After clicking the link we can _sometimes_ accidentally get a reference to the 'main' element on the page we just left
cy.findByRole("main").findByRole("button", { name: "Share post" });
// Cypress fails to find the 'Share post' button inside the previous page's `main` element, and the test fails
Enter fullscreen mode Exit fullscreen mode

Since a main existed on both the first and the second page, Cypress would occasionally get a reference to the wrong one 🙈

One way that we prevent this type of flake now is to make sure we look for an element that is unique to the new page, before continuing our test steps. The example above would become:

cy.findByRole("main").findByRole("link", { name: "Test article" }).click();
// Wait for the correct h1 element
cy.findByRole("heading", { name: "Test article", level: 1 });
// Now we can be sure the main below is the right one!
cy.findByRole("main").findByRole("button", { name: "Share post" });
Enter fullscreen mode Exit fullscreen mode

Needless to say, this in turn has made us more aware of pages missing headings, or headings with the wrong level, duplicated, etc. All of which is helping us tighten up our accessibility.

Javascript races 🏁

Aside from these accessibility concerns, we've also become very aware of some side effects of how we handle our Javascript.

We try our best to make sure our pages load quickly, with Javascript asynchronously layered on after the first page load to enhance functionality. In reality, this means it takes some amount of time before some elements on the page are fully initialized and clickable.

For example, when the logged out version of the home feed first becomes visible, the author names don't yet have a click handler to trigger the appearance of the profile preview card - we initialize this functionality after the page first loads.

As a human, it's pretty hard to click the button before it's initialized (largely because the page load is pretty fast 😄), but it's technically possible. Certainly Cypress is speedy enough to do this sometimes!

We've been working around this with a variety of solutions, including using cypress-pipe which lets us retry a click until a certain condition is met. For example, in the case of the feed preview cards we can do:

cy.findByRole('button', { name: 'Admin McAdmin profile details' })
  .pipe(click)
  .should('have.attr', 'aria-expanded', 'true')
Enter fullscreen mode Exit fullscreen mode

This tells Cypress - click this button until aria-expanded is equal to 'true' (meaning the preview card has opened). Don't worry - it won't retry forever - Cypress has a default waiting period for a condition to become true, and pipe will only retry for that long.

It's definitely made us think a lot more about progressive enhancement, and what is and isn't usable from the very first moment of page load.

Want to know more about E2E testing at Forem?

We've been adding to our E2E testing docs as we learn and evolve our approach, including the "gotchas" we've found along the way - feel free to check it out!

I'd love to hear about your experiences with Cypress and Testing Library too!

Discussion (8)

Collapse
jeremyf profile image
Jeremy Friesen

I've had a long-standing slow burning dislike of most E2E test suites. They were always flakey and subject to very brittle conditions.

However, I've come around to Cypress (we're at least on speaking terms). Very much because of the select by role and label being far more useful than CSS selector.

I definitely go to RSpec (or MiniTest) first but having Cypress at the other end of the test spectrum feels quite good.

Collapse
andrewbaisden profile image
Andrew Baisden

Cypress is amazing! Hoping to use it a lot more in future projects now that I have learned it.

Collapse
s_aitchison profile image
Suzanne Aitchison Author • Edited on

Yay! I would definitely recommend pulling in Testing Library to use with it too 🙌

Collapse
andrewbaisden profile image
Andrew Baisden

Cypress and React Testing Library is a dream team up in my opinion 😁

Collapse
liviufromendtest profile image
Liviu Lupei • Edited on

Accessibility means making your website usable for as many users as possible.

Some developers might think that this only refers to users with disabilities, but that's not true.

Cross-browser testing is an important part of an Accessibility checklist, at least in the European Union.

Before choosing an automated testing tool, I recommend that each team check their analytics to see what browsers their users are using.

You cannot run tests on Safari with Cypress.

And 20% of users worldwide are using Safari as their main browser (I'm actually one of them).

browser stats

To raise awarness, we even wrote an article:
What Happens When You Don't Test On Safari

Collapse
s_aitchison profile image
Suzanne Aitchison Author

This is definitely true, and only a small subset of accessibility failings can be caught with automated tools at all. It's certainly not a cure-all (it's not intended to be), but I love how Cypress + Testing Library helps us validate some core user flows.

Collapse
pedromagalhaes profile image
Pedro Magalhães

This always requires a predicted data right?

Do you provide an interface somewhere in a CMS to re-configure the tests?

Cheers and thanks for sharing this!

Collapse
snelson1 profile image
Sophia Nelson

Cool!