Why I rarely use `getByRole`: Testing Library and the first rule of ARIA

Note: I've gotten some feedback from a Testing Library maintainer, Matan Borenkraout, about these ideas. I haven't gotten around to incorporating that feedback yet. He had some good examples where ByText or even ByLabelText might not be able to find some elements due to how the label is created or how the implicit role is determined, so what I'm describing here might not handle all cases nicely. I don't think this blows up the premise I'm describing, but I'll be thinking that feedback over and expect I'll be updating the content below in the future.

**

Intro

To explain the reasons that I mostly stopped using ByRole locators and started specifying elements, I want to zoom way out and talk about the various pieces of the puzzle. Here's a rough table of contents.

• Background
• What problems Testing Library is intended to solve
• Why ByRole is too vague
• What makes something accessible
• What roles even are
• The first rule of ARIA (Accessible Rich Internet Applications)
• Semantic HTML, TypeScript, and using the platform
• What we expect developers and testers to know when writing and testing their code
• Situations where I still use ByRole
• Why I love ByLabelText
• The use of Test IDs
• Overall conclusions

Since this argument is all in the details, this is a relatively long post, so I'm skipping some background on the underlying ideas of front-end testing and locating elements. If you're new to these topics, I have a 2022 post on CSS-Tricks called Writing Strong Front-end Test Element Locators that starts more from the ground up. But for now, I'm going to assume you have some familiarity with how front-end testing works and are interested in best practices for locating elements.

And of course, feel free to skip ahead to the conclusions if this is all familiar stuff.

Background

Testing Library provides a great set of accessibility-focused tools to locating elements when writing tests. It was created by Kent C Dodds, one of my longtime favorite educators on front-end and testing topics. Since the initial release in 2018 as "React Testing Library", it's grown in popularity beyond its React component testing roots. It now has plugins for all major test runners and JavaScript frameworks, and in 2022 some of its patterns were adopted in Playwright's test runner as the recommended way to locate elements.

I do almost all of my testing in Cypress (and for clarity: I work there). I've added Cypress Testing Library to multiple projects over the years, and it's my go-to for a lot of testing tasks, especially when testing forms, using the ByLabelText locator.

One thing I've noticed over time is that I've mostly stopped using the main workhorse of Testing Library, the ByRole locator. When I wrote some guidance for my team a few years ago, I recommended not using it in most cases. I also mention this as a quick caveat when I teach testing or accessibility topics (for example, it comes up in passing in a blog post about user-focused locators that I wrote last year).

I haven't given it much thought over the years, it's just one tool in the Testing Library toolbox and it has its place. My recommendation was driven by what I expect ByRole to accomplish. But recently I've noticed many strong recommendations to use ByRole as the best way to find an element in a test, citing the reason that this helps ensure the underlying application being tested is accessible.

The short version

My argument in this post will be that getByRole is not strict enough to ensure that a given accessible role is correctly implemented. Locating elements this way can create misleading scenarios where your tests imply that an element is accessible, even if it's not. We'll get into more detail on that below, but first I want to give you the TLDR on what I do instead.

The recommended Testing Library approach to find and test a button is this:

screen.getByRole('button', { name: /submit/i }).click()

What I prefer to do is locate an element by a combination of two things that, if they are changed, should fail a test:

1. The element's specific tag name - button or a, for example
2. The element's accessible name - "Save" or "Home", for example

In order to do this in Testing Library, for buttons and links, I would use the ByText locator, like this:

// get a "Submit" button
screen.getByText('Submit', { selector: 'button' })

// get a "Home" link in the navigation element
screen.getByText('Home', { selector: 'nav a' })

I do something similar using the following format of the contains command in Cypress. I tend to only look for another locator when this isn't going to get me the element I need:

// get a "Submit" button
cy.contains('button', 'Submit')

// get a "Home" link in the navigation element
cy.contains('nav a', 'Home')

The long version

The rest of this post is really just me trying to push at this from a few different angles and think everything through. I often find I have some opinion that I think is right, and then when I try to write about it, I realize I don't know the exact source of why I think something, or what things might just be assumptions. So it helps to document all the pieces, and it lets somebody else cross-check the ideas.

What is Testing Library for?

The basic premise of Testing Library is summed up by this quote from the home page:

The more your tests resemble the way your software is used,
the more confidence they can give you.

The problems Testing Library solves are described in two ways. The first is about avoiding testing implementation details to have better confidence that test failures matter:

You want tests for your UI that avoid including implementation details and rather focus on making your tests give you the confidence for which they are intended.

The second is about how not testing implementation details makes refactoring less painful:

You want your tests to be maintainable so refactors (changes to implementation but not functionality) don't break your tests and slow you and your team down.

The issue of test stability over time, and avoiding testing "implementation details", had previously often been addressed with a Test ID based approach, which maximizes the stability of tests. Testing Library's authors make the argument that using [data-testid="login"] style locators ignores important user-facing aspects of the elements being tested.

This is true. Test IDs intentionally and explicitly ignore the nature of the elements being used. We'll talk more about them at the end -- there's no disagreement there.

My issue with ByRole locators is the same as the problem with Test IDs, just in a narrower area. If either of these are your only locators and no test ever specifies the specific elements you expect, developers have too much wiggle room to break accessibility while keeping tests green.

ByRole locators are too forgiving for safe refactoring

The Testing Library query documentation says this about the ByRole locator:

This should be your top preference for just about everything. There's not much you can't get with this (if you can't, it's possible your UI is inaccessible).

The Playwright documentation puts it in similar terms:

We recommend prioritizing role locators to locate elements, as it is the closest way to how users and assistive technology perceive the page.

Both give examples of doing something like this:

screen.getByRole('button', { name: /submit/i }).click()

The underlying idea I take from this that if an element passes the "role check", it's likely to be accessible: locating an element by its role is intended to achieve the goal of safe refactoring - where "changes to implementation but not functionality" do not break your tests.

However, even though a role is part of an element's "identity" in an accessibility sense, it's not enough. We could refactor an accessible component in a way that fully breaks all accessibility, but still have a green test that uses getByRole. Many implementation changes that leave role intact are actually critical changes in functionality.

Here's an example of this kind of refactoring:

What makes a button accessible?

HTML elements all have implicit roles that describe the nature of the element to people with disabilities, accessing the page using assistive technology. Buttons have the button role, anchor tags with href attributes have the link role, input elements with type text have a textbox role and so on.

When it comes to interactive elements, these roles are not usually useful by themselves. What is useful about them is that they represent an overall contract that element will fulfill. To use the button element as an example, here is at least a partial list of the accessibility "contract" that browser vendors fulfill for a button:

• It has the implicit role of button so that users know what it is - a screen reader, for example, would announce it as a button
• It has a default browser button style to visually distinguish it from surrounding content
• It is placed in the tab order of the document and can be focused with the keyboard
• When focused it receives a focus outline
• It can be activated with a mouse click, or the enter key, or the space bar
• If placed inside a form element, activating the button by any of these methods will submit the form (because its default type is submit)
• It will receive specific styling, customizable by the user, in Windows High Contrast Mode (used by people with certain visual disabilities)
• It has default browser styles for hover and "pressed" states

That's a lot of behavior that the browser implements for a humble button. The browser also manages this button's presence in the Accessibility Tree, which is how assistive technology like a screen reader can announce the button's existence, what it does, and how the user understands what actions are possible.

And here's what may be surprising: the browser only does this stuff for real button elements. Adding role="button" to a div or a span does not trigger any of this in the browser. If role="button" is used on say, a div with an onClick handler, the entire contract of a button must be implemented by the developer, using extra HTML attributes, CSS rules, and JavaScript listeners. And even then, there will still be gaps - especially in Windows High Contrast Mode, which ignores role="button".

This is the main reason I avoid ByRole locators most of the time. Let's look at this locator again:

screen.getByRole('button', { name: /submit/i }).click()

This will absolutely find a true button element, because of its implicit role, but it will also have no problem finding a div with role="button".

Since these two elements can be very different from an accessibility point of view, it does not seem at all safe to depend on ByRole locators during a UI refactor. Accessibility can change radically depending on the underlying HTML, CSS, and JavaScript, even when roles are preserved during what is intended to be passive refactoring.

So, in the cases where there is a semantic HTML element available, I prefer to just specify that element, and that way the tests will fail if somebody refactors to a pattern that uses non-semantic elements to recreate existing HTML functionality.

This element-level approach also nudges developers towards accessibility success, by forcing them to either follow the first rule of ARIA in their refactoring, or modify the test code if they change the implementation. That means that in a pull request, for example, a reviewer would see unexpected test changes resulting from a refactor (where tests shouldn't change) and hopefully ask why semantic HTML was no longer being used. There's at least an extra layer of protection there.

The first rule of ARIA

ARIA stands for Accessible Rich Internet Applications and it's a specification that teaches us how to create online experiences that are likely to be functional for people with disabilities.

The first rule of ARIA is:

If you can use a native HTML element HTML51 or attribute with the semantics and behavior you require already built in, instead of re-purposing an element and adding an ARIA role, state or property to make it accessible, then do so.

The reason for this is to take advantage of what the browser provides, which is highly likely to work correctly with assistive technology, instead of doing a custom implementation of the same functionality.

One way to think about this is: the absolute best you can do for users when implementing a custom button, is to faithfully recreate everything the browser already does in the "button" contract, except your version won't support High Contrast Mode.

There is no value to writing and owning code that reimplements browser functionality - your company doesn't need it, you don't need it, and your users definitely don't need it.

Related practices

A lot of Testing Library advocates are also fans of other good practices, like using explicit types with TypeScript.

A ByRole locator essentially says that there is only one required property of an element, and everything else about its accessibility contract is optional - the test will accept whatever has that one single property. Specifying the correct semantic element, on the other hand, is a way to force the test to only except something where the browser is implementing the full contract. Following the first rule of ARIA in our tests, as well as our code, gives us the most "type safety" in this sense.

The alternative solution is to assert all of the expected accessibility behaviors in your tests, in addition to locating the element by its role, which is absolutely the way to go if you own the code that implements that functionality.

Another good practice is in web development is "using the platform", not fighting it. The first rule of ARIA perfectly aligns with this: using correct semantic elements allows browsers and operating systems to meet the needs of disabled users, and provides the best compatibility with assistive technology. When people say the web is "accessible by default", semantic HTML is at the heart of this statement.

Developers can still override things and break semantic elements of course, but semantic HTML is still a better foundation and more likely to be accessible. So having our tests depend on this aspect of code seems worth it.

What we expect developers to know

Because of the first rule of ARIA, I don't see any valid use case for role="button" in code. We should only see the role attribute being used for custom widgets that do not have native HTML equivalents.

The Testing Library docs point out something similar to the first rule of ARIA, but don't quite make the same conclusion:

Please note that setting a role and/or aria-* attribute that matches the implicit ARIA semantics is unnecessary and is not recommended as these properties are already set by the browser

This recommendation makes clear that it is expected and desired that developers know they should prefer maintaining the native semantics of HTML. Since they need this knowledge to choose the correct accessible implementation of a given role, we should take the mystery out of it and just specify the element that should be there in our tests. Then there is no need to hope that future developers know all the same nuances about accessibility. They can learn them when their non-semantic HTML causes a test failure.

When I still use ByRole

I'll take a ByRole selector any time for roles like menu, menuitem, tabpanel, treegrid... there are lots of useful accessible roles that are not yet implemented in standard HTML. Selecting them by role is a great idea, especially as it allow you to refactor into semantic HTML without breaking a test, when elements with those roles do make their way into browsers.

There are also some rare cases where ARIA roles are needed, even when there is overlap with semantic HTML elements. This might happen when working around some specific browser bug with the native element, for example, where owning the implementation of the accessibility contract is worth it because it lets you do something you can't do otherwise.

Why I love ByLabelText

OK, enough about why ByRole is not my thing. Let's talk about some good stuff. Here's my favorite way to locate a form field in Cypress:

cy.findByLabelText('Username').type('myusername')

This is not built in to Cypress -- I use the Testing Library plugin. It's elegant, readable, concise, and requires form fields to have a specific programmatically associated label present, which is also needed for assistive technology.

If I wasn't using Testing Library, I might define a custom command that knows how to find labels in my application, executing something like this:

cy.contains('Username').next('input').type('my-username')

But this depends on knowing the exact DOM structure, and here we don't care about that. There are a lot of correct ways to label a form input, it doesn't matter too much which way is used. One good thing is that here we are specifying a real input element, but Testing Library can do that too if we want to, using the selector option:

cy.findByLabelText('Username', {
    selector: 'input'
  }).type('myusername')

This option is explained in the docs:

If it is important that you query a specific element (e.g. an ) you can provide a selector in the options.

This is also presumably why ByText also takes a selector option. Even though we want to avoid testing implementation details when they don't matter, Testing Library has built-in options so that you can still specify them when they do matter.

In general the risk of developers creating custom inputs built out of divs and spans is pretty low, so I don't do that very often in practice. But it might be useful to use this pattern at the component test level, to give the developers instant feedback if they decide to get creative and reimplement an input out of a custom div.

So why don't we pass this selector option to our ByRole locators, where the risks of using non-semantic elements are higher? Primarily because ByRole doesn't support a selector option. That's for a pretty good reason as far as I can tell, since ByRole is specifically intended to avoid specifying the implementation.

It would be pretty pointless and circular to write code that looked like this:

getByRole('button', {
  name: /submit/i,
  selector: 'button' // ⚠️ not possible, just an example
}).click()

In the above format, the selector property would just override the role. On the other hand, in the ByLabelText scenario, since many elements can have similar labels, passing a selector option to further narrow what is acceptable makes more sense. You could have a Shipping and a Billing form on the same page with the same labels. You could have a select and an input with the same label text.

Or you might want to pass something like a [data-testid="header"] input selector if there are two instances of similar forms on the same page.

Oh yeah, what about data-testid?

As a big believer in accessibility, and a longtime user of Testing Library, I want to take a second to stick up for the good old Test ID pattern, which still has its place and can be used in conjunction with specifying the element, or the role, or anything else.

I strongly believe you need to cover the accessibility of your app with automated tests, and that means specifying the parts of the DOM that matter. The HTML we ship to users is our product and we should all pay attention to the actual code we ship. There is an enormous, expensive, complicated system of servers, databases, and various tubes often involved in sending it from us to our users. We might as well send the right HTML.

How I feel about Test IDs is this: if you've already specified the correct HTML output in one place, for example in component tests, then it's fine to use Test IDs elsewhere if you want.

Especially if you have a separate QA team, not as familiar with what the "right" HTML and roles are, we don't need to block them from workflows using generic helper locators, as long as there are tests owned and run by the developers writing code that will fail if HTML accessibility is broken. You must make sure your something in your system will fail if accessibility regressions happen.

On a personal level I'd be happy if everything went red when accessibility broke, because it's a critical thing for me. But as long as accessibility is specified, tested, and owned, then testing some workflows using Test IDs might actually have some upsides for certain teams.

It's also the case that many applications are not built in an accessible way where roles, or even elements themselves, are appropriate for their functionality. Those apps still need to be tested, and if it's not easy or possible to remediate things on the fly, a generic Test ID is a better choice than something that, say, specifies the current wrong role that something happens to have, and then causes confusion.

Test IDs also come in handy when disambiguating different instances of similar elements, and in a few other situations. It's just important to know what they do and do not cover.

I'm hoping to make a PR to the Cypress Best Practices documentation, which already endorses both Test IDs and Testing Library, to get into some of this kind of detail about how to choose the approach. But as you can see, what I've got here currently long and unwieldy. I'm hoping writing this post will help me form a much shorter proposed update to the docs.

If you've ready this far: thanks! And please let me know if you have thoughts or feedback. Am I missing something? Is the ByRole locator awesome in a way I'm not accounting for?

Conclusion

Here's the understanding I'm working with at the moment:

• Semantic HTML, when available, is always preferred over custom role attributes for accessibility. This is the first rule of ARIA.
• While users do perceive the role of an element, what they they depend on is its entire contract, so a ensuring a role exists is not enough to guarantee behavior doesn't change in refactors.
• This means we should not use a ByRole locator if the role we are selecting on can be fulfilled by a native HTML element.
• Instead we should specify the HTML element itself in our tests. It will help developers do the right thing, and provide better feedback when refactoring, giving you one extra layer of resistance to accessibility bugs.
• We can do this in Testing Library by passing a selector option to our ByText and ByLabelText locators. And we can likely do something like it in our framework of choice even if Testing Library is not available, such as with cy.contains('selector', 'text').

Writing tests this way aligns your tests and your development practices with the first rule of ARIA, which will give you a great foundation for building accessible apps and avoid some common accessibility pitfalls that trap even the most well-intentioned of us.

Last note: Testing Library is still great and I'll still add it to every project to augment whatever I'm doing.