When a development team sets up their UI automation tests with accessibility, they tend to think they are done!
"We included accessibility as part of our UI testing process, so we can now track how accessible we are!" However, if you asked most dev teams what they plan to do with results, the answers can be a mixed bag.
If your development team wants to truly build and sustain accessibility in their environment then you need to maximize the value of the results from accessibility testing! Lets dive into how!
Consistency is Key
The first thing consider to maximize your results is consistency. There are many accessibility libraries that exist and sometimes it can be confusing which one to start with. If you choose a particular ruleset such as axe or pa11y for the tests, then stick with it!
Some development teams will use one ruleset for one level of test and entirely different one in a different level of test. This can cause confusion and different results. It may seem simple, but different results can make learning about accessibility difficult for some developers who parse the results. One ultimate goal is to educate developers as they code, and having those consistent results can help ensure this!
Enforce It
If your development is up and running and fully using automated accessibility in UI tests, the best action to take is to enforce the issues that arise from the tests.
Code Reviews
One way in which accessibility in the development process can be improved is through code reviews. A lot of development teams currently have an "honor system" for automated tests or manual accessibility testing, in which the developer claims they did those steps. Which is why they tend to continue to have issues and not see a downward trend.
Adding enforcement of critical and serious accessibility issues from automated tests in code reviews can help ensure that teams fix the issues proactively before they check in code!
Github Actions (Quality Gates)
Another form of enforcement is through github actions! Github actions allow you to run specific actions on commit of new code.
For example, I am a developer who creates a component. I finish and I create a PR. If your tests are properly setup, a github action will run and check the accessibility of the new content you just created!
Github actions are simple and effective to setup and can easily make enforcing the automated checks on every single PR!
For an example of a working Github Action, you can visit my personal Github accessibility page!
Linting
Linting is another simple way to enforce accessibility issues being fixed before release. Although separate from the UI automated tests, linting checks are so simple that they can help your dev team keep issue counts down!
Accessibility linting has many benefits that can greatly improve your development team, including:
Catching issues as you build HTML content
Building accessibility knowledge for your developers
Allows your team to build an accessible gate on commit
Customizable to the framework you are developing in
Reporting Results
The final thing to maximizing results is gathering reporting from your tests. Most development teams tend to create a standard check and check if it fails. However, with accessibility defects there tends to be more data needed in order to fix issues.
There are a number of ways to ensure you get the most data out of the tests your create.
Axe HTML Reporter - Generates a nice report from the axe results objects for teams to use to see the state of the content tested.
Axe pretty print console logger - A console logger that puts the results in a nice summary table.
Jest/Mocha HTML report - You can take advantage of out of the box reporters from test frameworks like Jest/Mocha if you properly label your test cases
In Summary
If your development team has fully integrated accessibility testing as part of their UI automated tests, then it is time to get the most out of those results. Using the suggestions above, you and your team can build a culture of developing content with accessibility in mind!
Top comments (0)