The use of automated accessibility tools has gained massive popularity for its ability to quickly and seamlessly get development teams to test for accessibility. One problem however, is that each day it seems like there is a new automated accessibility tool that is being touted or talked about.
This has made the concept of using automation a bit fuzzy and somewhat difficult to navigate as to how to best use accessibility automation. It has even driven some development teams to say that automation isn't practical since there are so many options!
As web or mobile developers your job can already be packed full of processes, code reviews, and more information that your brain can handle. So what is the most practical way in which we can bring automated accessibility into the development lifecycle, while effectively making it apart of the development culture? Let's dive in!
Phasing In Automation
The most practical way for accessibility automation to become a cultural habit, is to phase it into the development lifecycle piece by piece. Just like anything new, we cannot simply "flip a switch" and tell mobile or web developers to "start using automation".
We need to effectively and efficiently add it into the development lifecycle phase by phase in order to gain developer buy-in and trust.
Phase 1: Linting
The first phase, and quite frankly the easiest, is linting. A linter is a tool that statically analyzes source code to flag programming errors, bugs, stylistic errors.
If you are a mobile or web developer it is a no brainer to have linting for your code even outside of accessibility. However, within most linters exist accessibility rules. Yes, the rules are few in numbers, however using this to enforce accessibility gives dev teams:
- Catching issues as you build content
- Building accessibility knowledge for your developers
- Fix the basic of basic issues early
If you want more information on linters that fit your web development needs check out a previous article I wrote on Web Development Linting
iOS and Android both have linters available to use to check your code for accessibility issues. Androids linting is much more robust than iOS, but iOS does allow for custom linting rules (within reason)
Phase 2: Automated Accessibility Libraries
Once linting has gotten a strong presence on your development team, the next thing to do is start building your UI automated tests for accessibility. There are two parts to this, the first one being finding an accessibility library.
An automated accessibility library has built in checks that generically scan your UI content for accessibility issues. It is as simple as adding the library, making a test case that checks for issues, and then return pass/fail if it finds any issues. The checks will be more in depth than linting as it will be a part of your built application in mobile or web.
Automated libraries have many benefits including:
- Consistent set of rules running on all new content
- Ability to build gates on commit or push
- Quick and easy to add into currently UI testing structure
Web Automated Libraries
- Axe-core - Axe-core is one of the most popular libraries that has MANY different integrations into testing frameworks
- PA11y - It runs accessibility tests on your pages via the command line or Node.js, so you can automate your testing process
Mobile Automated Libraries
Mobile is a little bit different. Automated libraries that are free are not as abundant as web, so adding them to UI tests is mostly done themselves (see next phase). The other options is to take advantage of the automated scanning tools in XCode (Accessibility inspector) and Android (Google accessibility scanner)
Phase 3: Accessibility Regression
The final phase of automation is creating accessibility regression tests. Truth be told, it may take development teams a bit to get to this phase and that is okay!
The idea behind writing accessibility regression tests is that the development team can go above and beyond a standard accessibility testing library. They can write their own tests that check for the accessible functionality of their content.
For example, I could write a web test case that checks to ensure that aria-expanded is properly being set, every-time a user clicks on expand/collapse section. In mobile, I could ensure that when a modal or popover appears that focus is put into it.
To get to this stage offers large benefits including:
- Easier debugging of accessibility issues
- Including accessibility tests as part of definition of done
- Ensuring coded content is functionally accessible.
Whew! As you can see, accessibility automation can be a lot. Even in the phases that were just broken down. But taking it phase by phase, only helps to limit the push back that will inevitably happen.
Each of the phases talked about above have no time limit or time frame. They take as long as they need to take. Each development team, company, or even single developer is different! The end result is still the same, building a culture of accessibility that lasts, and developers that give a damn about the content they create!
Top comments (6)
Great article- i use AxeDevTools IGT+ WAVE + Google Lighthouse accessibility score when app ready post UIT and it helps reducing bugs in UI. Similarly Google Accessibility Scanner for Android apps and Accessibility Inspector of XCode in iOS to catch on native.
One thing worth to mention is that Google Lighthouse uses Axe, but limited to 25 (if I recall correctly) rules.
True- None of these tools represent 100% of real user with accessibility needs hence using those screen reader for laptops (NVDA,JAWS) and voice over,talk back in mobiles are critical and that helps in finding essential defects which are otherwise unattended by scanners. Example: FBN three finger movements of users in firefox using voice over in mobiles are not covered in any scanners and not discoverable by code audits hence manual test matters a lot
Meanwhile none of the tools ready with WCAG 2.2 new rules (as of 25Feb2022) hence audits are just pre-requisites and not actual tests in my test strategy.
Note on Axe: AxeDevTools IGT got better coverage when comparing with lighthouse but looking forward to see what we have from Axe Auditor and those bundle of new tools from Deque in 2023
Yes, some cases require manual testing. The complete Accessibility audit is a combination of automated + manual testing.
P.S. Great post.
This article was curated as a part of #76th Issue of Software Testing Notes Newsletter.
Great article Mark on accessibility automation. Your article is informative, comprehensive, none-bias, and easy to follow!