Mobile Web Accessibility Testing Using Appium

A trend being seen in UI test automation is the use of hybrid testing frameworks such as Appium to test mobile versions of applications. Appium as described by its website is "an open source test automation framework for use with native, hybrid and mobile web apps."

For the longest time Appium had been used to test native mobile applications or even hybrid applications made in frameworks such as Cordova. However, over the course of this year, more and more development teams have learned you can also simulate the mobile browser and test your mobile site in an actual device AND still write automated tests for it.

This has left more development teams asking, "How can automated accessibility testing fit into this testing model?". The answer is actually a lot easier than you may think! Let's take a look at how we can integrate accessibility testing into your Appium test cases!

How It Works

At its core, Appium uses WebDriver protocols to drive iOS and Android applications or web applications. Since it uses WebDriver protocols, this means we can use something like axe-core to run as asynchronous JavaScript in the browser, and get the results back!

It can also be used across a variety of different development languages, such as JavaScript, Python, Java etc. So whatever language you are using you can use axe-core to execute in the browser.

How does this look in practice? Let's take a look at a JavaScript setup and how Appium and axe-core can work together!

Implementing Axe with Appium

The first thing to do in your project is install @axe-core/webdriverjs package. This package is an axe-core integration that actually works in tandem with WebDriver (Selenium) to do all of the axe-core injection into the browser for you!

Once you have it installed, add it into your test spec like so:

const AxeWebdriverJS = require('@axe-core/webdriverjs');

Now created two objects axeDriver and driver at the top of your spec, along with two other objects that contain the desired capabilities and local server URL for Appium.

let axeDriver, driver;
let desiredCaps = {
    'automationName': 'xcuitest',
    'platformName' : '<platform_type>',
    'platformVersion' : '<version_platform>',
    'deviceName' : '<device_name>', 
    'udid' : '<id_of_device>',
    'browserName' : 'Safari'
};
let localServer = "http://localhost:4723/wd/hub"

Next, in your setup function, take your driver that has your server that is using Appium and the desired capabilities of it and wrap it with the AxeWebdriverJS package.

before(async () => {
  driver = await new selenium.Builder().usingServer(localServer)
            .withCapabilities(desiredCaps).build();
  axeDriver = await new AxeWebdriverJS(driver);
});

Now that your axeDriver object is ready for use, create a test case that checks to see if the mobile web page 'is accessible'. To do this, go to the page of your choice using driver.get(url)and then create create an object called axeResults and set it equal to axeDriver.analyze().

When we run the analyze function it runs the axe-core ruleset against the page in its current state and returns back the results of the scan. We can now check the axeResults object and use it in an assertion to check that the length of the violations is equal to 0, meaning there no accessibility issues on the page.

it('is accessible', async () => {
  await driver.get('https://www.spacejam.com/1996/');
  const axeResults = await axeDriver.analyze();
  assert(axeResults.violations.length === 0, "a11y issues found!");
});

You can now replicate the exact same setup in the rest of your test specs and have an accessibility test case in each one!

You can see a full example project here: https://github.com/Steady5063/appium-axe-example

In Summary

As you can see, adding automated accessibility testing to your Appium mobile web testing is quite simple. By adding accessibility testing to your Appium tests you can now:

• Catch up to 30-40% of accessibility issues in your applications mobile state
• Cut down on manual testing
• Add accessibility into your CI/CD metrics dashboard

All these help make accessibility testing easier and more efficient for your development team and development process!