A few weeks ago we brought you a rundown of our five favourite automation testing tools for Android, and the response was overwhelming. So we’ve written a follow-up article, this time for iOS developers.
The testing challenges vary slightly for Android and iOS testing. For one thing there are far more Android models out there than iOS, which means lots more potential bugs. Also, the two variants have their own default languages, which further complicates things.
But, on the whole, the challenges are similar. Whether you’re battle-testing an iOS or an Android app, you’re looking for automation testing tools that offer robust performance and quick, reliable results. If the testing solution offers cross-platform and multilingual functionality, then that’s a major bonus.
Our mobile team have used practically all the different testing tools available, and it was tough to whittle them down to five. But, after plenty of consideration, here’s our final list.
1) Appium
Developers have come to rely on Appium as one of their favourite testing tools. It is popular among both native and hybrid mobile app developers, thanks largely to its flexibility and vast open-source community.
Built and supported by Sauce Labs, Appium is a gateway to a trove of advice and support: if you stumble into any problems during testing, its huge open-source family will point you the way out.
What’s more, it’s one of the most user-friendly iOS testing tools out there. You don’t need to install an SDK in your app, and, because it uses a standard API, there’s no need to recompile the application.
The final key advantage is versatility, a key requirement for any mobile automation testing team. Appium works with any testing framework and a full suite of programming languages, including Java, JavaScript, Python and PHP. If you’re using Java, you can also configure it to work with different apparatuses.
In terms of downsides, some users find the reports insufficient. What’s more, the tests can be slow due to the remote webdriver dependency, and there’s no parallel execution – although this problem is nullified if you use Sauce Labs’ mobile cloud for script-execution. A bigger frustration is the fact that you need a local iOS device available, and connected, in order to download and configure the Appium environment.
On the whole though, Appium more than justifies its place atop our list of the best automation testing tools. Simple, accessible and widely adopted, there’s no more reliable product on the market.
What it offers:
- Open source-backed testing
- Works across iPhone, iPad and Android
How much does it cost?
It’s completely free.
What are its biggest strengths?
- There’s a huge open-source community: Appium is by far the leading open-source test framework for cross-platform (mobile) native test automation.
- You don’t need to recompile your app, or introduce any additional software.
- It works across a variety of testing platforms and languages.
2) Calabash
Like Appium, Calabash is an open-source mobile automated testing project. And, like Appium, it’s defined by its simplicity. Although it isn’t quite as all-encompassing as Appium, it does have some specific advantages.
The biggest strength is that Calabash harnesses Cucumber functionality, which means that tests can be written in plain English rather than labyrinthine chains of code. So the task of testing can be delegated to any team member, programmer or otherwise.
The second big plus is that Calabash is plugged into Xamarin – which, as we said in our Android article, is probably the best invention since the Game Boy. Xamarin provides a portal to over 1,000 different devices in the cloud, which have been put there so you can test your apps. Once you’ve written your test, just log into the Xamarin test cloud and away you go.
Again, there aren’t too many negatives. However Calabash is a bit of a mixed bag when it comes to versatility. Although it can run on physical devices as well as simulators – unlike some rival products – it doesn’t support a multiplicity of programming languages. In fact you’re pretty much stuck with Ruby. What’s more, you’ve got to install a bunch of stuff on your application to use it, like an Objective-C framework.
There are other functional limitations too. The app needs to be installed on the device every time you test it, which is a bit of a drag. And although in theory you can run several test cases concurrently, occasionally you might run into problems.
Overall though, we love this tool – and, if you’re a bit light on technical programming expertise in your company, it’s ideal.
What it offers:
- Ability to write in plain English
- Capacity to test both local and hybrid apps
- Xamarin and Cucumber functionality
How much does it cost?
Again, absolutely nothing.
What are its biggest strengths?
- Practically anyone can use it
- You have access to a huge bank of cloud devices
- It’s pretty versatile – works on physical devices and simulators
3) XCTest/XCUITest
The native library that comes with Apple’s Xcode, XCTest and XCUITest complement one another. One tests the backend stuff, the other (as its acronym might suggest) tests out the user interface – the way your app renders for the end user. They’re both excellent options, with all the advantages you’d expect from Apple’s ecosystem.
XCTest and XCUITest are a foundation stone for various other testing tools, notably Appium, which rely on their platform. So there’s a strong argument to be made for cutting out the middleman and going straight to the default.
If you do decide to stick with XC, you’ll find that the tests are seriously fast and the testing kit has no problem adapting to Apple’s Xcode releases (which have been known to scupper other tools). There’s native iOS language support, so you can write your tests with the same language you’ve used to build the application. And there’s practically no setup required – just read Apple’s documentation and you’re practically off and running.
Finally, the XC tools provide a portal to a vast cache of cloud-based devices – just like Calabash. By wiring into Sauce Labs Real Device Cloud (similar to the Xamarin vault), you’ll get fast, accurate feedback about your app, and zero in on any defects before they do damage. Alternatively, you can test on an iOS simulator if that’s more your speed.
Drawbacks? Well there’s no cross-platform support and you’re somewhat bound to the two iOS programming languages, Swift and Objective-C. If you want to write in a different language, you’re better off switching to an overlay solution such as Appium. Most significantly of all, XCTest isn’t always the most stable. If you’re looking for a tool that won’t crumble under any testing conditions, well, keep looking.
What it offers:
- Integral tool for testing iOS apps
- Comes with Apple’s Xcode development tools
- You can choose between unit and UI variants
How much does it cost?
It comes with Xcode, so it’s free.
What are its biggest strengths?
- Synchronicity with Apple’s suite
- Fast and easy to use
4) KIF — Keep-It-Functional
Closely related to XCUITest, KIF is specifically designed to test the user interface. Again, its close relationship with Xcode carries distinct advantages: it integrates straight into the Xcode project, sparing you the hassle of using an additional web server or draining your memory with any additional packages.
Set-up is extremely simple. If your project is already set up with XCTests, all you need do is add the KIF pod to your project. And the tests are even faster than the XC tools.
Another key benefit is realism. Tests are extremely user-like, meaning that the test conditions replicate the way a user would actually use the app in real-life. What’s more, KIF can simulate a variety of different user interactions.
As with all the automation testing products on our list however, there are some downsides. You have to add the KIF framework to the project – although the syntax is fairly simple – and tests have to be written in Objective-C, so there’s even less linguistic variety than the XC testing tools (although this has its advantages, as it maximizes code integration and minimizes the amount of layers you require).
Overall, then, this is a pretty solid option and great if you want to focus purely on UI. One thing to note though: if KIF ends up sneaking into your final production code, Apple will reject your app. So don’t let that happen.
What it offers:
- UI-specific testing tool
- Closely related to XCUITest
How much does it cost?
Absolutely zero.
What are its biggest strengths?
- Synchronicity with Apple
- Realistic test conditions
5) Bugfender
Ok, a bit of self-promotion here. We wanted to mention our own solution, Bugfender. Although it doesn’t do exactly the same job as some of the other tools on our list (in fact we use several of these tools alongside it), it’s brilliant for finding problems and showing you why they happened.
We built Bugfender to solve those ‘well it works fine for me’ problems; those bugs that didn’t appear on our own devices. To solve this type of problem, we previously had to get physical access to the affected devices, or get our users to describe them. As developers will appreciate, this was rather problematic.
Start Fixing Bugs Faster Now for Free
Bugfender is the best remote logger for mobile apps.
What’s more, Bugfender logs continuously, so developers will get way more than just crash information – they get a holistic view of their app’s performance. And its remote web console allows developers to drill down to specific devices, locations, even specific users – great for providing personalized customer service.
We can’t claim Bugfender is a finished product. It’s still not available for Web, Windows and tvOS. But we’re expanding its capabilities all the time: we’ve recently added Unity iOS and Android support functionality, and now we’ve added a specialist plug-in for React Native, the hugely popular cross-platform framework for building native apps (you can find out more here). Another new addition is the in-app feedback tool, allowing our clients to connect with their user base and gain the insight to make their apps even better.
If you’re looking for a mobile-friendly test solution for either iOS or Android, Bugfender will provide exactly what you need.
What it offers:
- Remote logger and crash reporter for Android & iOS
- Cross platform tool designed to help developers find the bugs in their apps before their users do
- In-App User Feedback
How much does it cost?
The basic version is free, although you can upgrade for €49
What are its biggest strengths?
- Remote logger enables developers to replicate any bug from any user’s device.
- In-built crash-reporting tool offers added value.
- Remote web console allows users to drill down to specific devices.
Top comments (0)