WavebreakmediaMicro - Fotolia
Five mobile testing tools for developers to consider
This review of five mobile testing tools, Appium, Selendroid, ios-driver, Sauce Labs and Perfecto Mobile, will kick-start your research in a growing software market.
Mobile testing tools are experiencing a growth spurt. New products and services emerge nearly every day, and the time and energy needed to evaluate them may make all the free trials in the world seem not so free. This article looks at several mobile testing tools, listing their benefits, features and tradeoffs to help testers and IT managers make smarter investment decisions.
The first major challenge with mobile testing is the host -- that is, the device the tests will run on. Unlike a laptop or desktop, where the tests are developed and run on the same device, automated tests are typically planned and programmed on a laptop and run on some mobile device that may be plugged into the laptop and emulated. Or the mobile device might run somewhere else and be connected to the test driver over the Internet. Most large testing programs require testing on a handful, if not dozens, of machines, making physical plug-and-play untenable. These challenges should receive special attention when reviewing tools.
Appium
Appium is a small server written in node.js. Appium allows users to access their Android or iOS environment where they have the server running.
Since Appium uses the WebDriver framework, it can run tests in any language that WebDriver supports. Webdriver is a popular Web API that developers can program in many languages, including Ruby, Python, JavaScript and many more.
On the plus side, Appium is free and based on open source components. On the downside, that means setup can be a hassle; you have to install open source prerequisites and make sure the versions are compatible.
Selendroid and ios-driver
Two other options fairly similar to Appium are Selendroid (which recently replaced AndroidDriver) and ios-driver. They are both device-specific implementations of Selenium WebDriver for mobile devices. Selendroid is a framework for testing software on Android devices; ios-driver applies specifically to iOS devices.
With Selendroid, testers can run tests on an emulator or on an actual device. As of version 0.6.5, ios-driver does not support running tests on real devices. But that feature may be added in the next release.
Testers using Selenium products can benefit from the massive community of people that are also Selenium customers. Several websites offer free community-based support and testers can rest assured that most problems they experience will have been covered by someone else.
Sauce Labs
Sauce Labs offers a mobile testing product based on the Appium framework. Strictly speaking, Sauce Labs uses the Appium tool to host the tests, providing the environment for running digital tests. This approach provides the same benefits of freedom of programming language and dual (iOS, Android) platform support while removing the need for physical hardware.
In other words, Sauce Labs uses the Appium framework, but gives testers the device to run the tests on a virtual mobile device, then bills the device as a metered service by the CPU hour. This service is built in the same way as other Sauce Lab products, so testers get similar features, such as the potential to watch videos of tests as they run, or to link those errors reported in a test to a particular point in the video, making debugging much easier.
However, unlike the previously mentioned tools, mobile testing supported by the Sauce Labs platform is not free. The manual subscription plan is $12.00 a month, the individual plan is $49.00 a month and the small team plan is $149.00 a month. Enterprises should contact Sauce Labs for further pricing information.
Perfecto Mobile
Perfecto Mobile is not based on WebDriver. It has a native UI based on a proprietary TCL-based programming language. This approach lets testers who are not programmers select behaviors from a simple, no-coding development environment and write expected results in simple English. Tests designed with these proprietary tools may be more costly to move away from than tests written with open-source tools.
In addition to its native UI, Perfecto Mobile provides a RESTful API for integration with other interfaces including WebDriver and JUnit. Like Sauce Labs, Perfecto Mobile can provide either simulators or cloud-based real devices to run tests on. Perfecto Mobile recommends running tests on real devices. Perfecto Mobile provides live scheduled demos and prerecorded videos, but less extensive written documentation than the other products listed here.
The choice is yours
Driving the user interface with mobile testing tools is massively popular in the software world, and automation of mobile websites is quickly becoming common practice. This growth has caused an explosion in companies creating mobile testing tools. Now that testers have so many tools to choose from, the choice is more complicated, often made worse by folklore surrounding test automation and marketing material. Hopefully, this article provides a starting point for testers' research.
Update: This article was updated on November 13, 2014. The previous version stated that Perfecto Mobile provides simulations of cloud-based real devices. That has been corrected to read "either simulations or cloud-based real devices." Other clarifications about Perfecto Mobile have also been made.
