WavebreakMediaMicro - Fotolia
Top criteria that belong in a mobile app testing plan
Smartphones are ubiquitous, but no two users are the same -- bandwidth, devices, OSes and platforms differ. Here's how to test mobile apps exhaustively and prioritize what matters.
Clock apps wake us up, fitness trackers record our sleep -- and maybe even exercise -- and a litany of other apps fill the time in between. A mobile app is far more than another version of your organization's online application.
The number of worldwide smartphone users has increased from 2.5 billion in 2016 to 3.2 billion in 2019, according to Statista. That number is expected to reach 3.8 billion by 2021. This trajectory makes mobile apps an integral piece of practically every organization's business and technology strategies. Development teams must include mobile application testing in every test strategy.
Mobile app testing plans need to account for many unique factors and variables -- even developing these apps presents particular challenges. Let's look at the challenges of testing on many mobile devices, all of which can be taken anywhere, by users whose behavior ranges greatly.
Mobile device and OS fragmentation
Users run applications on diverse devices and operating systems. Some users accept and install every upgrade, others don't. Among mobile app end users, there's little consistency. To meet this challenge, organizations must develop and test multiple versions of applications. The mobile app testing plan must cover which device and operating system combinations are valuable to test, and how.
QA professionals should ascertain which devices customers are on, and how they use them. The selection of device and OS combinations involves risk analysis, so testers must understand the app's customers. Test leads often look at their organizations' market research to learn about customers' mobile preferences. Create personas and user value stories. The former outlines the personal characteristics of the customer and the latter describes what value they expect from the app.
Once you determine the essential device and OS combinations, plan the test approach. It is impractical to stock most test labs with all of the physical devices required to duplicate the user experience. One option is to use an emulator, a program that creates virtual devices to run the app. Emulators dramatically increase the combinations of device and OS and other factors that testers can assess. However, emulators cannot reproduce all of the physical characteristics of real devices. Some test leads opt for a combination of emulated and real devices -- to reduce risk, they use the latter for the tech target customers most often use. Another useful option is a device farm, such as AWS Device Farm, Perfecto Mobile or Sauce Labs, which offer real and cloud-based device testing. These services eliminate the need for a mobile test lab on site.
Network and performance testing
The mobility of a smartphone complicates application performance testing. Mobile users access apps from any place, so we must test in the -- sometimes literal -- field.
A mobile app testing plan should account for software use on 3G, 4G, 5G, Wi-Fi connections and airplane mode. Field testing should include places with spotty connections, such as buses and trains. Measure these key areas with mobile performance testing:
- app speed
- app download time
- data load speed
- data storage
- battery usage
Usability and user experience in the plan
To assess UX, start with the download and test all the workflows. Look at localization, how well the scrolling works and how the app fits on screens of various sizes. Test the buttons, bookmarks and links. Plan the tests for interruptions too. For example, what happens if a call or text comes in while the user makes a purchase?
Users are impatient. Whether it's a web or mobile application, customers will exit the screen if an app isn't user friendly.
