Extending Test Automation to Mobile Applications

Organizations need to implement automated testing strategies designed specifically for mobile applications.

Mobile device usage continues to gain momentum at increasing speed. Enterprises that delay mobile adoption face the danger of becoming competitively disadvantaged. But, before jumping in headlong, you need to be fully aware of the unique challenges that can arise when developing and implementing mobile applications—and be fully prepared.

Compared to the desktop environment, mobile applications have quite different user interface (UI) requirements, business process flows, and infrastructure dependencies. Coupled with the growing variety of mobile devices and operating systems, increased strain can be put on IT to build, port and maintain mobile applications, and this heightens the risk of quality and performance problems.

Functional Validation

Potential glitches can be avoided by putting certain mobile applications through what’s sometimes referred to as ‘functional validation’. In simple terms, functional validation is a way of testing mobile applications to ensure that they do what they are intended to do based on testing concepts such as test asset management, collaboration, reusability, accurate reporting and analysis.

The majority of testing on mobile devices is currently done manually. The problem with manually testing mobile applications is lack of accuracy, coverage and scalability, Manual testing also has considerable associated costs, especially if an application needs to be retested multiple times to ensure development and quality.

While a test engineer can manually key in a handful of transactions, he/she cannot continuously test and monitor the connection for application availability or test for all possible scenario permutations. Additionally, a tester is not able to quickly identify problems, fix them, and rerun all the tests required to validate the fix.

Using actual devices for testing

Many industry experts argue against using handsets for testing in favor of using a browser or emulator for multiple reasons. However, in order to ensure applications function and perform, they must be tested on mobile handsets and there are several reasons ranging from the theoretical to the practical.

Usability testing

Usability testing on emulators and browsers with any extension do not represent what will be shown on the actual device. With Android specifically, emulated “vanilla” environments almost never represent the actual behavior of the phone as vendors tend to modify the “source code” of the handset and add their own customization layer.

Environment related testing

Environmental testing is critical. Mobile applications rely on the location, accelerometer and network. Simply put, the test cannot be done in a simulated environment.

Interoperability testing

A mobile handset “lives” in a communicated environment and behaves accordingly, meaning an incoming phone call always receives priority over any application in use. Testing scenarios to see what happens to an application when a phone call is received is critical, as this often an everyday occurrence.

Network-related testing

Proper network related testing must be conducted. You can’t assume an application developed for Vodafone in the UK will work on the Verizon network in the US. Network neutrality has not been integrated in the cellular world, which means testing must be done taking into account the need for users to navigate multiple carriers.

Security

Security is a hot-button issue with mobile users. People are concerned about personal data, such as bank account numbers that remain on handsets, or passwords being displayed on the screen. Testing for these types of security concerns in a simulated environment is not a good use of time because what is required to be tested is the behavior of the actual handset. Not all handsets have the same security designs, so each device must be individually tested.

Performance Validation

A second issue facing IT when it comes to mobile applications is the need for performance validation – applications must be validated to ensure performance expectations are met. It’s worth remembering that the user expectations from these applications may even be higher than those of their desktop counter-parts. Performance validation for mobile applications is therefore crucial and, at times, more complex than for desktop applications.

In addition to this, mobile applications are affected by mobile network conditions such as bandwidth limitations that are more pronounced than land-line networks. These restrictions may adversely affect the communication between a handset and the back-end servers.

In order to reach the large number of simulated users that will be accessing the tested application at any given moment, performance validation needs to be automated, which can be achieved through simulated users running on emulators and browsers. Without this kind of automation it is virtually impossible to create a realistic load on the application servers and infrastructure.

When replicating the appropriate network conditions you need to assure that a simulated mobile user experiences the same bandwidth throttling and network delays that a real mobile user would. In order to complete the test, real handsets executing the relevant business processes should be running parallel to the automated performance test. This will allow the end user experience to be measured while also testing the behavior of the application’s servers and infrastructure under load.

In Conclusion

Automated functional and performance testing has a proven track-record in desktop-based applications. It helps companies deploy higher-quality software applications, reduce business risk, and accelerate problem to resolution. This will ultimately help organizations avoid costly interruptions and errors. The growing demand for instant information, especially through mobile applications, should encourage organizations to adopt automated testing strategies and solutions that are designed specifically for their mobile application needs. 

Edward
Edward has worked at HP since 2007 is currently Sector Head—Retail and Consumer Goods at HP Software. His responsibilities include: the vertical transformation of the UK go-to-market strategy, sales execution for HP Software and to build a trusted, secure and vertically credible external perception of HPSW.
Gal Tunik
Gal Tunik explains the necessity of test automation in mobile applications, citing the unique challenges that that platform raises.

The Related Post

A sampling of some free, online, and easy-to-use mobile device emulators that can help get you started with testing. ScreenFly A free, customizable tool to test your website on any screen size, including desktops, tablets, televisions, and mobile phones.
To start with, we need a Test schedule. The same is created in the process of developing the Test plan. In this schedule, we have to estimate the time required for testing of the entire Data Warehouse system. There are different methodologies available to create a Test schedule. None of them are perfect because the ...
In today’s mobile-first world, a good app is important, meaning an effective Mobile Testing strategy is  essential.  
Drawing from the Greek mythology of the lotus eaters, Anne-Marie Charrett warns testers to be weary of enjoying early success too soon upon finding high impact bugs.
Steps that will enable you to identify the weaknesses of your new app, its vulnerabilities and strengths. So you’ve just finished developing a nifty, customisable app that can help farmers track their produce from source to market via their mobile phone. You’re elated and want to get started marketing it right away. Not to burst ...
What you need to know for testing in the new paradigm This two part article analyzes the impact of the Internet of Things (IoT) product development on traditional testing. Part one of this series starts with a wide view on the IoT, embedded systems and device development aspects of testing. Part two, to be published ...
I’ve spent the last six months or so testing mobile apps for both iOS and Android. Here’s eight of my key lessons learned: Automated UI testing tools for mobile apps are immature: whilst tools like WebDriver for automated UI testing of web apps are very mature, automated UI testing of native mobile apps is the ...
Whether Or Not You Have a Mobile App You’re walking down the street. You see something interesting, and you want to know more about it. What do you do? Do you wait until you get home, open up your laptop, and type “google.com” into your search bar?
LogiGear Magazine – November 2011 – Mobile Application Testing Issue
iOS culture, even in many large organizations with skilled engineers, is behind on up-to-date testing practices. Agile development has long been all the rage; indeed, in most modern development shops the great agile methodologies are old hat. If you come from a software background like Ruby on Rails, Python, or certain Java niches, you may–until ...
Test engineers face a rapidly changing mobile application landscape, making mobile test automation a necessity. We know that mobile apps are becoming increasingly complex along with the technological advances in tablets and smartphones. Test engineers have to address multiple challenges while testing data-centric mobile apps, including the growing diversity of device features, platforms, and technologies. Fortunately, ...
To ensure the success of an app, QA must be involved in all stages of development. Quality Assurance (QA) plays a vital role in the development of mobile applications, but many overlook the critical nature of this piece of the app development process. To ensure the success of an app, QA must be involved in ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe