What Matters in Mobile Testing?

Devices matter. We don’t yet trust the mobile devices like we trust desktops and laptops. In the course of testing traditional web applications, rarely do you have to think about the model of the actual machine. In mobile, however, the behavior of an application can vary from device to device. You can no longer just think about browser/OS combinations.

Mobile emulators aren’t good enough. They get you part way there, but you still have to physically test on many devices.

The version of the OS can be quite important. Just as you think about the physical device, you also must consider how nuances of each OS version can affect application behavior: iOS 3, 4 or 5; Android 2.1, 2.2 , or 2.3; Blackberry 5, 6, or 7; Windows Mobile 7.0 or 7.5; etc. Add that to your test matrix.

Mobile web apps can become native apps. Tools like PhoneGap and Appcelerator enable web applications to be compiled into native applications. Now you might need to test a version of the application that runs in browsers and a version that runs natively.

Native phone features must now be considered. With the ability to turn web applications into native applications, this opens up access to native phone features, like the camera, accelerometer, address book, etc. These are not things we are all that familiar with.

Mobile web applications tend to be heavy on the JavaScript. Many factors lead to a thick-client architectural style for mobile applications. Business logic is pushed to the client-side—i.e. JavaScript —and the server-side becomes a set of services. Suffice to say that you now may find yourself much more concerned about how to test JavaScript.

Offline capability is important. This may become a first class concern. There are several reasons why a phone can lose connectivity, such as Airplane mode and driving into a tunnel, and mobile applications must be designed to deal with this fact—yet, another thing to add to your testing list.

Connection quality is unpredictable. Remember when you had to worry about dial-up users? It’s like that. Even if you assume your target user base will have at least 3G, connectivity can degrade or be lost at any time. One minute you’re on Wi-Fi, the next on 3G, and the next on Edge.

Data plans might be metered. Many users do not have unlimited data usage. Now the application architecture and design has to take into account how it will need to respect the user’s data plan. And you have to test it.

Instrumentation and test tool support is lagging on mobile devices. Any time there is emerging technology, the development of supporting tools (like testing tools) lags. For example, there are not currently a great many mature tool options available to automate tests of mobile applications across the spectrum of devices, OSes, browsers, etc.

The network provider might matter. Perhaps a particular network provider throttles connection speeds. Perhaps a particular network provider configures its phones in a particular way that might be important to know.

You have a lot to consider building your test matrix for testing mobile applications. The test strategies are still evolving. We are all still figuring out what matters in mobile application testing. Good luck!

John Roets
John Roets has been providing software engineering leadership, specifically to application development for the web, since 1998. John is currently a Senior Architect at ITX Corporation in Rochester, NY; providing solutions to a multitude of clients, focusing his attention (at the moment) on mobile applications. He is just as likely to engage you in discussion about design patterns, databases, and the latest web technologies as he is about business strategy, software development methodologies and organizational structure.  John can be contacted through http://jroets.blogspot.com/</em
John Roets
John Roets has been providing software engineering leadership, specifically to application development for the web, since 1998. John is currently a Senior Architect at ITX Corporation in Rochester, NY; providing solutions to a multitude of clients, focusing his attention (at the moment) on mobile applications. He is just as likely to engage you in discussion about design patterns, databases, and the latest web technologies as he is about business strategy, software development methodologies and organizational structure.

The Related Post

Here’s some good news: Jonathan Kohl has a new book out titled Tap Into Mobile Application Testing. This is a Leanpub release and is still in Beta, but the beauty of buying through Leanpub – you get all the author updates as they make them! However I will be keeping an eye out for the print-on-demand ...
Manual testing teams may not be able to test all the processes with each build Test automation of applications has been around for many years. There are many of us in the automated testing field that started very early in the test automation phase, but the introduction of mobile devices has brought on a new angle ...
To help testers gain an edge, here’s a list of free resources Mobile testing is making leaps and bounds of progress in the overall testing space. As this field is highly dynamic, a tester must constantly evolve and improvise his or her knowledge of mobile testing. To help software testers gain an edge, I have compiled the following list ...
Steps that will enable you to identify the weaknesses of your new app, its vulnerabilities and strengths. So you’ve just finished developing a nifty, customisable app that can help farmers track their produce from source to market via their mobile phone. You’re elated and want to get started marketing it right away. Not to burst ...
This is the second part of a two part article that analyzes the impact of product development for the internet of things (IoT) on software testing.  Part one of this article (LogiGear Magazine, Sept 2014) gave a wide view on the IoT, embedded systems, and the device development aspects of testing on these projects. This ...
What you need to know for testing in the new paradigm This two part article analyzes the impact of the Internet of Things (IoT) product development on traditional testing. Part one of this series starts with a wide view on the IoT, embedded systems and device development aspects of testing. Part two, to be published ...
LogiGear_Magazine_December 2016_Riding the New Software Testing Wave  
Organizations need to implement automated testing strategies designed specifically for mobile applications. Mobile device usage continues to gain momentum at increasing speed. Enterprises that delay mobile adoption face the danger of becoming competitively disadvantaged. But, before jumping in headlong, you need to be fully aware of the unique challenges that can arise when developing and implementing ...
                                                                                                              ...
Test engineers face a rapidly changing mobile application landscape, making mobile test automation a necessity. We know that mobile apps are becoming increasingly complex along with the technological advances in tablets and smartphones. Test engineers have to address multiple challenges while testing data-centric mobile apps, including the growing diversity of device features, platforms, and technologies. Fortunately, ...
To start with, we need a Test schedule. The same is created in the process of developing the Test plan. In this schedule, we have to estimate the time required for testing of the entire Data Warehouse system. There are different methodologies available to create a Test schedule. None of them are perfect because the ...
Mobile is no longer an area that a few UX people specialize in, and we need to start designing and testing everything for smartphones and tablets as well as computers. If you’re new to mobile usability testing, fear not. It is not as hard as you might think but there are some key differences from ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe