Strategies to Approach Mobile Web App Testing
Mobile web technology has been continuously changing over the past few years, making “keeping up” challenging. In this article, Raj Subramanian covers the latest trends and changes happening in the mobile web and how testers can prepare for them.
A couple of years ago, I decided to focus on testing mobile applications. Today, I’ve reached a point where the majority of my testing experience is with mobile web apps, and I find myself spending a great deal of time researching approaches, tools, and networking with other mobile testers to get a better grasp of the subject. Given the past of change, I eventually came to the conclusion that this would never stop; that I would need to continually, constantly invest time and effort into keeping up.
Before we jump into testing them, let’s spend a moment or two on the applications themselves. On a high level we have three types of mobile applications — native, web, and hybrid. Each one has its own advantages and disadvantages. Native applications are specifically designed to run on device OS and machine firmware, while web applications pull resources from the Internet each time they run (mobile compatible websites). Finally, there are hybrid applications that have a native shell and are implemented using web technologies. I personally have worked with all three types and understand that there are multiple factors in each one of them. A tester needs to be aware of this and think about test objectives, test approach, test design, and test execution specific to the type of application being tested.
The Move toward Responsive Web Design
The current direction in web evolution is “responsive web design.” The simplest way to explain the concept is to say that every website should render “properly” for every form factor of a particular device. Another way to put it is that responsive design attempts to bridge the gap between a desktop browser experience and the web experience on a mobile device by creating websites that work both on a full-scale desktop computer and a mobile device. To do that, the site has to change dynamically when viewed on a mobile device. If we can get this conversion to happen automatically, then the programmer only needs to create one site, saving development time.
Of course, this automatic translation can have errors; that is where the new testing challenges come in. Users don’t want to see errors and they demand access to Internet on their devices. Here are some statistics of just how wide this difference in use is:
- 28 percent of Internet usage comes from mobile phones. It is projected to take over desktop usage by 2014.
- 48 percent of users say that if they arrive on a business site that isn’t working well on mobile, they take it as an indication of the business simply not caring.
- 62 percent of companies that designed a website specifically for mobile had increased sales.
- 90 percent of people use multiple screens to access the web.
- About 91 percent of US citizens have their mobile device with them 24/7.
- The number of unique screen resolutions rose from 97 in 2010 to about 232 in 2013.
- Only 55 percent of companies are currently conducting any online user experience testing.
To sum it up: If your customers are in a position to expect the software to work on “their” devices, then responsive web design, and a test strategy for it, should probably be on your learning list. The world is heading in this direction and it is here to stay at least until the next game changer appears. Some good examples of responsive web sites are the websites of Microsoft, Disney, and the Boston Globe; there are others listed at www.mediaqueri.es.
The Right Testing Approach
Testing these responsive websites is a challenge, as there are different devices, form factors, screen resolutions (over 200 unique ones), browsers, and countless permutations. When I wanted to learn to test responsive sites, there was no “book,” so I experimented with different approaches. Here are a few of the things I tried, which resulted in me learning new techniques:
- Networking with mobile testers and consultants both at my company and elsewhere
- Keeping track of new mobile development and testing news and technologies
- Researching different tools to help out in testing
- Evaluating how rapid software testing approaches could help out in mobile testing
- Researching production defects. This is to help focus testing on types of problems the users are seeing in the application
- Attending mobile-focused conferences
Another thing to keep in mind is that because mobile is such a vast domain, it’s important to try out different approaches, rather than just stick to a set of scripted test cases and hope you have met the so-called “100 percent coverage.” These test cases may be effective in covering some aspects of the application, but they rarely cover the aspect of usability and testing from the end-user point of view. I have seen this happen time and again throughout my experience as a mobile tester. Don’t fall into the trap of writing detailed test cases for every scenario. By doing this, you will end up consuming valuable testing time that could have actually been spent on exploring the application. The mantra I follow is to perform “very high-level test cases complemented with exploratory testing.”
The “learn-and-experiment” approach worked well for me while testing my first responsive website. For example, my project implemented totally new pages and functionality that interacted with multiple systems. This being the case, it did not make much sense to waste time in writing detailed test cases, as there are only so many interactions we could cover. Instead, we wrote high-level test cases and covered more interaction scenarios by doing exploratory testing.
Finally, never hesitate to challenge anything when working in a mobile web project; always ask questions. No one can predict exactly how a new feature would, could, or will be used in the real world, and this is especially true in mobile and the responsive web design world. There are always challenges, and our job as testers is to challenge ideas and question idealistic thinking.
This article originally appeared in StickyMinds at http://www.stickyminds.com/article/unwrapping-box-strategies-approach-mobile-web-app-testing
