Strategize Your ’Mobile App Testing’ with ’M-Analytics’

picture6

Convergence of Social Media, Mobile, Analytics, & Cloud [SMAC] is one of the hottest trends these days. It is a major business agenda forcing organizations to rethink their strategies and increase technology investments in this direction.

What’s ‘Mobile Analytics’?

Mobile analytics involves measuring and analyzing data generated by mobile platforms and properties, such as mobile sites and mobile applications. It lets you track, measure and understand how your mobile users are interacting with your mobile sites and mobile apps. You can think of it as a superset of ‘Cookies’ which is used for Web traffic. With mobile analytics data, you can improve your cross-channel marketing initiatives, optimize the mobile experience for your customers, and grow mobile user engagement and retention.

Users statistics – number of Users, their characteristics, and where they come from

User behaviors – what actions your users are taking

Usage – Active users, sessions, session lengths, frequency, retention, etc.

Technical – Devices, carriers, firmware versions, errors

Collect comprehensive data for your mobile apps on all major operating systems – Installs, launches, average session length, and crashes

  • Visualize user navigation paths
  • Measure in-app payments and revenue
  • Geo-location analytics & targeting
  • Mobile engagement analysis
  • Mobile campaign analysis
  • Customized reports specific to your business App store statistics

Mobile Analytics & Testing

Testing services too are witnessing a growth in S-M-A-C with each evolving independently. In my earlier post “Benefits of Mobile App Testing in the Cloud” we looked at Mobile & Cloud integration for a better Mobile App Test Strategy. Objective of this post is to highlight the usage of ‘Analytics’ in ‘Mobile App Testing’!

The focal point of any Test strategy revolves around the customer (the Use-case). Wouldn’t it be better if as a Testing team you know your customer segment? App-usage workflow? Priority App features that customers use? What specific action caused the app to crash? Which features do users ignore, and why? Sounds good? Yeah! ‘Mobile Analytics’ can provide the answers to all these questions and more. Analytics can give you actionable insights into every aspect of your app resulting in an ‘Effective’ Mobile App Test Strategy.

You can only Test so much

Traditionally, Mobile App testing takes place before IT deploys an app, and in a controlled environment with a subset of users. But in the mobile era, it would be nearly impossible to test an app on every possible device and operating system combination a user could have. Consequently, the data IT gathers during mobile app testing doesn’t always reflect what will happen when an app goes into production.

There is no such thing as version one-and-done of a Mobile app. You’re always going to be iterating. That’s where Mobile Analytics comes in. Real-time data from Users who are using the app to perform real business tasks offers a more complete picture — and more actionable intelligence — than traditional App testing alone.

picture7

Structured Testing

A single bad experience can result in end users dumping the application. Using ‘Mobile Analytics’ in Mobile app testing, QA teams can adopt a structured testing approach focused on end-users. Structured tests help to proactively discover and remediate a broad number of potential application adoption issues – by exploring the app with a user mindset, seamlessly report back on the defects, provide feedback on the design, while slashing test cycle time. Leverage mobile analytics with actionable insights to improve your test effectiveness. Optimize your test cases and device testing matrix by continually measuring the user experience.

Customize Tests based on User Segments

Mobile Analytics helps you to see how different user segments interact with your App. Demographics, geographical location, platform, device used and time zone are all common examples of User segments that have different app usage patterns. Armed with this information, you can start tailoring your App Test Strategy for the appropriate segments to ensure a positive experience for all users.

Real-world Usability Tests

With ‘Mobile Analytics’ your App users effectively become testers. Individual profiles mean that you can inspect both one-night-stand installs and loyal users in order to determine what is successful and what is not. Get a high-level summary of how users interact with app content. Visualize the path users traveled from one Screen or Event to the next. This report can help you discover what app content keeps users engaged with your app. See user movement between Screens, Events, or a blended view of both Screens and Events. With this added information, Test teams can prioritize tests to focus on more relevant real-world usability tests.

A/B Testing Results

For most apps, the largest single one-day decrease in retention takes place on Day One; understanding the points at which users churn out and addressing them with A/B tests therefore has the most potential in absolute terms for retaining users in the crucial first moments after an app is launched.

The ability to analyze every aspect of your app during an A/B test is the most dynamic and informative way to ensure that you encourage optimal user engagement. What if you already know the ‘Grey’ area that has experienced a statistically significant change based on the test? This insight provides you with strong, math-driven A/B results quickly and easily. This understanding of behavioral analytics will also help you to identify where bottlenecks in the app might exist, which will have a profound effect on the kind of A/B testing you will do to address them.

Performance Tests

Mobile users have zero patience for poor performing mobile apps. Fix it or they’ll delete you. Test teams can leverage Mobile Analytics to analyze mobile app performance, stability, resource utilization, network latency and other factors to ensure an acceptable user experience.

Use App Speed reports to see how long different requests take to load in your app. Or Crashes & Exceptions report to identify the name and a brief description of the top Exceptions, or technical errors. You can define additional exception types (like Network failures and empty search results) in your app tracking code for more detail on other exceptions. Evaluate the performance of each screen in your app. Tracking scammers who simulate installs on some kind of data center with proxies and VPNs to tunnel out into a target country.

Not just Testing | It’s Quality Assurance

In technical landscape ‘Quality Assurance’ has replaced ‘Testing’. Now-a-days QA teams are not just responsible for reporting defects but to provide a deep App analysis and actionable insights with strategic recommendations to optimize App’s user experience – which features to spend resources on, and which flaws to fix or prioritize. User actions speak louder than words. Discover what works, and what doesn’t. Mobile Analytics allows data to be transformed first into insight and then into product improvements. Say in a location-based application, testing two different ways to interact with a map will likely increase use over time, thus impacting the entire life-cycle of the application.

For a Mobile QA team the recommended improvements to product features can be far more substantial when they are abstracted into best practices, made available to the entire organization, and implemented proactively in future development. Some improvements are so basic and broad – such as the results of fundamental A/B tests on pricing, tutorial flow, UI placement, etc. — that they can be assumed to be universally true. Establishing awareness of them throughout the organization can save untold time and effort in the future, allowing products to get to market faster and with higher quality.

Some real Examples

  1. Some XYZ App Users weren’t completing a workflow, and they were all abandoning the app at the exact same point — Recordings showed that users pinched and zoomed on their touchscreens at that point to more clearly see numbers they needed to enter. But there was a button they needed to press to confirm their entries on that screen, and it disappeared when they zoomed in, making it look like the app had frozen.
  2. For dating apps — Use of push notifications to tell the user that there’s something new, someone they can date or someone who wants to meet them etc. motivated them to open the app again.
  3. An app’s response timefor a certain action is a fraction of a second longer than it should be. Meanwhile, the analytics may also show that it takes twice as long as it should for a user to process an order in a customer relationship management app.
  4. Basic mobile app analytics report that a certain app crashes 10% of the time, typically about 15 seconds after it’s opened. In-app analytics reported that the crashes occur when users press the Add New Customer button on one of the app’s sub-menus. And the developer knew exactly which part of the code to fix.
  5. Why a user might have abandoned an app after only a few seconds. Perhaps the menu system is too confusing? The splash screen not impressive enough? The loading time too cumbersome?

Mobile Analytics give QA teams the information they need to determine whether products and campaigns are successful — and how to fix them if they’re not. They identify inefficiencies in Mobile App and potential solutions. With in-app analytics, QA can learn what causes apps to crash and what features users like. With this insight, QA team can streamline the app Test strategy, suggest improvements to back-end performance and identify bugs and poor design components.

Recognize that the data is important, but understand what it means. While raw data is important, it’s the organization and utilization of the data – the analytics – that really drives the push for mobile app companies to continuously strive for application optimization. A lean comprehensive overview of all of the aspects of your app in a quick and efficient manner helps you strategize the Testing solution.

 

Deepanshu Agarwal

Deepanshu Agarwal is a Software Testing professional, Team Lead and Consultant with a proven track record of 6.5+ years’ of strategic QA vision.

An accomplished and result-driven professional with proven ability to direct and improve quality programs, thereby ensuring effective & efficient project delivery. He started blogging at ‘Testing Mobile Apps‘ but later merged it into his other blog series at Software Testing Studio.

Deepanshu Agarwal
Deepanshu Agarwal is a Software Testing professional, Team Lead and Consultant with a proven track record of 6.5+ years’ of strategic QA vision. An accomplished and result-driven professional with proven ability to direct and improve quality programs, thereby ensuring effective & efficient project delivery. He started blogging at Testing Mobile Apps, but later merged it into his other blog series at Software Testing Studio.

The Related Post

To ensure the success of an app, QA must be involved in all stages of development. Quality Assurance (QA) plays a vital role in the development of mobile applications, but many overlook the critical nature of this piece of the app development process. To ensure the success of an app, QA must be involved in ...
A fully integrated test automation platform for mobile testing Mobile application testing is taking on greater significance as just about everything done on the desktop is transitioning to mobile. It’s taken a while for mobile testing in the consumer space to be made much of a priority. Now that sensitive data is being passed back ...
 LogiGear_Magazine_October_2014_Testing_Smart_and_Mobile
In the last issue on testing the SMAC stack we talked about the social and mobile aspects of testing. We will be referring to them in this article. In this issue part 2, we focus on the Analytics and Cloud aspect. The goal of this article is to understand a simple landscape of analytics and cloud.
Here’s some good news: Jonathan Kohl has a new book out titled Tap Into Mobile Application Testing. This is a Leanpub release and is still in Beta, but the beauty of buying through Leanpub – you get all the author updates as they make them! However I will be keeping an eye out for the print-on-demand ...
In today’s mobile-first world, a good app is important, meaning an effective Mobile Testing strategy is  essential.  
I am not a big fan of concepts which moves industry standards to IT. I am rather a Agile and Scrum guy. Managing multiple projects at once and trying to set a highest quality standard is a challenge and this book shows how industrial language can be translated into software development. I do not think that it ...
Will testers be among the first IT professionals to shift their toolset and workflows from desktops and laptops to tablets and smartphones? As I’m sure you already know, a monumental shift from desktop to mobile is upon us. Not only have consumer applications started leaving the desktop behind, but B2B applications are also starting their ...
LogiGear Magazine – September 2013 – Mobile Testing
                                                                                                              ...
Steps that will enable you to identify the weaknesses of your new app, its vulnerabilities and strengths. So you’ve just finished developing a nifty, customisable app that can help farmers track their produce from source to market via their mobile phone. You’re elated and want to get started marketing it right away. Not to burst ...
To help testers gain an edge, here’s a list of free resources Mobile testing is making leaps and bounds of progress in the overall testing space. As this field is highly dynamic, a tester must constantly evolve and improvise his or her knowledge of mobile testing. To help software testers gain an edge, I have compiled the following list ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe