Convergence of Social Media, Mobile, Analytics, & Cloud [SMAC] is one of the hottest trends these days. It is a major business agenda forcing organizations to rethink their strategies and increase technology investments in this direction.
What’s ‘Mobile Analytics’?
Mobile analytics involves measuring and analyzing data generated by mobile platforms and properties, such as mobile sites and mobile applications. It lets you track, measure and understand how your mobile users are interacting with your mobile sites and mobile apps. You can think of it as a superset of ‘Cookies’ which is used for Web traffic. With mobile analytics data, you can improve your cross-channel marketing initiatives, optimize the mobile experience for your customers, and grow mobile user engagement and retention.
Users statistics – number of Users, their characteristics, and where they come from
User behaviors – what actions your users are taking
Usage – Active users, sessions, session lengths, frequency, retention, etc.
Technical – Devices, carriers, firmware versions, errors
Collect comprehensive data for your mobile apps on all major operating systems – Installs, launches, average session length, and crashes
- Visualize user navigation paths
- Measure in-app payments and revenue
- Geo-location analytics & targeting
- Mobile engagement analysis
- Mobile campaign analysis
- Customized reports specific to your business App store statistics
Mobile Analytics & Testing
Testing services too are witnessing a growth in S-M-A-C with each evolving independently. In my earlier post “Benefits of Mobile App Testing in the Cloud” we looked at Mobile & Cloud integration for a better Mobile App Test Strategy. Objective of this post is to highlight the usage of ‘Analytics’ in ‘Mobile App Testing’!
The focal point of any Test strategy revolves around the customer (the Use-case). Wouldn’t it be better if as a Testing team you know your customer segment? App-usage workflow? Priority App features that customers use? What specific action caused the app to crash? Which features do users ignore, and why? Sounds good? Yeah! ‘Mobile Analytics’ can provide the answers to all these questions and more. Analytics can give you actionable insights into every aspect of your app resulting in an ‘Effective’ Mobile App Test Strategy.
You can only Test so much
Traditionally, Mobile App testing takes place before IT deploys an app, and in a controlled environment with a subset of users. But in the mobile era, it would be nearly impossible to test an app on every possible device and operating system combination a user could have. Consequently, the data IT gathers during mobile app testing doesn’t always reflect what will happen when an app goes into production.
There is no such thing as version one-and-done of a Mobile app. You’re always going to be iterating. That’s where Mobile Analytics comes in. Real-time data from Users who are using the app to perform real business tasks offers a more complete picture — and more actionable intelligence — than traditional App testing alone.
Structured Testing
A single bad experience can result in end users dumping the application. Using ‘Mobile Analytics’ in Mobile app testing, QA teams can adopt a structured testing approach focused on end-users. Structured tests help to proactively discover and remediate a broad number of potential application adoption issues – by exploring the app with a user mindset, seamlessly report back on the defects, provide feedback on the design, while slashing test cycle time. Leverage mobile analytics with actionable insights to improve your test effectiveness. Optimize your test cases and device testing matrix by continually measuring the user experience.
Customize Tests based on User Segments
Mobile Analytics helps you to see how different user segments interact with your App. Demographics, geographical location, platform, device used and time zone are all common examples of User segments that have different app usage patterns. Armed with this information, you can start tailoring your App Test Strategy for the appropriate segments to ensure a positive experience for all users.
Real-world Usability Tests
With ‘Mobile Analytics’ your App users effectively become testers. Individual profiles mean that you can inspect both one-night-stand installs and loyal users in order to determine what is successful and what is not. Get a high-level summary of how users interact with app content. Visualize the path users traveled from one Screen or Event to the next. This report can help you discover what app content keeps users engaged with your app. See user movement between Screens, Events, or a blended view of both Screens and Events. With this added information, Test teams can prioritize tests to focus on more relevant real-world usability tests.
A/B Testing Results
For most apps, the largest single one-day decrease in retention takes place on Day One; understanding the points at which users churn out and addressing them with A/B tests therefore has the most potential in absolute terms for retaining users in the crucial first moments after an app is launched.
The ability to analyze every aspect of your app during an A/B test is the most dynamic and informative way to ensure that you encourage optimal user engagement. What if you already know the ‘Grey’ area that has experienced a statistically significant change based on the test? This insight provides you with strong, math-driven A/B results quickly and easily. This understanding of behavioral analytics will also help you to identify where bottlenecks in the app might exist, which will have a profound effect on the kind of A/B testing you will do to address them.
Performance Tests
Mobile users have zero patience for poor performing mobile apps. Fix it or they’ll delete you. Test teams can leverage Mobile Analytics to analyze mobile app performance, stability, resource utilization, network latency and other factors to ensure an acceptable user experience.
Use App Speed reports to see how long different requests take to load in your app. Or Crashes & Exceptions report to identify the name and a brief description of the top Exceptions, or technical errors. You can define additional exception types (like Network failures and empty search results) in your app tracking code for more detail on other exceptions. Evaluate the performance of each screen in your app. Tracking scammers who simulate installs on some kind of data center with proxies and VPNs to tunnel out into a target country.
Not just Testing | It’s Quality Assurance
In technical landscape ‘Quality Assurance’ has replaced ‘Testing’. Now-a-days QA teams are not just responsible for reporting defects but to provide a deep App analysis and actionable insights with strategic recommendations to optimize App’s user experience – which features to spend resources on, and which flaws to fix or prioritize. User actions speak louder than words. Discover what works, and what doesn’t. Mobile Analytics allows data to be transformed first into insight and then into product improvements. Say in a location-based application, testing two different ways to interact with a map will likely increase use over time, thus impacting the entire life-cycle of the application.
For a Mobile QA team the recommended improvements to product features can be far more substantial when they are abstracted into best practices, made available to the entire organization, and implemented proactively in future development. Some improvements are so basic and broad – such as the results of fundamental A/B tests on pricing, tutorial flow, UI placement, etc. — that they can be assumed to be universally true. Establishing awareness of them throughout the organization can save untold time and effort in the future, allowing products to get to market faster and with higher quality.
Some real Examples
- Some XYZ App Users weren’t completing a workflow, and they were all abandoning the app at the exact same point — Recordings showed that users pinched and zoomed on their touchscreens at that point to more clearly see numbers they needed to enter. But there was a button they needed to press to confirm their entries on that screen, and it disappeared when they zoomed in, making it look like the app had frozen.
- For dating apps — Use of push notifications to tell the user that there’s something new, someone they can date or someone who wants to meet them etc. motivated them to open the app again.
- An app’s response timefor a certain action is a fraction of a second longer than it should be. Meanwhile, the analytics may also show that it takes twice as long as it should for a user to process an order in a customer relationship management app.
- Basic mobile app analytics report that a certain app crashes 10% of the time, typically about 15 seconds after it’s opened. In-app analytics reported that the crashes occur when users press the Add New Customer button on one of the app’s sub-menus. And the developer knew exactly which part of the code to fix.
- Why a user might have abandoned an app after only a few seconds. Perhaps the menu system is too confusing? The splash screen not impressive enough? The loading time too cumbersome?
Mobile Analytics give QA teams the information they need to determine whether products and campaigns are successful — and how to fix them if they’re not. They identify inefficiencies in Mobile App and potential solutions. With in-app analytics, QA can learn what causes apps to crash and what features users like. With this insight, QA team can streamline the app Test strategy, suggest improvements to back-end performance and identify bugs and poor design components.
Recognize that the data is important, but understand what it means. While raw data is important, it’s the organization and utilization of the data – the analytics – that really drives the push for mobile app companies to continuously strive for application optimization. A lean comprehensive overview of all of the aspects of your app in a quick and efficient manner helps you strategize the Testing solution.
