What you need to know to get started
Introduction to mobile application testing
Gone are the days when the telephone was an appliance that sat in a corner and had to ring to get our attention, or a computer was a machine only few people used — these devices are now an extension of our being — a window to the world and virtual servants that do as they are told. Computers became a rage and changed how we humans thought, behaved, learned and existed.
Mobile technology and smart devices are the trend now and will change the future of the world as we know it. We all can vouch for that, can’t we?
Now, it would be amateurish if I were to list what we use these mobile devices for. You all know it — perhaps better than I do.
So let’s get straight to what this tutorial is all about:
This tutorial will serve as both an introduction, and your guide, to mobile testing. So, read through!
Types of mobile testing
Broadly speaking, there are two kinds of testing that take place on mobile devices:
Hardware testing
Testing of the device, including the internal processors, internal hardware, screen sizes, resolution, space or memory, camera, radio, Bluetooth, WIFI, etc. This is sometimes referred to as, simply, mobile testing.
Software or Application testing:
The applications that work on mobile devices, and their functionality, are tested. This is called mobile application testing to differentiate it from the earlier method. And among mobile applications, there are a few basic differences that are important to understand:
Native apps: A native application is one that is created for use on a platform like mobile and tablets.
Mobile web apps: These are server-side apps to access website(s) on mobile using different browsers, like Chrome and Firefox, by connecting to a mobile network or wireless network like WIFI.
Hybrid apps: Combinations of native app and web app. They run on devices or offline and are written using web technologies like HTML5 and CSS.
There are a few basic differences that set these app types apart:
- Native apps have single-platform affinity while mobile web apps have cross-platform affinity.
- Native apps are written in platforms like SDKs while mobile web apps are written with web technologies like HTML, CSS, ASP.NET, Java, and PHP.
- For a native app, installation is required; for mobile web apps, no installation is required.
- Native apps can be updated from the play store or app store while mobile web apps are centralized updates.
- Many native apps don’t require an Internet connection, but for mobile web apps it’s a must.
- In general, native apps works faster than mobile web apps.
- Native apps are installed from app stores like Google Play Store or the Apple app store while mobile web apps are websites and are only accessible through the Internet.
From here on, the rest of this article is about mobile application testing.
Significance of mobile application testing
Testing applications on mobile devices is more
challenging than testing web apps on desktops. This is due to:
- A different range of mobile devices with different screen sizes and hardware configurations like hard keypad, virtual keypad (touch screen) and trackball, etc.
- Wide varieties of mobile devices like HTC, Samsung, Apple and Nokia.
- Different mobile operating systems like Android, Symbian, Windows, Blackberry and IOS.
- Different versions of operation systems like iOS 5.x, iOS 6.x, BB5.x, BB6.x, etc.
- Different mobile network technologies like GSM and CDMA.
- Frequent updates — like Android 4.2, 4.3, 4.4; or iOS-5.x, 6.x — and with each update a new testing cycle is recommended to make certain no application functionality is impacted.
As with any application, mobile application testing is very important, as the clientele is frequently in the millions for a certain product — and a product with bugs is never appreciated. It often results in monetary losses, legal issues and irreparable brand image damage.
Basic differences between mobile and desktop application testing
A few obvious aspects that set mobile app testing apart from desktop testing:
- On a desktop, the application is tested on a central processing unit. On a mobile device, the application is tested on handsets like Samsung, Nokia, Apple and HTC.
- Mobile device screen size is smaller than desktop.
- Mobile devices have less memory than a desktop.
- Mobiles use network connections like 2G, 3G, 4G or WIFI whereas desktops use broadband or dial up connections.
- The automation tool used for desktop application testing might not work on mobile applications.
Types of mobile app testing
To address all the above technical aspects, the following types of testing are performed on mobile applications:
- Usability testing Used to ensure that the mobile app is easy to use and provides a satisfactory user experience to the customers.
- Compatibility testing Testing of the application in different mobile devices, browsers, screen sizes and OS versions according to the requirements.
- Interface testing Testing of menu options, buttons, bookmarks, history, settings, and navigation flow of the application.
- Services testing Testing the services of the application online and offline.
- Low-level resource testing Testing of memory usage, auto deletion of temporary files, local database growth issues, etc.
- Performance testing Testing the performance of the application by changing the connection from 2G, 3G to WIFI, sharing the documents, battery consumption, etc.
- Operational testing Testing of backup and recovery plan if battery goes down, or data loss while upgrading the application from the store.
- Installation tests Validation of the application by installing/uninstalling it on the devices.
- Security testing Testing an application to validate if the information system protects data or not.
Mobile application testing strategy
The test strategy should make sure that all the quality and performance guidelines are met. A few pointers in this area:
#1 Selection of the devices
Analyze the market and choose those devices that are widely used. (This decision mostly relies on the clients. The client or app builders consider the popularity factor of each device, as well as the marketing needs for the application, to decide which handsets to use for testing.)
#2 Emulators
The use of these is extremely helpful in the initial stages of development, as they allow for quick and efficient checking of the app. An emulator is a system that runs software from one environment to another environment without changing the software itself. It duplicates the features and work of a real system.
Types of Mobile Emulators
Device emulator: provided by device manufacturers.
Browser emulator: simulates mobile browser environments.
Operating system emulator: Apple provides emulators for iPhones, as does Microsoft for Windows phones and Google for Android phones.
See the accompanying section for a sampling of freely available, online, easy-to-use mobile device emulators.
#3 Physical devices
After a satisfactory level of development is complete for the mobile app, you can move to test on physical devices for more real-life scenarios-based testing.
#4 Cloud computing-based testing
Cloud computing is something you should seriously consider. It is basically running devices on multiple systems or networks via the Internet, where applications can be tested, updated and managed. For testing purposes, it creates the web-based mobile environment on a simulator to access the mobile app.
Pros of Cloud Computing
- Backup and recovery
Cloud computing automatically takes a backup of your data from a remote location, making recovery and restoration of data easy. And, the storage capacity is unlimited. - Accessibility
Clouds can be accessed from different devices and from anywhere. - Convenience
Cloud computing is cost-efficient, easy to use, maintain and update. - Fast and quick deployment
- Web-based interface
- Efficiency
Can run the same script on several devices in parallel.
Cons of Cloud Computing
- Less control
Since the application runs on a remote or third party environment, the user has limited control and access over the functions. - Internet connectivity issues
The setup is on the Internet. Network issues affect the availability and functionality. - Security and privacy issues
Cloud computing is Internet computing and nothing on the Internet is completely secure, so the risks of data hacking are higher.
#5 Automated vs. Manual testing
If the application contains new functionality, test it manually.
If the application requires testing once or twice, do it manually.
Automate the scripts for regression test cases. If regression tests are repeated, automated testing is perfect for that.
Automate the scripts for those complex scenarios which are time-consuming if executed manually.
Two kinds of automation tools are available to test mobile apps:
- Object based mobile testing tools
Automation by mapping elements on the device screen into objects. This approach is independent of screen size and mainly used for Android devices.
Examples: ranorex, jamo solution - Image based mobile testing tools
Create automation scripts based on the screen coordinates of elements.
Examples: Sikuli, Egg Plant, RoutineBot
#6 Network configuration
This is also a necessary part of mobile testing. It’s important to validate the application on different networks like 2G, 3G, 4G or WIFI.
Test cases for testing a mobile app
In addition to functionality-based test cases, mobile application testing requires special test cases which should cover following scenarios:
- Battery usage
It’s important to keep a track of battery consumption while running applications on mobile devices. - Speed of the application
The response times on different devices, with different memory parameters, with different network types, etc. - Data requirements
For installation, as well as to verify that the user with a limited data plan will able to download and run it. - Memory requirements
Again, to download, install and run. - Functionality of the application
To ensure that the application is not crashing due to network failure or anything else.
Typical activities and proceedings in testing mobile applications
The scope of the testing depends on the amount of requirements to be checked or the extent of changes made to the app. If the changes are few, a round of sanity testing will do. In case of major and/or complex changes, a full regression is recommended.
An example application testing project
ILL (International Learn Lab) is an application designed to help the admin and publisher create websites in collaboration. Using a web browser, instructors choose from a set of features to create a class that meets their requirements.
Mobile Testing process:
Step #1 Identify the types of testing
As the ILL application is applicable to browsers, it’s mandatory to test this application on all supported browsers using different mobile devices. We need to do usability, functional and compatibility testing on different browsers with the combinations of manual and automated test cases.
Step #2 Manual and Automated testing
The methodology followed for this project is Agile, with an iteration of two weeks. Every two weeks, the dev team releases a new build to the testing team and the testing team runs their test cases on the QA environment. The automation team creates scripts for the set of basic functionality and runs the scripts that help determine if the new build is stable enough to test. The manual testing team tests the new functionality.
JIRA is used for writing of acceptance criteria, maintaining of test cases and logging/re-verification of defects. Once the iteration gets over, an iteration planning meeting is held where the dev team, product owner, business analyst, and QA team discuss what went well and what needs to improve.
Step #3 Beta Testing
Once the regression testing is completed by the QA team, the build moves into User Acceptance Testing (UAT). UAT is done by the client. They re-verify all the bugs to make sure every bug was fixed and the application is working as expected on every approved browser.
Step #4 Performance test
The performance testing team tests the performance of the web app using JMeter scripts and with different loads on the application.
Step #5 Browser testing
The web app gets tested across multiple browsers, using both different simulation tools as well as real physical mobile devices.
Step #6 Launch plan
After every 4th week the testing moves into staging, where a final round of end-to-end testing on these devices is performed to make sure the product is ready for production. And then, it goes Live!
Conclusion
Designing the right test strategy, choosing the right mobile simulators, devices and mobile testing tools can help ensure that we have 100% test coverage and help us include security, usability, performance, functionality and compatibility-based tests into our test suites.
This article originally appeared in Software Testing Help at http://www.softwaretestinghelp.com/beginners-guide-to-mobile-application-testing/
