The Automation of Mobile Testing – Should We or Shouldn’t We . . .

6.3 -Sep-2014-03Manual testing teams may not be able to test all the processes with each build

Test automation of applications has been around for many years. There are many of us in the automated testing field that started very early in the test automation phase, but the introduction of mobile devices has brought on a new angle in test automation of applications. Mobile device testing introduces new technical challenges along with new processes and testing tools. But should we use test automation for mobile applications that are on mobile devices? Many of us have mobile devices and we get our applications from the stores on Android or iOS and we go about our business.

Quality, performance and business rules?

As we know, there are at least three kinds of applications for Mobile Devices; Native, Mobile Web, and Hybrid. Like any application, developers need to build it, and testers test it. Now, when we introduce methodology into the mix, we add challenges. This agile development process introduces tight timelines and tight delivery dates. Developers are challenged to get the business processes into the applications as quickly as possible. This adds multiple builds daily, sometimes hourly. In the desktop and standard web testing process, whether automated or manual, we would expect to regression test each build, right? How many testers would you need to do that when builds are being delivered so quickly? Can manual testers do that and keep up with what they have tested, along with making sure that the business processes have been tested each time?

When there is a time crunch, usually there will be decisions made to stick to the critical business processes or new functionality and leave it at that. It sounds great, but is that the best way to test and make sure a quality product has been delivered to production? Users are accustomed to application pages being rendered within a few seconds, which means response times need to be verified before going into production. Users are accustomed to the application functionality being consistent each and every build. If there is a new build, users are not expecting to lose functionality or any visual effects. Our eyes are accustomed to application pages looking great each and every build. How is a small test team going to do that? Honestly, a manual testing team will probably not be able to test all the processes each time and will revert to the 80/ 20 rule, or something close. Now, there are some teams out there that will claim to have an army of testers that they will call up and have the start testing the build. That is awesome, however, even that army of testers may take the common path or the happy path. They may not have all the business rules or the defects to look at to see what the challenges or critical areas of the app are. How do they report back to a central point and collaborate on what they have tested and what the results were? How would they also add requirements trace-ability into the process? Unless this is a seasoned group, that will be a challenge.

Where test automation comes in.

From wherever you are in the world, using test automation can make testing more complete and add requirements coverage, test coverage, trace-ability, defect tracking, and overall test reporting easier to manage. From one report, management can see what percentage of the application has been tested, what defects have been introduced and what defects have been fixed. With test automation, when a defect is found, it is very easy to recreate that defect and the steps for it. When manual testers test and find a defect, if the steps are complex enough, then chances are, the tester will not follow the same path and the defect may not be reproduced. That may be a big issue.

With test automation, test scripts can be run off hours if needed. Now with mobile device testing, that will add complexity because some environments and devices will need to be configured to run the test. That may or may not be able to be automated. Either way, the scripts can be run and tests can be run without manual intervention once set up. With test automation, when builds are done multiple times during the day, we can find the changes to the application much faster than a manual tester.

How fast can scripts do that?

Well, that would depend upon the environments being used, the complexity of the application under test, and the performance of the application.

For the most part, running automated scripts will always test more functionality than a manual tester. If there is a critical path that needs to be tested, the scripts can be run specifically for that and they will be executed the same each and every time. For many teams, cost comes into the picture. Most departments have budgets and test automation tools are usually pretty expensive. Once research is done, it will be found that there are many tools out there that can create scripts for a fairly inexpensive cost with even cheaper renewal costs. For the teams that do not have much experience with test automation, it would be better to try a small project with one of the less expensive tools. And remember, cheaper cost does not mean cheaper in the way the tool works, but it may mean that you don’t get all the bells and whistles. Compromises will always need to be added.

Should you use test automation for mobile device testing?

We can ask the same question for the test automation of a desktop application. It does not just need to be answered for mobile device testing. If your team plans to create subsequent updates to the app, or create the application in phases, I would highly recommend some sort of test script automation. If the plan is to just create a quick application one time and that is all, then it is up to the team to decide whether or not it is cost effective to automate the testing process. There will always be a benefit to test automation as a whole because test automation is not just script execution. Test automation is the whole testing process. That is where the real benefit for the whole project comes in. But for development and testers, script automation will help by allowing the manual testers to move onto some other critical tasks and leave the regression to the automation team.

Whether you have 4 or 400 testers, there will always be something that will be missed. Adding test automation will add test coverage and higher requirements coverage to your testing results. Mobile Device test automation will use a lot of the same processes that standard test automation uses. The tools may be different, but the rest will be similar.

(This article was originally published on www.olenick.com/blog/automation-mobile-testing/)

 

Sam Mullin

Sam Mullin is a specialist in Mobile Device Test Automation. He has contributed to test planning, design, and execution of automated testing efforts within the healthcare, electricity, and banking industries. He continues to work on providing specialized support for mobile device test automation using various mobile device testing tools. Currently, Sam is working for Olenick and Associates located in Chicago Illinois (www.olenick.com)

Sam Mullin
Sam Mullin is a specialist in Mobile Device Test Automation. He has contributed to test planning, design, and execution of automated testing efforts within the healthcare, electricity, and banking industries. He continues to work on providing specialized support for mobile device test automation using various mobile device testing tools. Currently, Sam is working for Olenick and Associates located in Chicago Illinois

The Related Post

A sampling of some free, online, and easy-to-use mobile device emulators that can help get you started with testing. ScreenFly A free, customizable tool to test your website on any screen size, including desktops, tablets, televisions, and mobile phones.
What you need to know to get started Introduction to mobile application testing Gone are the days when the telephone was an appliance that sat in a corner and had to ring to get our attention, or a computer was a machine only few people used — these devices are now an extension of our ...
LogiGear_Magazine_December 2016_Riding the New Software Testing Wave  
Convergence of Social Media, Mobile, Analytics, & Cloud [SMAC] is one of the hottest trends these days. It is a major business agenda forcing organizations to rethink their strategies and increase technology investments in this direction.
Here’s some good news: Jonathan Kohl has a new book out titled Tap Into Mobile Application Testing. This is a Leanpub release and is still in Beta, but the beauty of buying through Leanpub – you get all the author updates as they make them! However I will be keeping an eye out for the print-on-demand ...
What you need to know in order to have effective and reliable Test Automation for your mobile apps I realized that Test Automation interfaces are pivotal to effective and efficient Test Automation, yet very few people trying to test their mobile apps seemed to know how their automated tests connected with the apps they wanted ...
A fully integrated test automation platform for mobile testing Mobile application testing is taking on greater significance as just about everything done on the desktop is transitioning to mobile. It’s taken a while for mobile testing in the consumer space to be made much of a priority. Now that sensitive data is being passed back ...
To start with, we need a Test schedule. The same is created in the process of developing the Test plan. In this schedule, we have to estimate the time required for testing of the entire Data Warehouse system. There are different methodologies available to create a Test schedule. None of them are perfect because the ...
                                                                                                              ...
Will testers be among the first IT professionals to shift their toolset and workflows from desktops and laptops to tablets and smartphones? As I’m sure you already know, a monumental shift from desktop to mobile is upon us. Not only have consumer applications started leaving the desktop behind, but B2B applications are also starting their ...
I am not a big fan of concepts which moves industry standards to IT. I am rather a Agile and Scrum guy. Managing multiple projects at once and trying to set a highest quality standard is a challenge and this book shows how industrial language can be translated into software development. I do not think that it ...
Mobile is no longer an area that a few UX people specialize in, and we need to start designing and testing everything for smartphones and tablets as well as computers. If you’re new to mobile usability testing, fear not. It is not as hard as you might think but there are some key differences from ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe