Manual testing teams may not be able to test all the processes with each build
Test automation of applications has been around for many years. There are many of us in the automated testing field that started very early in the test automation phase, but the introduction of mobile devices has brought on a new angle in test automation of applications. Mobile device testing introduces new technical challenges along with new processes and testing tools. But should we use test automation for mobile applications that are on mobile devices? Many of us have mobile devices and we get our applications from the stores on Android or iOS and we go about our business.
Quality, performance and business rules?
As we know, there are at least three kinds of applications for Mobile Devices; Native, Mobile Web, and Hybrid. Like any application, developers need to build it, and testers test it. Now, when we introduce methodology into the mix, we add challenges. This agile development process introduces tight timelines and tight delivery dates. Developers are challenged to get the business processes into the applications as quickly as possible. This adds multiple builds daily, sometimes hourly. In the desktop and standard web testing process, whether automated or manual, we would expect to regression test each build, right? How many testers would you need to do that when builds are being delivered so quickly? Can manual testers do that and keep up with what they have tested, along with making sure that the business processes have been tested each time?
When there is a time crunch, usually there will be decisions made to stick to the critical business processes or new functionality and leave it at that. It sounds great, but is that the best way to test and make sure a quality product has been delivered to production? Users are accustomed to application pages being rendered within a few seconds, which means response times need to be verified before going into production. Users are accustomed to the application functionality being consistent each and every build. If there is a new build, users are not expecting to lose functionality or any visual effects. Our eyes are accustomed to application pages looking great each and every build. How is a small test team going to do that? Honestly, a manual testing team will probably not be able to test all the processes each time and will revert to the 80/ 20 rule, or something close. Now, there are some teams out there that will claim to have an army of testers that they will call up and have the start testing the build. That is awesome, however, even that army of testers may take the common path or the happy path. They may not have all the business rules or the defects to look at to see what the challenges or critical areas of the app are. How do they report back to a central point and collaborate on what they have tested and what the results were? How would they also add requirements trace-ability into the process? Unless this is a seasoned group, that will be a challenge.
Where test automation comes in.
From wherever you are in the world, using test automation can make testing more complete and add requirements coverage, test coverage, trace-ability, defect tracking, and overall test reporting easier to manage. From one report, management can see what percentage of the application has been tested, what defects have been introduced and what defects have been fixed. With test automation, when a defect is found, it is very easy to recreate that defect and the steps for it. When manual testers test and find a defect, if the steps are complex enough, then chances are, the tester will not follow the same path and the defect may not be reproduced. That may be a big issue.
With test automation, test scripts can be run off hours if needed. Now with mobile device testing, that will add complexity because some environments and devices will need to be configured to run the test. That may or may not be able to be automated. Either way, the scripts can be run and tests can be run without manual intervention once set up. With test automation, when builds are done multiple times during the day, we can find the changes to the application much faster than a manual tester.
How fast can scripts do that?
Well, that would depend upon the environments being used, the complexity of the application under test, and the performance of the application.
For the most part, running automated scripts will always test more functionality than a manual tester. If there is a critical path that needs to be tested, the scripts can be run specifically for that and they will be executed the same each and every time. For many teams, cost comes into the picture. Most departments have budgets and test automation tools are usually pretty expensive. Once research is done, it will be found that there are many tools out there that can create scripts for a fairly inexpensive cost with even cheaper renewal costs. For the teams that do not have much experience with test automation, it would be better to try a small project with one of the less expensive tools. And remember, cheaper cost does not mean cheaper in the way the tool works, but it may mean that you don’t get all the bells and whistles. Compromises will always need to be added.
Should you use test automation for mobile device testing?
We can ask the same question for the test automation of a desktop application. It does not just need to be answered for mobile device testing. If your team plans to create subsequent updates to the app, or create the application in phases, I would highly recommend some sort of test script automation. If the plan is to just create a quick application one time and that is all, then it is up to the team to decide whether or not it is cost effective to automate the testing process. There will always be a benefit to test automation as a whole because test automation is not just script execution. Test automation is the whole testing process. That is where the real benefit for the whole project comes in. But for development and testers, script automation will help by allowing the manual testers to move onto some other critical tasks and leave the regression to the automation team.
Whether you have 4 or 400 testers, there will always be something that will be missed. Adding test automation will add test coverage and higher requirements coverage to your testing results. Mobile Device test automation will use a lot of the same processes that standard test automation uses. The tools may be different, but the rest will be similar.
(This article was originally published on www.olenick.com/blog/automation-mobile-testing/)
