Four Drivers for Delivering Software Test Automation Success

Test automation provides great benefits to the software testing process and improves the quality of the results. It improves reliability while minimizing variability in the results, speeds up the process, increases test coverage, and ultimately can provide greater confidence in the quality of the software being tested. However, automation is not a silver bullet. It also brings some problems. The solution for test automation is to first define the test methodology, then choose the right enabling technology to help you implement the methodology. The chosen methodology should provide measurable improvements to the following four success drivers:

  • Visibility
  • Re-usability
  • Scalability
  • Maintainability

The four drivers above have a direct impact on three factors, manageability, productivity and cost efficiency, which ultimately lead to the tangible benefits that we are looking for a successful automation program to deliver.

Consider the following Figure 1. This diagram shows the results of a good test automation methodology. Good automation provides optimum productivity to the software testing effort. Hence, it leads to higher quality of the software releases. Test automation visibility provides measurability and control over the software development process, which keeps it manageable. With good visibility established, you can make effective management decisions about if, when, and how to do training and auditing to address the quality of tests. Reusability and scalability of test automation improves test productivity. However, productivity should be defined by (1) the quantity of tests (driven by reusability and scalability), and (2) the quality of tests (visibility into what the tests are actually doing helps improve the tests qualitatively). Quality and quantity are two different things. When test automation is reusable and scalable, the issue of quantity is resolved. When test automation is highly maintainable, the cost of ownership is minimized. Building test automation in a way that is more maintainable will lower the total cost of ownership of the test assets, making the overall testing effort more cost effective.

Figure 1. Outflows of Test Automation.

The quality of tests is mostly affected by the training and skills of the test staff. Automating a bad test does not improve its quality; it just makes it run faster. Good test design is a critical and often overlooked aspect of test automation. Test automation visibility by itself does not provide high quality tests. It merely enables us to have a view into how well the test designers are trained. Addressing the training issues will help in addressing test case quality issues.


Securing and developing competency is a secret to automation success.


Visibility, reusability, scalability, and maintainability lead to productivity and are the drivers for following benefits:

  • Improved time-to-market.
  • Improved quality of releases.
  • Improved predictability.
  • Improved Test/QA communication.
  • Higher test coverage.
  • Lower testing costs.
  • Earlier detection of bugs.
  • Lower technical support costs.
  • More effective use of testing resources.
  • Improved customer confidence and adoption.

After the test methodology and tools are set, the next step is to put the right people in place with the proper skills and training to do the work. The key to automation success is to focus your resources on the test production. That is to improve the quantity and quality of the tests, not to spend too many resources on automation production.


The most essential element to achieve these benefits is the methodology, not the tools.


In evaluating the return on investment with test automation, you need to look at the big picture. Think of the return-on-investment (ROI) equation, with the benefits on one side, and the costs on the other. For the benefits, consider the productivity, both in quantity and quality of tests. For the cost side of the equation, think about the reusability, scalability and maintainability of the tests in the context of the phases of a test automation effort:

  • Deployment costs
  • Test automation creation costs
  • Execution costs
  • Maintenance costs

Evaluate the ROI by considering if the costs are justified by:

  • Faster/more tests?
  • Faster/more test cycles?
  • Better test coverage in each cycle?
  • Higher quality of tests?

To make these evaluations, you need to have good visibility.

Often, the costs and benefits of test automation are uncertain due to lack of visibility. Management just does not know how much money it will spend on it, and how much benefit it will get from it. People are uncertain about how to quantify the ROI and how to set up a way to monitor it to see if they are on target. This is symptomatic to starting a project, and six months later something gets derailed.


Excellent visibility leads to effective management of test automation production.


Defining the test methodology before choosing the right enabling technology should provide measurable improvements in visibility, reusability, scalability, and maintainability yielding many tangible benefits from test automation efforts. To read more about this, see LogiGear’s book Global Software Test Automation: An Executive Snapshot of the Industry.

LogiGear Corporation

LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers public and corporate software-testing training programs worldwide through LogiGear University. LogiGear is a leader in the integration of test automation, offshore resources and US project management for fast and cost-effective results. Since 1994, LogiGear has worked with hundreds of companies from the Fortune 500 to early-stage startups, creating unique solutions to exactly meet their needs. With facilities in the US and Vietnam, LogiGear helps companies double their test coverage and improve software quality while reducing testing time and cutting costs. For more information, contact Joe Hughes + 01 650.572.1400

LogiGear Corporation
LogiGear Corporation provides global solutions for software testing, and offers public and corporate software testing training programs worldwide through LogiGear University. LogiGear is a leader in the integration of test automation, offshore resources and US project management for fast, cost-effective results. Since 1994, LogiGear has worked with Fortune 500 companies to early-stage start-ups in, creating unique solutions to meet their clients’ needs. With facilities in the US and Viet Nam, LogiGear helps companies double their test coverage and improve software quality while reducing testing time and cutting costs.

The Related Post

I’ve been teaching a lot lately, was in India for one week, and I’m off to Seattle in two weeks to teach on performance topics. I thoroughly enjoy teaching, it allows me to stay sharp with current trends, and provides a nice break from the “implementation focus” that I generally have day to day.
All too often, software development organizations look at automating software testing as a means of executing existing test cases faster. Very frequently there is no strategic or methodological underpinning to such an effort. The approach is one of running test cases faster is better, which will help to deliver software faster. Even in organizations that ...
One of my current responsibilities is to find ways to automate, as much as practical, the ‘testing’ of the user experience (UX) for complex web-based applications. In my view, full test automation of UX is impractical and probably unwise; however, we can use automation to find potential UX problems, or undesirable effects, even in rich, ...
The Cloud demands that we be as nimble as possible, delivering features and fixes in almost real-time fashion. Both customer and provider rely on software development that can maintain quality while being light on its feet and constantly moving. In addition, Cloud-oriented systems tend to be highly complex and dynamic in structure — more than ...
An automation framework is a way to organize your code in meaningful manner so that any person who is working with you can understand what each file contains. Automation frameworks differ based on how you organize your code – it can be organized based on your data, so that any person who wants to use ...
LogiGear Magazine September Issue 2020: Testing Transformations: Modernizing QA in the SDLC
When Netflix decided to enter the Android ecosystem, we faced a daunting set of challenges: 1. We wanted to release rapidly (every 6-8 weeks). 2. There were hundreds of Android devices of different shapes, versions, capacities, and specifications which need to playback audio and video. 3. We wanted to keep the team small and happy. ...
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Introduction There are many potential pitfalls to Manual Software Testing, including: Manual Testing is slow and costly. Manual tests do not scale well. Manual Testing is not consistent or repeatable. Lack of training. Testing is difficult ...
I feel like I’ve spent most of my career learning how to write good automated tests in an Agile environment. When I downloaded JUnit in the year 2000 it didn’t take long before I was hooked – unit tests for everything in sight. That gratifying green bar is near-instant feedback that everything is going as ...
LogiGear Magazine – The Big Testing Issue – April 2012
For this interview, we talked to Greg Wester, Senior Member Technical Staff, Craig Jennings, Senior Director, Quality Engineering and Ritu Ganguly, QE Director at Salesforce. Salesforce.com is a cloud-based enterprise software company specializing in software as a service (SaaS). Best known for its Customer Relationship Management (CRM) product, it was ranked number 27 in Fortune’s 100 ...
Over the years, we’ve provided an extensive number of articles that provide a wealth of knowledge about Test Automation. Below is a compilation of some of those articles. Guide to Automated Voice Apps Testing This article explores some of the basic test tools you’ll need and how to blend the use different automated testing tools ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe