The Importance of Volume in Automated Software Testing

All too often, software development organizations look at automating software testing as a means of executing existing test cases faster. Very frequently there is no strategic or methodological underpinning to such an effort. The approach is one of running test cases faster is better, which will help to deliver software faster.

Even in organizations that start to build a test automation effort on a good foundation of strategy, methodology, and supporting tools, taking such a narrow view of the test automation effort misses a huge dynamic.

Simply running existing test cases faster reduces testing time but does not increase testing capacity. Less time is taken to test, a good thing, but software testing may still be falling woefully behind software development and still represent a huge bottleneck. This is further exacerbated by the rapid increases in productivity that software development has undergone and continues to enjoy.

This approach can also lead to another detrimental dynamic. From this approach, often times organizations look to hand the test automation off to inexpensive low-skill organizations that have relatively little experience with testing and test automation – often times offshore. While this can work given the relatively modest goals for automation it creates a future problem. At some point in time the organization is going to come to realize that they need to substantially increase the volume of testing in order to keep pace with development, and they will have entirely the wrong team in place to accomplish this.

More than Automation, the Goal Should be High Volume Automation

To be truly successful in test automation requires a change in emphasis. While it is commendable to do the existing testing faster, the goal must be to do a very high volume of test automation. This will allow software testing to truly keep pace with development. By setting this as a goal, and putting in place the necessary processes, organization, and partners, a software testing organization can keep pace with development and make valuable contributions to software quality.

The Keys to High Volume Testing and Test Automation

What, then, are the keys to success in high-volume test automation? There are many key ingredients, including:

  • An overall software testing strategy that informs methodology and tool selection.
  • A methodology that supports the strategy and makes it easy to design and implement tests. As important, the methodology must also make it easy to maintain automated test cases. An organization does not want to trade the bottleneck of manual testing with a new bottleneck – maintaining test cases. Easy to maintain and reuse test cases are critically important to obtaining high levels of test automation and testing.
  • Software testing tools that support the strategy and methodology and make it easy to design, implement, maintain, and reuse test cases.

Another factor that can be critical to the success of such an effort is engaging the right partner to facilitate the process. Organizations that feel they are weak in the strategy and process department when it comes to software testing should look to a partner who can help them to align quality assurance objectives with business objectives. Such an organization can help to:

  • Define and refine objectives
  • Align objectives and business vision
  • Maximize test efficiencies with the latest methodologies and tools

Companies with a handle on strategy and process may wish to engage a partner who understands how to translate the vision into a high-volume testing program; one with experience implementing high-volume methodologies and the tools that support them. Equally as important is a partner with experience employing global resource strategies to help keep costs down.

Benefits of High Volume Testing

There are many benefits to a testing program that sets goals for very high volumes of developed, maintained, and executed test cases. These include:

  • A software testing organization that can keep up with the pace of development
  • Increased testing coverage
  • Improved software quality (assuming that good test cases are designed and automated)

Conclusion

Successful test automation programs do not simply aim to run existing test cases faster. Truly successful automated software testing programs set much more lofty goals. They set the goal of providing truly high volume testing and test automation. They implement the right strategies, methodologies, and tools to support high volume. Doing so, testing can keep pace with development and make big contributions to software quality.

Rob Pirozzi
Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success.

The Related Post

The Cloud demands that we be as nimble as possible, delivering features and fixes in almost real-time fashion. Both customer and provider rely on software development that can maintain quality while being light on its feet and constantly moving. In addition, Cloud-oriented systems tend to be highly complex and dynamic in structure — more than ...
One of the basic challenges with test automation is adoption. I can’t tell you how many times I’ve cataloged licenses for a company and found out they already have many different automation software packages, none of which is being used. Traditionally I’ve been told that is because the tools don’t work and that the teams ...
How to do UI test automation with the fewest headaches I’m currently interviewing lots of teams that have implemented acceptance testing for my new book. A majority of those interviewed so far have at some point shot themselves in the foot with UI test automation. After speaking to several people who are about to do ...
Developers of large data-intensive software often notice an interesting — though not surprising — phenomenon: When usage of an application jumps dramatically, components that have operated for months without trouble suddenly develop previously undetected errors. For example, the application may have been installed on a different OS-hardware-DBMS-networking platform, or newly added customers may have account ...
One of my current responsibilities is to find ways to automate, as much as practical, the ‘testing’ of the user experience (UX) for complex web-based applications. In my view, full test automation of UX is impractical and probably unwise; however, we can use automation to find potential UX problems, or undesirable effects, even in rich, ...
LogiGear Magazine – The Big Testing Issue – April 2012
Divide and conquer was a strategy successfully employed by ancient Persian kings against their Greek enemies. It is a strategy that can still be used successfully today. Fundamentally, by dividing something into smaller more manageable pieces (in the case of the ancient Persians, they divided the Greek city states), it becomes much more manageable.
There are few topics in quality assurance testing that cause as much confusion as smoke testing versus sanity testing. The two names would seem to describe very different practices— and they do! But people still get them confused, since the distinction is somewhat subtle.
“Testing Applications on the web” – 2nd EditionAuthors: Hung Q. Nguyen, Bob Johnson, Michael HackettPublisher: Wiley; edition (May 16, 2003) This is good book. If you test web apps, you should buy it!, April 20, 2001By Dr. Cem Kaner – Director of Florida Institute of Technology’s Center for Software Testing Education & Research Book Reviews ...
Investing in Test Automation training will increase your team’s productivity. The availability of reliable jobs in a competitive US market seems to be constantly embattled with competition and replacements of artificial intelligence (AI). In 2016, Foxconn replaced 60,000 employees with robots. However, the growth of Test Automation as an occupation has highlighted an intriguing option ...
An Overview of Four Methods for Systematic Test Design Strategy Many people test, but few people use the well-known black-box and white-box test design techniques. The technique most used, however, seems to be testing randomly chosen valid values, followed by error guessing, exploratory testing and the like. Could it be that the more systematic test ...
This book isn’t for everyone, but everyone can get some value out of it. What I mean by that rather confusing statement is that folks working in Agile environments will likely want to throw the book across the room while folks in more bureaucratic environments like CMMI or other waterfall environments will likely get a ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe