Software Testing 3.0: Organization and Process Ramifications

Introduction

Many companies have come to realize that software testing is much more than a task that happens at the end of a software development cycle. They have come to understand that software testing is a strategic imperative and a discipline that can have a substantial impact on the success of an organization that develops software. Many of these companies are coming to embrace the ideas that are central to the latest phase in the evolution of software testing, Software Testing 3.0 (See the links below for more on Software Testing 3.0).

Implementing Software Testing 3.0 is a major undertaking, and as such, should not be embarked upon without adequate planning. While implementing Software Testing 3.0 will require substantial effort, the benefits in terms of software quality, cost and time savings, and mitigation of risk from in market failures are substantial.

Software Testing 3.0 has both organizational and process ramifications. These ramifications need to be fully understood to insure success implementing Software Testing 3.0. This paper will explore these organizational and process ramifications to help with an organization’s planning efforts.

Organizational Ramifications – Independent Software Testing Organization

One of the most fundamental concepts of Software Testing 3.0 is the need to make the software testing organization its own organization with its own budget. There are many reasons why this is so critically important:

    • Fiscal Oversite and AccountabilitySoftware testing is a very substantial part of any development process. It is estimated that software testing represents 40% of a development effort. As such, software testing is a large consumer of company resources, both money and people. Creating a separate organization with its own budget makes it much easier for an organization to understand and track the true costs of their testing efforts.In addition, a separate organization and budget makes it much easier to track and understand return on investment and hold the testing organization accountable for its expenses. Software testing as its own organization makes it much easier to track expenses as well as attribute cost savings.When looking at the cost savings that may be attributed to software testing it should be understood that calculating a return on software testing can be a difficult task. Unlike software development which yields a product that generates revenue, software testing is a cost savings effort. Effective software testing reduces defects released to customers and the associated high costs of implementing fixes for an in-market product. The easiest thing to measure in software testing is reductions in released bugs, for which most organizations should have the necessary data to calculate a cost savings. Measuring a return on more effective test coverage can probably be at least partially represented by the overall reductions in released bugs. Measuring a return on time to market may be more difficult. If a well implemented software testing effort helps to reduce overall time to market then it is probably reasonable to attribute at least some portion of early revenue to testing.
    • More Effective Stewardship of Product Quality / More Reliable CommunicationPutting the people who are in charge of developing a product, who are often under extreme schedule demands, in charge of the process that tests that software and potentially results in product delays is an inherent conflict of interest. It is like putting the rabbits in charge of the lettuce. There is a good chance that the results will not be what one would prefer.An engineering organization in charge of software testing may be tempted to under-resource testing in order to get a product to market faster. They may also be tempted to down-play the severity of encountered problems as they work toward critical delivery milestones.When a software testing is its own organization, it is organizationally empowered to provide more reliable information about the quality of the software under test and its readiness for market. It is much less likely that such an organization would underreport or gloss over potentially serious problems.
    • Risk Mitigation and Management

Managing risk has become critically important in the modern business environment. One of the largest risks that a software development organization faces is delivering to market software with substantial or catastrophic defects. Such an incident can damage a product’s and company’s reputation beyond repair. It can lead to declining revenue and the increased costs of trying to fix an in-market product. It can impact company morale and lead to employee turnover raising the costs of attracting and retaining talent.

With so much at stake, the organization that can have such a profound impact on software quality needs to be organizational empowered and have an equal seat at the table with development. This can give the appropriate amount of emphasis to the the information provided by software testing – the information that is critically important for engineering and business management to make informed ship / no-ship decisions.

Process Ramifications

In addition to the organizational changes that need to be made, there are many coinciding process changes that also need to be implemented. These include:

    • Involvement Early and OftenSoftware testing needs to be involved at the earliest stages of the product development process. They should be involved in early product definition meetings, as well as development design meetings. Doing so will give them critical insight into the product under development and the design trade-offs and compromises that may have been made. This information can be invaluable as software test cases are designed and automated yielding test cases that more effectively test both the design of a feature as well as the “spirit” of the feature.
    • Strategy before ToolsThere must be in place an overall strategy and methodology for software testing and automation prior to selecting any supporting tools. The strategy and methodology drives tool selection, as well as training and people development, automation efforts, and offshoring efforts. Strategy and methodology provide the roadmap for the other processes.
    • Metric Selection and AcceptanceA critically important process in Software Testing 3.0 is metric reporting to engineering and business management. Metrics, statistics about the software under test, provide the management organizations with insight into the quality of software under test and its readiness for market. A Software Testing 3.0 effort needs to have a set of defined metrics that are agreed to by all parties as well as a process for regularly reporting the metrics.
    • The Ability to Say NoBusiness management needs to make it clear that the software testing organization has the right to stand up and say that they do not feel that the a software product is ready for market without the “messenger being shot”. Only when software testing feels that they are truly empowered to provide their opinion will the company realize the maximum benefit from the software testing organization. Of course it is incumbent upon them to exercise their right “responsibly” with the appropriate well reasoned arguments in support of their assertions. They also need to understand that business management may override them because of other business considerations, but at least they will do so with a full and clear understanding of the risks they are incurring.
    • Develop Human Capital When implementing Software Testing 3.0, what was a task becomes a discipline. Because of this, it is important to develop and nurture that discipline, similar to the way a development organization is developed and nurtured. Part of this is providing software testers with adequate and appropriate training. Training needs to include strategy, methods, and tools. Training may also need to include “soft skills” such as social customs and language if offshoring is involved. In addition, attention needs to be paid to providing software testers (designers and automation engineers) with a career path both within testing and the overall organization. This does not mean that a tester becomes an engineer! If the testing organization has been doing its job right a software tester is not really trained to be a development engineer. The skills and training are different. It does, however mean, that positions within testing, engineering, and business management should be open to individuals in the testing organization.

Such “human capital” development within the testing organization, both on and offshore, will help in attracting and retaining talent.

Conclusion

Software Testing 3.0 can reap substantial rewards for any software development organization – rewards such as cost savings, increased testing coverage, fewer defects released to market, improved quality, and faster time to market. Implemented properly, a development organization can turn their Testing 3.0 efforts into a strategic competitive advantage. Implementing Software Testing 3.0 properly means paying close attention to the organizational and process imperatives for such an effort, which will ultimately make the effort easier to implement and more successful.

Other Software Testing 3.0 Articles

Rob Pirozzi

Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success. Proven ability to build and maintain strong relationships, contribute to target organization success, and deliver results. Website: http://www.robpirozzi.com/

Rob Pirozzi
Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success.

The Related Post

“Combinatorial testing can detect hard-to-find software faults more efficiently than manual test case selection methods.” Developers of large data-intensive software often notice an interesting—though not surprising—phenomenon: When usage of an application jumps dramatically, components that have operated for months without trouble suddenly develop previously undetected errors. For example, newly added customers may have account records ...
One of the most dreaded kinds of bugs are the ones caused by fixes of other bugs or by code changes due to feature requests. I like to call these the ‘bonus bugs,’ since they come on top on the bug load you already have to deal with. Bonus bugs are the major rationale for ...
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Article Synopsis There are many misconceptions about Software Testing. This article deals with the 5 most common misconceptions about how Software Testing differs from other testing. Five Common Misconceptions Some of the most common misconceptions about ...
One of the most common challenges faced by business leaders is the lack of visibility into QA activities. QA leaders have a tough time communicating the impact, value, and ROI of testing to the executives in a way that they can understand. Traditional reporting practices often fail to paint the full picture and do not ...
Let’s look at a few distinctions between the two process improvement practices that make all the difference in their usefulness for making projects and job situations better! An extreme way to look at the goals of these practices is: what makes your work easier (retrospective) versus what did someone else decide is best practice (post-mortem)? ...
Regardless of the method you choose, simply spending some time thinking about good test design before writing the first test case will have a very high payback down the line, both in the quality and the efficiency of the tests. Test design is the single biggest contributor to success in software testing and its also ...
Introduction Keyword-driven methodologies like Action Based Testing (ABT) are usually considered to be an Automation technique. They are commonly positioned as an advanced and practical alternative to other techniques like to “record & playback” or “scripting”.
March Issue 2019: Leading the Charge with Better Test Methods
Introduction All too often, senior management judges Software Testing success through the lens of potential cost savings. Test Automation and outsourcing are looked at as simple methods to reduce the costs of Software Testing; but, the sad truth is that simply automating or offshoring for the sake of automating or offshoring will only yield poor ...
MARCH 2016_ TEST DESIGN ISSUE
This article was originally featured in the May/June 2009 issue of Better Software magazine. Read the entire issue or become a subscriber. In my travels, I’ve worked with a number of companies that have attempted to assess the quality of their testing — or worse, their testers — using poorly considered metrics. Sometimes the measurement ...
People who follow me on twitter or via my blog might be aware that I have a wide range of interests in areas outside my normal testing job. I like to research and learn different things, especially psychology and see if it may benefit and improve my skills and approaches during my normal testing job. ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe