Introduction
Many companies have come to realize that software testing is much more than a task that happens at the end of a software development cycle. They have come to understand that software testing is a strategic imperative and a discipline that can have a substantial impact on the success of an organization that develops software. Many of these companies are coming to embrace the ideas that are central to the latest phase in the evolution of software testing, Software Testing 3.0 (See the links below for more on Software Testing 3.0).
Implementing Software Testing 3.0 is a major undertaking, and as such, should not be embarked upon without adequate planning. While implementing Software Testing 3.0 will require substantial effort, the benefits in terms of software quality, cost and time savings, and mitigation of risk from in market failures are substantial.
Software Testing 3.0 has both organizational and process ramifications. These ramifications need to be fully understood to insure success implementing Software Testing 3.0. This paper will explore these organizational and process ramifications to help with an organization’s planning efforts.
Organizational Ramifications – Independent Software Testing Organization
One of the most fundamental concepts of Software Testing 3.0 is the need to make the software testing organization its own organization with its own budget. There are many reasons why this is so critically important:
-
- Fiscal Oversite and AccountabilitySoftware testing is a very substantial part of any development process. It is estimated that software testing represents 40% of a development effort. As such, software testing is a large consumer of company resources, both money and people. Creating a separate organization with its own budget makes it much easier for an organization to understand and track the true costs of their testing efforts.In addition, a separate organization and budget makes it much easier to track and understand return on investment and hold the testing organization accountable for its expenses. Software testing as its own organization makes it much easier to track expenses as well as attribute cost savings.When looking at the cost savings that may be attributed to software testing it should be understood that calculating a return on software testing can be a difficult task. Unlike software development which yields a product that generates revenue, software testing is a cost savings effort. Effective software testing reduces defects released to customers and the associated high costs of implementing fixes for an in-market product. The easiest thing to measure in software testing is reductions in released bugs, for which most organizations should have the necessary data to calculate a cost savings. Measuring a return on more effective test coverage can probably be at least partially represented by the overall reductions in released bugs. Measuring a return on time to market may be more difficult. If a well implemented software testing effort helps to reduce overall time to market then it is probably reasonable to attribute at least some portion of early revenue to testing.
- More Effective Stewardship of Product Quality / More Reliable CommunicationPutting the people who are in charge of developing a product, who are often under extreme schedule demands, in charge of the process that tests that software and potentially results in product delays is an inherent conflict of interest. It is like putting the rabbits in charge of the lettuce. There is a good chance that the results will not be what one would prefer.An engineering organization in charge of software testing may be tempted to under-resource testing in order to get a product to market faster. They may also be tempted to down-play the severity of encountered problems as they work toward critical delivery milestones.When a software testing is its own organization, it is organizationally empowered to provide more reliable information about the quality of the software under test and its readiness for market. It is much less likely that such an organization would underreport or gloss over potentially serious problems.
- Risk Mitigation and Management
Managing risk has become critically important in the modern business environment. One of the largest risks that a software development organization faces is delivering to market software with substantial or catastrophic defects. Such an incident can damage a product’s and company’s reputation beyond repair. It can lead to declining revenue and the increased costs of trying to fix an in-market product. It can impact company morale and lead to employee turnover raising the costs of attracting and retaining talent.
With so much at stake, the organization that can have such a profound impact on software quality needs to be organizational empowered and have an equal seat at the table with development. This can give the appropriate amount of emphasis to the the information provided by software testing – the information that is critically important for engineering and business management to make informed ship / no-ship decisions.
Process Ramifications
In addition to the organizational changes that need to be made, there are many coinciding process changes that also need to be implemented. These include:
-
- Involvement Early and OftenSoftware testing needs to be involved at the earliest stages of the product development process. They should be involved in early product definition meetings, as well as development design meetings. Doing so will give them critical insight into the product under development and the design trade-offs and compromises that may have been made. This information can be invaluable as software test cases are designed and automated yielding test cases that more effectively test both the design of a feature as well as the “spirit” of the feature.
- Strategy before ToolsThere must be in place an overall strategy and methodology for software testing and automation prior to selecting any supporting tools. The strategy and methodology drives tool selection, as well as training and people development, automation efforts, and offshoring efforts. Strategy and methodology provide the roadmap for the other processes.
- Metric Selection and AcceptanceA critically important process in Software Testing 3.0 is metric reporting to engineering and business management. Metrics, statistics about the software under test, provide the management organizations with insight into the quality of software under test and its readiness for market. A Software Testing 3.0 effort needs to have a set of defined metrics that are agreed to by all parties as well as a process for regularly reporting the metrics.
- The Ability to Say NoBusiness management needs to make it clear that the software testing organization has the right to stand up and say that they do not feel that the a software product is ready for market without the “messenger being shot”. Only when software testing feels that they are truly empowered to provide their opinion will the company realize the maximum benefit from the software testing organization. Of course it is incumbent upon them to exercise their right “responsibly” with the appropriate well reasoned arguments in support of their assertions. They also need to understand that business management may override them because of other business considerations, but at least they will do so with a full and clear understanding of the risks they are incurring.
- Develop Human Capital When implementing Software Testing 3.0, what was a task becomes a discipline. Because of this, it is important to develop and nurture that discipline, similar to the way a development organization is developed and nurtured. Part of this is providing software testers with adequate and appropriate training. Training needs to include strategy, methods, and tools. Training may also need to include “soft skills” such as social customs and language if offshoring is involved. In addition, attention needs to be paid to providing software testers (designers and automation engineers) with a career path both within testing and the overall organization. This does not mean that a tester becomes an engineer! If the testing organization has been doing its job right a software tester is not really trained to be a development engineer. The skills and training are different. It does, however mean, that positions within testing, engineering, and business management should be open to individuals in the testing organization.
Such “human capital” development within the testing organization, both on and offshore, will help in attracting and retaining talent.
Conclusion
Software Testing 3.0 can reap substantial rewards for any software development organization – rewards such as cost savings, increased testing coverage, fewer defects released to market, improved quality, and faster time to market. Implemented properly, a development organization can turn their Testing 3.0 efforts into a strategic competitive advantage. Implementing Software Testing 3.0 properly means paying close attention to the organizational and process imperatives for such an effort, which will ultimately make the effort easier to implement and more successful.
Other Software Testing 3.0 Articles
- Software Testing 3.0: Delivering on the Promise of Software Testing
- Software Testing 3.0: Case Study #1
- Software Testing 3.0: Case Study #2
Rob Pirozzi
