Software Testing 3.0: Organization and Process Ramifications

Introduction

Many companies have come to realize that software testing is much more than a task that happens at the end of a software development cycle. They have come to understand that software testing is a strategic imperative and a discipline that can have a substantial impact on the success of an organization that develops software. Many of these companies are coming to embrace the ideas that are central to the latest phase in the evolution of software testing, Software Testing 3.0 (See the links below for more on Software Testing 3.0).

Implementing Software Testing 3.0 is a major undertaking, and as such, should not be embarked upon without adequate planning. While implementing Software Testing 3.0 will require substantial effort, the benefits in terms of software quality, cost and time savings, and mitigation of risk from in market failures are substantial.

Software Testing 3.0 has both organizational and process ramifications. These ramifications need to be fully understood to insure success implementing Software Testing 3.0. This paper will explore these organizational and process ramifications to help with an organization’s planning efforts.

Organizational Ramifications – Independent Software Testing Organization

One of the most fundamental concepts of Software Testing 3.0 is the need to make the software testing organization its own organization with its own budget. There are many reasons why this is so critically important:

    • Fiscal Oversite and AccountabilitySoftware testing is a very substantial part of any development process. It is estimated that software testing represents 40% of a development effort. As such, software testing is a large consumer of company resources, both money and people. Creating a separate organization with its own budget makes it much easier for an organization to understand and track the true costs of their testing efforts.In addition, a separate organization and budget makes it much easier to track and understand return on investment and hold the testing organization accountable for its expenses. Software testing as its own organization makes it much easier to track expenses as well as attribute cost savings.When looking at the cost savings that may be attributed to software testing it should be understood that calculating a return on software testing can be a difficult task. Unlike software development which yields a product that generates revenue, software testing is a cost savings effort. Effective software testing reduces defects released to customers and the associated high costs of implementing fixes for an in-market product. The easiest thing to measure in software testing is reductions in released bugs, for which most organizations should have the necessary data to calculate a cost savings. Measuring a return on more effective test coverage can probably be at least partially represented by the overall reductions in released bugs. Measuring a return on time to market may be more difficult. If a well implemented software testing effort helps to reduce overall time to market then it is probably reasonable to attribute at least some portion of early revenue to testing.
    • More Effective Stewardship of Product Quality / More Reliable CommunicationPutting the people who are in charge of developing a product, who are often under extreme schedule demands, in charge of the process that tests that software and potentially results in product delays is an inherent conflict of interest. It is like putting the rabbits in charge of the lettuce. There is a good chance that the results will not be what one would prefer.An engineering organization in charge of software testing may be tempted to under-resource testing in order to get a product to market faster. They may also be tempted to down-play the severity of encountered problems as they work toward critical delivery milestones.When a software testing is its own organization, it is organizationally empowered to provide more reliable information about the quality of the software under test and its readiness for market. It is much less likely that such an organization would underreport or gloss over potentially serious problems.
    • Risk Mitigation and Management

Managing risk has become critically important in the modern business environment. One of the largest risks that a software development organization faces is delivering to market software with substantial or catastrophic defects. Such an incident can damage a product’s and company’s reputation beyond repair. It can lead to declining revenue and the increased costs of trying to fix an in-market product. It can impact company morale and lead to employee turnover raising the costs of attracting and retaining talent.

With so much at stake, the organization that can have such a profound impact on software quality needs to be organizational empowered and have an equal seat at the table with development. This can give the appropriate amount of emphasis to the the information provided by software testing – the information that is critically important for engineering and business management to make informed ship / no-ship decisions.

Process Ramifications

In addition to the organizational changes that need to be made, there are many coinciding process changes that also need to be implemented. These include:

    • Involvement Early and OftenSoftware testing needs to be involved at the earliest stages of the product development process. They should be involved in early product definition meetings, as well as development design meetings. Doing so will give them critical insight into the product under development and the design trade-offs and compromises that may have been made. This information can be invaluable as software test cases are designed and automated yielding test cases that more effectively test both the design of a feature as well as the “spirit” of the feature.
    • Strategy before ToolsThere must be in place an overall strategy and methodology for software testing and automation prior to selecting any supporting tools. The strategy and methodology drives tool selection, as well as training and people development, automation efforts, and offshoring efforts. Strategy and methodology provide the roadmap for the other processes.
    • Metric Selection and AcceptanceA critically important process in Software Testing 3.0 is metric reporting to engineering and business management. Metrics, statistics about the software under test, provide the management organizations with insight into the quality of software under test and its readiness for market. A Software Testing 3.0 effort needs to have a set of defined metrics that are agreed to by all parties as well as a process for regularly reporting the metrics.
    • The Ability to Say NoBusiness management needs to make it clear that the software testing organization has the right to stand up and say that they do not feel that the a software product is ready for market without the “messenger being shot”. Only when software testing feels that they are truly empowered to provide their opinion will the company realize the maximum benefit from the software testing organization. Of course it is incumbent upon them to exercise their right “responsibly” with the appropriate well reasoned arguments in support of their assertions. They also need to understand that business management may override them because of other business considerations, but at least they will do so with a full and clear understanding of the risks they are incurring.
    • Develop Human Capital When implementing Software Testing 3.0, what was a task becomes a discipline. Because of this, it is important to develop and nurture that discipline, similar to the way a development organization is developed and nurtured. Part of this is providing software testers with adequate and appropriate training. Training needs to include strategy, methods, and tools. Training may also need to include “soft skills” such as social customs and language if offshoring is involved. In addition, attention needs to be paid to providing software testers (designers and automation engineers) with a career path both within testing and the overall organization. This does not mean that a tester becomes an engineer! If the testing organization has been doing its job right a software tester is not really trained to be a development engineer. The skills and training are different. It does, however mean, that positions within testing, engineering, and business management should be open to individuals in the testing organization.

Such “human capital” development within the testing organization, both on and offshore, will help in attracting and retaining talent.

Conclusion

Software Testing 3.0 can reap substantial rewards for any software development organization – rewards such as cost savings, increased testing coverage, fewer defects released to market, improved quality, and faster time to market. Implemented properly, a development organization can turn their Testing 3.0 efforts into a strategic competitive advantage. Implementing Software Testing 3.0 properly means paying close attention to the organizational and process imperatives for such an effort, which will ultimately make the effort easier to implement and more successful.

Other Software Testing 3.0 Articles

Rob Pirozzi

Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success. Proven ability to build and maintain strong relationships, contribute to target organization success, and deliver results. Website: http://www.robpirozzi.com/

Rob Pirozzi
Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success.

The Related Post

Internet-based per-use service models are turning things upside down in the software development industry, prompting rapid expansion in the development of some products and measurable reduction in others. (Gartner, August 2008) This global transition toward computing “in the Cloud” introduces a whole new level of challenge when it comes to software testing.
Companies generally consider the software they own, whether it is created in-house or acquired, as an asset (something that could appear on the balance sheet). The production of software impacts the profit and loss accounts for the year it is produced: The resources used to produce the software result in costs, and methods, tools, or ...
For mission-critical applications, it’s important to frequently develop, test, and deploy new features, while maintaining high quality. To guarantee top-notch quality, you must have the right testing approach, process, and tools in place.
VISTACON 2010 – Keynote: The future of testing THE FUTURE OF TESTING BJ Rollison – Test Architect at Microsoft VISTACON 2010 – Keynote   BJ Rollison, Software Test Architect for Microsoft. Mr. Rollison started working for Microsoft in 1994, becoming one of the leading experts of test architecture and execution at Microsoft. He also teaches ...
First, let me ask you a few questions. Are your bugs often rejected? Are your bugs often assigned back to you and discussed back and forth to clarify information? Do your leaders or managers often complain about your bugs?
The Testing Domain Workbook is the most extensive and exhaustive work you will ever find on a specific testing technique (or related techniques if you include equivalence class analysis and boundary testing as the book does). What I like best is the combination of academic background and roots combined with practical experience and industrial practice. All the concepts are ...
David S. Janzen – Associate Professor of Computer Science Department California Polytechnic State University, San Luis Obispo – homepage LogiGear: How did you get into software testing and what do you find interesting about it? Professor Janzen: The thing I enjoy most about computing is creating something that helps people. Since my first real job ...
The key factors for success when executing your vision.   There is an often cited quote: “…unless an organization sees that its task is to lead change, that organization—whether a business, a university, or a hospital—will not survive. In a period of rapid structural change the only organizations that survive are the ‘change leaders.’” —Peter ...
Plan your Test Cases with these Seven Simple Steps What is a mind map? A mind map is a diagram used to visually organize information. It can be called a visual thinking tool. A mind map allows complex information to be presented in a simplified visual format. A mind map is created around a single ...
Please note: This article was adapted from a blog posting in Karen N. Johnson’s blog on July 24, 2007. Introduction The password field is one data entry field that needs special attention when testing an application. The password field can be important (since accessing someone’s account can start a security leak), testers should spend more ...
Alexa Voice Service (AVS): Amazon’s service offering for a voice-controlled AI assistant. Offered in different products. Source: https://whatis.techtarget.com/definition/Alexa-Voice-Services-AVS Autopilot Short for “automatic pilot,” a device for keeping an aircraft on a set course without the intervention of the pilot. Source: https://en.oxforddictionaries.com/definition/us/automatic_pilot Blockchain Infrastructure: A complex, decentralized architecture that orchestrates many systems running asynchronously over the ...
Has this ever happened to you: You’ve been testing for a while, perhaps building off of a branch, only to find out that, after all of this time, there is something big wrong. It’s a bad build and now you have to go backwards, fix something, and get a new build. Basically, you just wasted ...

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe