Software Testing 3.0: Delivering on the Promise of Software Testing

This is an adaptation of a presentation entitled Software Testing 3.0 given by Hung Nguyen, LogiGear CEO, President, and Founder. The presentation was given as the keynote at the Spring 2007 STPCON conference in San Mateo, California.

Watch for an upcoming whitepaper based on this topic.

Introduction

This first article of this two article series (The Early Evolution of Software Testing), discussed the early phases of software testing (Software Testing 1.0 and 2.0). It highlighted how both of these phases have largely underperformed expectations and discussed the shortcomings of these early phases of software testing.

Software Testing is now on the threshold of the next major evolutionary development in software testing – Software Testing 3.0. Many organizations are starting to realize the strategic importance of Software Testing 3.0, and others are starting to adopt Software Testing 3.0 principals in their Software Testing programs that are now finally delivering better software faster and meeting management expectations.

This article discusses Software Testing 3.0 and how it delivers on the promise of software testing. It also discusses why senior management will find this evolutionary step strategically important.

Software Testing 3.0

The foundation for Software Testing 3.0 is a strategic end-to-end framework for change based upon a strategy to drive testing activities, tool selection, and people development. Fundamental to its success is the realization that testing is its own function, and an integral part of the development process. Testing must have its own budget to build the necessary infrastructure, as well as clear leadership and ownership. It must provide executive management with visibility into the status of the software under test so that executives can make intelligent and informed decisions about projects and have confidence that they are developing and delivering a quality product that will meet customer needs.

Moving from Software Testing 2.0 to Software Testing 3.0 requires a strategic commitment on the part of the organization, and elevation of software testing from a task or tactical effort to a “strategic effort”. An organization must form a team or task force to address the needs of people, practice, and process (SP3T). This task force will develop a team-based methodology to drive tool selection (tools that support the methodology) as well as people development (training in the methodology, tools, and software testing techniques that support the methodology), and determination of necessary metrics to provide management with visibility into the status and quality of the software under test.

This transformation of culture, methodology, beliefs, and techniques will yield a team-based software testing program that is test design focused with an emphasis on testing as a whole rather than in parts. It will address feature reliability; compatibility, usability; and interoperability; performance acceptability, availability, and scalability; security; as well as accessibility. More importantly, it will yield a software testing program that:

  • Improves quality by increasing testing coverage and effectiveness
  • Decreases testing time
  • Decreases costs
  • Provides the necessary information for informed decision making
  • Provides several assets, including:
    • Tested, high quality software
    • Reusable software tests that codify organization knowledge about the software under test and provide the basis for the testing of future generations of that software
  • Focuses on test design and agile test development practices, and
  • Extends test automation to multiple platforms including non-GUI platforms

What Software Testing 3.0 Means for Senior Management

Software Testing 3.0, treating software testing as a strategic effort, enables an organization to turn quality into a strategic advantage. To be convinced of the value of this proposition, it is important for senior management to understand the “whys” of Software Testing 3.0.

    • Why Treat Software Testing as a Strategic Initiative?Software development represents 40% of the product release budget for any company that develops software for resale (LogiGear internal studies). Software testing typically represents 40% of the development budget. Any company task that represents that much money should be treated as a strategic initiative. Strategically “doing-the-right-thing” coupled with “doing it right” can have a profound impact both on the outcome as well as the cost. Conversely, treating it as a tactical initiative can (and will) yield sub-optimal results and lead to unnecessary costs. Put simply, Software Testing 3.0 will save time and money, and result in higher quality software.
    • Why Should Software Testing be its Own Organization with a Separate Budget?There are several reasons why software testing should be a separate organization from development, with its own budget. First, it is important to realize that software testing performs functions that are very different from software development. Development is a creative process of providing new functionality and features. Testing is a destructive process where testers actually try to “break” the software so that customers do not.Testers are not developers. The skills and experience needed differ from those of a developer. Having testing as its own organization with its own budget allows testing-centric management to nurture and develop testing-specific skills in their testers (skills in methodology, tools, metric reporting, etc.).Having testing as its own organization also gives it the independence necessary to “stand up” to development when there are problems with the software under test. It is much less likely that development will be able to ram sub-standard code through an independent testing organization to meet delivery promises without a red flag being waved to senior management. While this may potentially cause delays in delivery, these are delays that can prevent significant down-stream costs as well as lost revenue and customer good-will that results when bad software makes it to market. Having software testing and its budget separate prevents testing getting short changed or circumvented and will help to deliver higher quality products.Software Testing’s budget should also be separate because there are large amounts of money involved. Having the budget separate gives the organization the ability to exert more direct fiscal control and oversight. With an independent budget, software testing becomes much easier to measure the return and the costs savings that may be realized from initiatives such as automation or offshoring of testing efforts.
    • Why is Visibility so Important?Software Testing performs several key functions for the benefit of the company, product development, and company management. Testing:
      • Is a service to the company that helps the organization produce and release higher quality software
      • Is also a service to the development team to help it produce higher quality code
      • Is an information service for senior management providing information about product quality
      • Organizational improvement
      • Better confidence in the quality (consistency and dependability) of the software
      • Better decision making, and better confidence in the decision making because it is based on data
      • Fewer “surprises”
      • Better allocation and utilization of resources and budget
      • Ability to spend more time and money on development and less on maintenance
      • Delivering higher quality software

Testing performs these services by providing key metrics or data, including test cases run, testing coverage statistics, defect reports and more. This data provides visibility into the status and quality of the software under test. This data can be used by the testing group itself to further refine and enhance testing efforts. Development can use this data to gain an understanding of what parts of the product are causing problems, which may lead to new processes or development tactics to deliver better code to testing. Senior management uses this data to make critical decisions about product schedules and delivery, as well as resource allocations.

With visibility comes:

Conclusion

Software testing has undergone an evolutionary process. The early phases emphasized tactics and tools and yielded sub-optimal results. As with any evolutionary process, it is imperative for an organization to continue to evolve, or perish. Software Testing 3.0 is the next phase in the evolutionary development of software testing.

Organizations that make the transition to viewing testing as a strategic initiative – organizations that put testing on a even footing with development by making testing its own organization with its own budget, place methodology and strategy first and then move on to tool selection – will be able to turn quality into a competitive advantage.

  • They will have happier customers.
  • They will have a happy sales force with less turnover.
  • They will meet or exceed their marketing and promotional claims.
  • They will deliver quality software to channels and customers in a timely manner.
  • They will minimize potentially damaging product failures.

All of this will help to improve sales and generate revenue, while reducing costs and speeding product to market. Software Testing 3.0 is an evolutionary imperative for any organization that develops software.

 

Hung Nguyen

Hung Nguyen co-founded LogiGear in 1994, and is responsible for the company’s strategic direction and executive business management. His passion and relentless focus on execution and results has been the driver for the company’s innovative approach to software testing, test automation, testing tool solutions and testing education programs.

Hung is co-author of the top-selling book in the software testing field, “Testing Computer Software,” (Wiley, 2nd ed. 1993) and other publications including, “Testing Applications on the Web,” (Wiley, 1st ed. 2001, 2nd ed. 2003), and “Global Software Test Automation,” (HappyAbout Publishing, 2006). His experience prior to LogiGear includes leadership roles in software development, quality, product and business management at Spinnaker, PowerUp, Electronic Arts and Palm Computing.

Hung holds a Bachelor of Science in Quality Assurance from Cogswell Polytechnical College, and completed a Stanford Graduate School of Business Executive Program.

Rob Pirozzi

Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success. Proven ability to build and maintain strong relationships, contribute to target organization success, and deliver results. Website: http://www.robpirozzi.com/

Hung Q. Nguyen
Hung Nguyen co-founded LogiGear in 1994, and is responsible for the company’s strategic direction and executive business management. His passion and relentless focus on execution and results has been the driver for the company’s innovative approach to software testing, test automation, testing tool solutions and testing education programs. Hung is co-author of the top-selling book in the software testing field, “Testing Computer Software,” (Wiley, 2nd ed. 1993) and other publications including, “Testing Applications on the Web,” (Wiley, 1st ed. 2001, 2nd ed. 2003), and “Global Software Test Automation,” (HappyAbout Publishing, 2006). His experience prior to LogiGear includes leadership roles in software development, quality, product and business management at Spinnaker, PowerUp, Electronic Arts and Palm Computing. Hung holds a Bachelor of Science in Quality Assurance from Cogswell Polytechnical College, and completed a Stanford Graduate School of Business Executive Program.
Hung Q. Nguyen on Linkedin
Rob Pirozzi
Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success.

The Related Post

With complex software systems, you can never test all of the functionality in all of the conditions that your customers will see. Start with this as a fact: You will never test enough! Step 2 in getting started is to read and re-read The Art of Software Testing by Glenford Myers. This classic will set the ...
Companies generally consider the software they own, whether it is created in-house or acquired, as an asset (something that could appear on the balance sheet). The production of software impacts the profit and loss accounts for the year it is produced: The resources used to produce the software result in costs, and methods, tools, or ...
People who follow me on twitter or via my blog might be aware that I have a wide range of interests in areas outside my normal testing job. I like to research and learn different things, especially psychology and see if it may benefit and improve my skills and approaches during my normal testing job. ...
Karen N. Johnson began as a technical writer in 1985 and later switched to software testing in 1992. She maintains a blog at TestingReflections, a collaborative site where she is featured as a main contributor. In her latest entry, she discusses search testing with different languages. Here is an excerpt from her blog: “I started ...
One of the most dreaded kinds of bugs are the ones caused by fixes of other bugs or by code changes due to feature requests. I like to call these the ‘bonus bugs,’ since they come on top on the bug load you already have to deal with. Bonus bugs are the major rationale for ...
Alexa Voice Service (AVS): Amazon’s service offering for a voice-controlled AI assistant. Offered in different products. Source: https://whatis.techtarget.com/definition/Alexa-Voice-Services-AVS Autopilot Short for “automatic pilot,” a device for keeping an aircraft on a set course without the intervention of the pilot. Source: https://en.oxforddictionaries.com/definition/us/automatic_pilot Blockchain Infrastructure: A complex, decentralized architecture that orchestrates many systems running asynchronously over the ...
The V-Model for Software Development specifies 4 kinds of testing: Unit Testing Integration Testing System Testing Acceptance Testing You can find more information here (Wikipedia): http://en.wikipedia.org/wiki/V-Model_%28software_development%29#Validation_Phases What I’m finding is that of those only the Unit Testing is clear to me. The other kinds maybe good phases in a project, but for test design it ...
Test design is the single biggest contributor to success in software testing. Not only can good test design result in good coverage, it is also a major contributor to efficiency. The principle of test design should be “lean and mean.” The tests should be of a manageable size and at the same time complete and ...
VISTACON 2010 – Keynote: The future of testing THE FUTURE OF TESTING BJ Rollison – Test Architect at Microsoft VISTACON 2010 – Keynote   BJ Rollison, Software Test Architect for Microsoft. Mr. Rollison started working for Microsoft in 1994, becoming one of the leading experts of test architecture and execution at Microsoft. He also teaches ...
LogiGear Magazine March Testing Essentials Issue 2017
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Article Synopsis There are many misconceptions about Software Testing. This article deals with the 5 most common misconceptions about how Software Testing differs from other testing. Five Common Misconceptions Some of the most common misconceptions about ...
LogiGear Magazine – February 2014 – Test Methods and Strategies

Leave a Reply

Your email address will not be published.

Stay in the loop with the lastest
software testing news

Subscribe