| This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. |
Quality cost is the sum of all costs a company invests into the release of a quality product. When developing a software product, there are 4 types of quality costs: prevention costs, appraisal costs, internal failure costs, and external failure costs.
- Prevention costs represent everything a company spends to prevent software errors, documentation errors, and other product-related errors. These include requirements and usability analysis, for example. Dollars spent on prevention costs are the most effective quality dollars, because preventing errors from getting into the product is much cheaper than fixing errors later. If there is an error in a requirement or the intended usability, and money is spent on developing the software to the erroneous requirement, the costs of identifying the error, determining how to fix it, and then developing new code to correct it will arise later.
- Appraisal costs include the money spent on the actual testing activity. Any and all activities associated with searching for errors in the software and associated product materials falls into this category. This includes all testing by the developers themselves, by an internal test team, and by an outsourced software test organization. This also includes all associated hardware, software, labor, and other costs. Once a product is in the coding phases, the goal is to do the most effective appraisal job, so that internal failure work is streamlined, well-managed, and prevents skyrocketing external failure costs.
- Internal failure costs are the costs of coping with errors discovered during development and testing. These are bugs found before the product is released. As we mentioned previously, the further in the development process the errors are discovered, the more costly they are to fix. So, the later the errors are discovered, the higher their associated internal failure costs will be.
- External failure costs are the costs of coping with errors discovered after the product is released. These are typically errors found by your customers. These costs can be much higher than internal failure costs, because the stakes are much higher. These costs include post-release customer and technical support. Errors at this stage can also be costly in terms of your company’s reputation and may lead to lost customers.
The following table shows examples of costs that fall into each of the four categories of quality costs.
| Prevention | Appraisal |
|---|---|
| Staff training Requirements analysis Early prototyping Fault tolerant design Defensive programming Usability analysis Clear specification Accurate internal documentation Pre-purchase evaluation of the reliability of development tools |
Design reviews Code inspection Glass box testing Black box testing Beta testing Test Automation Usability testing Pre-release out-of-box testing by customer service staff |
| Internal Failure | External Failure |
| Bug fixes Regression testing Wasted in-house user time Wasted tester time Wasted writer time Wasted marketer time Wasted advertisements Direct cost of late shipment Opportunity cost of late shipment |
Technical support calls Answer books (for support) Investigating complaints Refunds and recalls Interim bug fix releases Shipping product updates Warranty, liability costs PR to soften bad reviews Lost sales Lost customer goodwill Supporting multiple versions in the field Reseller discounts to keep them selling the product |
The Four Categories of Quality Costs
All the costs mentioned above can be effectively reduced through smarter test efforts that include a high degree of Test Automation. Test Automation when done right leads to greater test coverage, resulting in higher-quality products. Higher-quality products require less technical support, fewer patches, and lead to greater customer satisfaction. Smarter Automated Testing also speeds up the release process and incrementally reduces the manual test costs. But most of all, more test coverage gives you and your customers more confidence in your product. You will feel more comfortable knowing that there are not bugs lurking in your software that have not been exposed yet because of insufficient test coverage. You will also not have to scramble at the last minute (typically, on the first day of that much-needed vacation) to deal with a problem and fix it to your customer’s satisfaction in a rush.
The solution to quality cost problems is to get a better understanding of your investment in product quality and manage your costs better. The first place most organizations look for a better understanding is in the highest cost area: the software test effort, or lack thereof. For example, if you do not test at all, your testing or appraisal cost is low. You will ship on time, but your external failure costs will skyrocket. Your prevention and appraisal costs will result in finding errors that can be corrected while they are still internal failures, where they are cheaper to deal with.
The goal of understanding quality costs is to analyze where you spend your time and money to get the most bang for the buck. It is well known that it is faster and cheaper to find and fix a bug during unit testing done by developers early in the development cycle. Should we then spend most of our time/budget on unit testing? No. There are many limitations to unit testing. Unit testing is not capable of finding many varieties of bugs, including graphical user interface (GUI) bugs, usability problems, End-to-End bugs, and configuration bugs. For most organizations, getting a better unit test effort will help you release a better product sooner. It is not a replacement for the test effort done by skilled software testers, but it may reduce the time that test effort takes. Understanding quality costs will hopefully help you shift some of your test effort to the most cost-effective places.
In the following, the total quality cost is shown in the upper bathtub-shaped curve. On the bottom axis is the quality of performance, ranging from totally defective to zero defects. On the left axis is the cost per good unit of product. You can see that with highly defective software, your prevention and appraisal costs are very low, but your failure costs are very high, yielding a high total quality cost. With zero defect software, likewise, your failure costs are very low, but your prevention and appraisal costs are very high. To optimize your total quality costs, you want to be between these extremes, at the bottom of the bathtub curve.
Theoretical Model of Optimum Quality Cost
This offers 2 challenges. First, a sufficiently sophisticated accounting system allowing a typical mid-sized company to track the total cost of quality has yet to be developed. To optimize total quality cost, you need to have the appropriate categories in your accounting system and keep track of the related costs. Second, you need to be able to track your external quality costs. You may not even have enough information from customers on why the software is not working for them. How are you going to know what to book into your accounting system for external failure costs? The point here is that while capturing this data is difficult and expensive, you know that the benefit is reducing your overall cost of quality. You need to determine if the benefits of tracking your total quality cost will give you enough of a return on investment to make setting up the appropriate accounting system and paying for the implementation of the program worthwhile.
Conclusion
Taking the time to better understand you quality costs will help you to much more cost effectively deliver a quality software product. You will be able to optimize the various costs to achieve the best quality achievable at a more reasonable price.
[1] J. M. Juran, Quality Control Handbook, 3rd Edition. New York: McGraw Hill, 1974.
Rob Pirozzi
