SMU CSE 8314 / NTU SE 762-N Software Measurement and Quality Engineering

1. SMU CSE 8314 / NTU SE 762-NSoftware Measurement and Quality Engineering Module 10 Cost of Quality Analysis

2. Customer Value Low Cost or High Productivity Short Cycle Time or Schedule Quality (Fewer Defects; Customer satisfaction) We Will Examine Value from Three Perspectives

3. The Cost of Quality

4. Customer Value Low Cost or High Productivity Short Cycle Time or Schedule Quality (Fewer Defects; Customer satisfaction) The Cost of Quality Quality costs money But it can save money The issue: how to save more than it costs

5. Quality Engineering and Value • Quality engineering techniques can add value by • reducing the non-value-added costs • reducing the cost of non-value-added tasks, especially the essential ones • Showing reductions in cost is essential to obtain long term support of quality improvement initiatives

6. Beware ... • Since quality improvement initiatives are also non-value-added (according to the strict definition), they are often viewed as suspect • Both software engineers and software managers tend to understand the concept of a non-value-added task, but may need to be taught more about the role of quality improvement techniques

7. Other Views • Some prefer to avoid semantic or emotional arguments by categorizing certain tasks as non-value-added essential (or even as value-added), even though they really are not ... • quality improvement tasks • many management tasks • tasks done for political reasons The objective is to improve quality, not to be precise about definitions and categories

8. Raising Awareness of Quality Issues

9. Raising Awareness is a Prerequisite toCost of Quality Analysis • Managers and technical staff must be convinced that • Quality problems are serious • Poor quality costs them money • Quality is worth fixing • Quality can be fixed by proper techniques

10. Why Do We Need COQ Analysis • An organization may seek to avoid COQ analysis because • It is a non-value-added activity But ... • Since we are imperfect, we have to invest in order to improve quality • The investment is non-value-added • The benefit is to reduce the total non-value-added costs

11. Quality Assurance vs Quality Engineering • In traditional software QA, software engineers are responsible for the value-added tasks and QA engineers are responsible for many non-value-added quality tasks • This leads to a negative bias against quality improvement tasks • With QE, everyone is responsible for engineering the process to improve

12. “Quality is Free”Juran, Crosby • What they mean is that, in the long run, low quality costs more than high quality. • Philosophical issue: • Quality means suited to a purpose or fit for a specific use • Quality is relative • “Quality” is not free if it exceeds the need Understanding the needs is essential.

13. If you Propose toImprove Quality ... • Managers and technical staff and even customers will ask a series of very logical questions • You must anticipate these questions and understand how to answer effectively • And sometimes redirect their focus

14. The First Question What does it cost to have quality? • This is what management and technical staff will typically ask when approached with a proposal to improve quality.

15. A Better Question In other words, what is the payoff? What is the return on investment for improved quality?

16. Return on Investment • This is the question you want them to ask. • Return on Investment is the way to convince management • But it may not always convince the technical staff

17. The Second Question Why should I spend my time on quality assurance tasks when I have real engineering work to do? • This is what technical staff will typically ask when approached with a proposal to improve quality.

18. We have the best software in the business! Technical Staff are Often Influenced by Pride in Their Work

19. Pride in Workmanship • Technical staff may need to be shown how quality improvement relates to their pride in their work • This sometimes involves cultural factors unique to the organization • And usually requires management support of activities that promote pride in workmanship

20. Cost of Hardware Quality • Traditional analysis of the cost of hardware quality puts the focus on production or manufacturing: • Failure Rates • From Design Defects • From Physical Deterioration • Failures lead to Costs • Repair Costs -- Return to acceptable performance • Scrap Costs -- Discard and Replace • Rework Costs -- Do it Over

21. Cost of Hardware Quality • These costs are measured in terms of labor and materials and deterioration Failure Cost = xx% of deterioration rate Quality Improvement Cost = yy% of production cost

22. The Problem with Software • Software production costs are minimal • So measuring quality improvement cost as “% of production costs” makes software look terrible • Software deterioration is 0 (see next slide) • So some established methods of measuring failure costs are not applicable

23. Software Does “Deteriorate” ... Software “deterioration” is due to changes in the environment, accidents, data base damage, and addition of new features

24. Software Cost is Mostly Development Cost Sources of Quality Costs Design (10.0%) Deterioration (60.0%) Production (30.0%) Typical Hardware Product Production (1.0%) Deterioration (0.0%) Design (99.0%) Typical Software Product

25. Categorizing Quality-Related Costs 1) Cost of Conformance • Things that improve quality • Prevention Costs - Prevent poor quality • Appraisal Costs - Detect poor quality 2) Cost of Non-Conformance • The price of failure to achieve quality • Internal Failures - Costs before product shipment • External Failures - Costs after product shipment Strategy: Invest in conformance to save in non-conformance

26. Quality Related Costs Cost of Conformance Cost of Non-Conformance Prevention Costs Appraisal Costs Failure Costs Internal External Categorizing Quality-Related Costs

27. Invest in Prevention Prevention Costs Evaluation and Appraisal Costs Failure Costs Root Cause Analysis Predictive Measurements Preventive Measures Tests Reviews Inspections Audits Evaluations Failures in the field Loss of customers Support costs to Save Here Invest Here

28. Defects in Process } • Inspections • Audits • Reviews Defects in Design Root Cause Analysis Process Improvements What process error resulted in this product problem? Example of Prevention During the Design Process: Is the Product OK? Is the Process OK?

29. Factors Affecting Investment in Prevention • Productivity of development environment • Mandated prevention activities • Technologies in use • Types of products under development • Development environment • Budget and schedule • Process maturity • Knowledge of development team about • Application domain and software engineering

30. 60 50 40 30 20 10 5 2 3 4 1 Effects of Maturity on Costs Total COQ External Failures Cost as a Percent of Development Cost Internal Failures Appraisal Prevention SEI Maturity Level As reported by Knox (see references)

31. Analyzing the Cost of Quality • A Process is • A collection of tasks, • Carried out in a particular order, • Producing particular artifacts. • Each task can be analyzed for its value produced and its costs Alternatively, you can look at the costs and value of each artifact produced by the process

32. Model of a Task • What is Needed to Perform the Task • Raw Mtls • Data • Labor • Requirements • Training • Other Resources • Results of Performing the Task • Product • Services • Information • Performance • Reliability • Testability • Etc. • Task Name • Value added by Task • quality • cost • cycle time • etc. Inputs Outputs

33. Net Cost of Quality For a task whose purpose is to improve quality: Total Cost of Performing the Task minus Total Savings equals Net Cost of Quality (COQ) Goal: net COQ should be negative

34. In the Next Module ... • We will show how to analyze cost of quality and value-added together

35. References • Crosby, Philip B. Quality is Free, New York, McGraw-Hill, 1979. • Deming, W. Edwards, Out of the Crisis, MIT Press, 1986, ISBN: 0911379010 • Juran, J. M., Juran on Leadership for Quality: An Executive Handbook, The Free Press, 1989. • Knox, 1993, Raytheon studies reported by Houston and Keats, Software Quality Matters, vol 5, no 1 (Spring, 1997), U. of Texas SW Quality Institute

36. END OF MODULE 10