SMU CSE 8314 Software Measurement and Quality Engineering

1. SMU CSE 8314 Software Measurement and Quality Engineering Module 16 Six Sigma Principles and Applications

2. Contents • Six Sigma Principles • Six Sigma Applications • Summary

3. Six Sigma Principles

4. When you Manufacture the Product, the Samples Vary

5. Upper and LowerSpecification Limits Lose Win Lose Lower Spec Limit Upper Spec Limit

6. Tight Upper and LowerSpecification Limits Lose Win Lose

7. Low Spec High Spec Normal Distribution of Valuesfor a Measurable Characteristic Frequency of Those Values Value of Characteristic

8. The Normal Distribution Curve

9. Characterizing the Curve • x is the value of an individual product parameter or characteristic (horizontal axis) • For rolling dice, it would be any number from 1 to 12 • N is the total number of products produced (population size) • For rolling dice, it would be the number of rolls

10. å x m = N (x-m)2 å s = N Mean and Standard Deviation • The mean, m, is the average: • The standard deviation, s, explains how much the individual samples vary from the mean:

11. Mean and Standard Deviation Mean (m) Number of Products with This Value - 1 + 1 + 2  Value of Product Characteristic (x)

12. What is the Standard Deviation? • It is a measure of the variability of data • In general, standard deviation is proportional to dispersion of the data • For normally distributed data: • 68.26% of the data are within 1  of the mean • 95.46% are within 2  of the mean • 99.73% are within 3  of the mean • 99.9937% are within 4  of the mean • 99.999943% are within 5  of the mean • 99.9999998% are within 6  of the mean

13. Defects and Spec Limits • A defect is any data outside the spec limits • For normally distributed data: • 68.26% of the data are within 1  of the mean • Thus if the spec limits are ±1 , 31.74% of the data are defective (317,400 per million) • ±2  = 4.54% defective (45,400 per million) • ±3  = .27% defective (2,700 per million) • ±4  = .0063% defective (63 per million) • ±5  = .000057% defective (0.57 per million) • ±6  = .0000002% defective (0.002 per million)

14. Most Products Have Sigma Level of Between 3 and 4 • This seems to be a natural tendency for human processes that have not been intentionally improved for quality • Examples: • IRS phone advice (1-2 sigma) • Mail delivery: (3-4 sigma) • Hospital billing errors (3-4 sigma) • But: accidental hospital fatalities are closer to 6 sigma

15. Computing Sigma and Mean when you Cannot Measure Every Case • You can take a sample from the complete population and use this to make an estimate of the mean and the standard deviation • This works better if the sample size is larger

16. “Sample” Method for Calculation of Mean and Standard Deviation • xis the value of a product parameter (horizontal axis) • n is the total number of products measured (population size)

17. The Normal Distribution Curve with Relaxed Specification Limits

18. Normal Distribution Curvewith Tight Specification Limits How can we Meet These Tight Specifications?

19. Method I Design for Producability(Relax the Specification Limits)

20. Analogies to Software • Less complex architecture or design results in fewer coding and maintenance errors • More precise or better understood requirements result in fewer design errors • Tests developed during requirements phase provide greater probability that system test will correctly test that the requirements are met

21. Method II Sorting the Output Normal Products at Normal Price Normal Products at Normal Price Premium Products at High Price

22. Analogies to Software • The focus of quality efforts on most used parts of the software • Testing • Inspections • Best software engineers • etc. • Apply the best people or most intense efforts to the riskiest parts of the design or the architecture

23. Method III Improve the Process(Reduce the Variance)

24. Analogies to Software • More effective architectures • Better designs • Better coding practices • Better testing practices • Detecting and correcting defects at every step of the process • Better planning of integration • ...

25. Process Phase Shift • All of the above assume that the process continues to produce the same normal distribution, centered about the same mean, from day to day and year to year • Or else variance improves as we get more experienced with the process • But in actual practice, the mean varies from day to day due to incidental aspects of the environment

26. Causes of Variance Cannot Always be Controlled • In the case of production, this could be due to such factors as: • Ambient temperature and humidity • State of repair of production equipment • Calibration of equipment • Morale of operators • Variations in characteristics of raw materials, etc

27. Variance in Design Processes • In the case of design processes, similar factors influence process variation • Thought experiment: identify some factors that might cause variance in day-to-day performance of software design and development tasks

28. Shift Right Many Data are Too Large for Spec Limits No Shift Most Data are Within Spec Limits Shift Left Many Data are Too Small for Spec Limits Examples of Phase Shift

29. Compensation for Phase Shift The same principles apply: • If you widen the spec limits, you leave more room for the shift to occur • Sometimes called “widening the road” • If you reduce the process variance you also leave more room for the shift to occur • Sometimes called “narrowing the car”

30. Widen the Road orNarrow the Car Shift Right Most Data are Still Within the Spec Limits No Shift Most Data are Within Spec Limits Shift Left Most Data are Still Within the Spec Limits Note: To accommodate a maximum ±1.5  phase shift, we redefine the goals. A 6  goal becomes a 4.5  goal after the phase shift.

31. “Six Sigma” is Really 4.5 Sigma 4.5  for shifted case 6  for normal case

32. But Remember ... • The goal is not the number • The goal is quality improvement to meet customer requirements and expectations • Six sigma methods help you improve, regardless of the number you are at

33. How Do We Show That the Shift Is Acceptable Relative to Tolerances? DESIGN TOLERANCE DESIGN MARGIN = -------------------------------------- PROCESS VARIANCE • This is a measure of the overall effect. • It is also called the Capability Index • We use the term CP for this

34. Both Factors Must be Analyzed • If you improve the design, it makes the process variations more tolerable • If you improve the process, it makes the design less critical to overall success • If you improve both, the quality gets significantly better -- significant leaps are possible

35. Design Tolerance • This is the extent to which the Design allows Variance in Production • It is the “Width of the Road” • Formula for Design Tolerance is: | Upper Spec Limit - Lower Spec Limit | • If both spec limits are the same distance from the mean, then design tolerance is: 2 * | Upper Spec Limit - µ |

36. Process Capability(Process Variance) • This factor reflects your quality goal. I.e., what is the probability that you will produce something within tolerance. • As a somewhat arbitrary choice, the process variation is usually selected to be: ± 3  • or, in other words, a range of 6 

37. Short Term Process Capability Index or Design Margin | Upper Spec Limit - Lower Spec Limit | CP = ------------------------------------------------ 6  OR | Upper Spec Limit - µ | CP = -------------------------------- 3 

38. Some Possible Goals • CP = 1 means the spec limits are ± 3 • a 3 sigma process • CP = 2 means the spec limits are ± 6 • a 6 sigma process • CP > 2 means it is relatively easy to meet the specification limits, because the design margins are wide enough relative to the process capability

39. Phase Shift Measurement | Target Mean - Actual Mean | K = phase shift = ------------------------------------------ 1/2 (USL-LSL) • LSL = Lower Spec Limit • USL = Upper Spec Limit • K = 0 means no shift • K = 1 half of the data are out of spec

40. CPK CPK = CP * (1-K) CPK < 0 means most samples are bad CPK = 0 means half bad, half good CPK > 1 means most are good This can be used to monitor the actual performance of a process to see if it is producing mostly good products

41. What is this Really Telling Us? • The spec limits for a process step are merely the outputs of the previous step • Thus the goal of each step is to have wide spec limits for the next step (design tolerance) and a low process variance (high quality output from each process step)

42. Rolling Throughput • The cumulative effect of the individual step variances for the entire process • It is analogous to the concept that for each step of the process, we have: • defects coming in (from prior steps), plus defects introduced (in this step), minus defects corrected. • By tracking the , Cp & Cpk values for a process, we can monitor • The overall quality of the result & the sources of the problems

43. Defect Accumulation F = Defects Found and Fixed Process Step I = Defects Input O = Defects Output O = I + C - F C = Defects Created

44. Computing the Rolling Throughput • If step i has probability pi of having a defect The overall probability of defects is: P =  pi Parts or StepsOverall Yield of Defect Free Products 3 4 5 6 1 93.3% 99.4% 99.97% 99.9996% 100 10% 53.6% 97.7% 99.966% 1000 --- 0.2% 77.2% 99.66% 150,000 --- --- --- 60%

45. Step i Step j Step k Application toSoftware Process • At each step, minimize the defects in the output • Measure Defects Found • Measure Defects Created • Set Targets for Defect Levels Allowed at Each Step • Perform Causal Analysis to Fix Sources of Defects

46. What does Six Sigma Cause One to Do Differently Short Term Impact: • Continuous Process Improvement • Know what you need • Know what your process can do • Measure and characterize the process Long Term Impact: • Process Reinvention • You cannot get there without it • You can tell where you need it the most

47. 3 Sigma vs. 6 Sigma 3 6 Defective

48. Application from Real Life(see “Pasco” in reference list) “U.S. 19 overpasses may get narrow lanes” • But experts fear the 11-foot-wide lanes, proposed to cut costs, also tread on safety. “The state transportation department’s latest plan to fix U.S. 19 ... drivers may start feeling squeezed from a different direction: the sides. ... state planners have proposed shaving 1 foot off the standard 12-foot-wide lanes for a series of overpasses. ... ‘It’s a bad idea,’ said David Willis, president of the AAA Foundation for Traffic Safety. “There’s a proven relationship between lane width and traffic safety.”

49. Grand Marquis 70 inches Fed Ex Truck 92 inches PSTA Bus 102 inches Overpass 11 feet (132 inches) Graph from Newspaper

50. Using Six Sigma