Measurement: Assessment and Metrics

1. Measurement: Assessment and Metrics Presented by Dr. Joan Burtner Certified Quality Engineer Associate Professor of Industrial Engineering and Industrial Management

2. Overview • Process Measurement as a Management Function • Project Management Metrics • Human Aspects of Data Gathering • Statistical Analysis • Theory of Variation • Process Capability • Acceptance Sampling Dr. Joan Burtner, Associate Professor of Industrial Engineering

3. Process Measurement as a Management Function • “Effective management of an organization depends on defining, gathering, and analyzing information that provides feedback on current performance as well as projecting future needs.” ETM627 Course Text p. 416 • “Analysis refers to extracting larger meaning from data and information to support evaluation, decision making, and improvement.” Baldrige National Quality Program 2005 • “Statistics is the science of turning data into information” ETM627 Course Text p. 417 Dr. Joan Burtner, Associate Professor of Industrial Engineering

4. Typical Project Management Metrics • Schedules met • Resources used • Costs versus budget • Project objectives met • Risks identified and eliminated or mitigated • Earned value analysis (planned vs. actuals) • Customer satisfaction Dr. Joan Burtner, Associate Professor of Industrial Engineering

5. Human Aspects of Data Gathering • Perception that excessive data collection and development of multiple metrics is: • A reflection of management’s obsession with numbers • Not necessarily helpful in producing a better product or service • Perception that organization is more interested in data collection than task performance • Lack of understanding of the connection between what workers produce and the metrics by which management assesses performance Dr. Joan Burtner, Associate Professor of Industrial Engineering

6. Statistical Analysis • Central Tendency • Mean • Median • Mode • Variation or Spread • Range • Standard Deviation • Variance Dr. Joan Burtner, Associate Professor of Industrial Engineering

7. Probability Distributions • Widely-used distributions • Normal • Exponential • Weibull • Poisson • Binomial • Negative Binomial • Hypergeometric • Graphs, Functions, and Applications • See page 429 of course text Dr. Joan Burtner, Associate Professor of Industrial Engineering

8. Advanced Statistical Methods for Managers • Basic Hypothesis Testing • One Sample t or Z Tests • Two Sample t or Z Tests • Advanced Hypothesis Testing • Design of Experiments (ANOVAs) • Regression (Simple, Multiple, Non-linear) • Visualization • Response Surface • Evolutionary Operation (EVOP) • Incremental search for more optimal points on a response surface Dr. Joan Burtner, Associate Professor of Industrial Engineering

9. Theory of Variation • Common Cause • Stable and predictable causes of variation • Inherent in all processes • Managers, not workers, are responsible for common cause variation • Special Cause • Unexpected or abnormal causes of variation • May result in sudden or extreme departures from normal • May also result in gradual shifts (trends) Dr. Joan Burtner, Associate Professor of Industrial Engineering

10. Control Chart Types • Control Charts • Variables – based on continuous data • X bar and R (mean and range) • Attributes - based on discrete data • P (proportion) • C (count) • Example of R Chart: Dr. Joan Burtner, Associate Professor of Industrial Engineering

11. Control Chart Calculations for Variables Charts • Xbar and R Control Chart Constants • Control Chart Calculations Dr. Joan Burtner, Associate Professor of Industrial Engineering

12. Process Capability • Analysis conducted on processes that have been shown to be “in-control” • Only common cause variation in range • Only common cause variation in mean • Two standard measures • Cp • Compares variability of process to specifications • Cpk • Compares variability of process to specifications • Is the process sufficiently centered? Dr. Joan Burtner, Associate Professor of Industrial Engineering

13. Process Capability Calculations • Calculations • Evaluation • Cpk > 1.33 Definitely Capable • 1.00 < Cpk < 1.33 Possibly Capable • Cpk  1.00 Not Capable Dr. Joan Burtner, Associate Professor of Industrial Engineering

14. Acceptance Sampling • Definition: Acceptance sampling is the process of sampling a batch of material to evaluate the level of nonconformance relative to a specified quality level. • Incoming product • Product moved from one process to another • Types of samples • Random • Stratified • Sampling Decision Process Figure 15.3 in course text • D number of defective items (not number of defects) • n sample size • C acceptance number Dr. Joan Burtner, Associate Professor of Industrial Engineering

15. Acceptance Sampling Not Recommended • Why not use sampling to collect data? (according to course text pp. 423-424) • Customer requires 100% inspection • Relatively small number of items or services allows for ‘economical’ 100% inspection • The inspection method is built into production so that no defectives can be shipped • Self-inspection by trained operators is sufficient for the nature of the product produced Dr. Joan Burtner, Associate Professor of Industrial Engineering

16. Sampling Plans • Standards • ANSI/ASQC Z1.4 which replaces MIL-STD 105 • ANSI/ASQC Z1.9 for variables • Potential errors (uncertainty risk) • Producer’s Risk “the probability of not accepting a lot, the quality of which has a designated numerical value representing a level that is generally desirable” • Consumer’s Risk “the probability of accepting a lot, the quality of which has a designated numerical value representing a level that is seldom desirable” Dr. Joan Burtner, Associate Professor of Industrial Engineering

17. References • Course Text: • Westcott, R.T., Ed. (2006). Certified Manager of Quality/Organizational Excellence Handbook (3rd ed.). Milwaukee: ASQ Quality Press. • Additional Sources • “Baldrige National Quality Award Criteria” www.quality.nist.gov • Christensen, E.H., Coombes-Betz, K.M., and Stein, M.S. (2006). The Certified Quality Process Analyst Handbook. Milwaukee: ASQ Quality Press. Dr. Joan Burtner, Associate Professor of Industrial Engineering

18. Contact Information • Email: Burtner_J@Mercer.edu • US Mail: Mercer University School of Engineering 1400 Coleman Avenue Macon, GA • Phone: (478) 301- 4127 Dr. Joan Burtner, Associate Professor of Industrial Engineering