1 / 25

Establishing the Integrity of Data: Measurement Systems Analysis

Establishing the Integrity of Data: Measurement Systems Analysis. prepared by Jeffrey T. Luftig, Ph.D. W. Edwards Deming Professor of Management Lockheed Martin Engineering Management Program University of Colorado at Boulder . Topics. Measurement Scales and Types of Data

Leo
Download Presentation

Establishing the Integrity of Data: Measurement Systems Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Establishing the Integrity of Data: Measurement Systems Analysis prepared by Jeffrey T. Luftig, Ph.D. W. Edwards Deming Professor of Management Lockheed Martin Engineering Management Program University of Colorado at Boulder

  2. Topics • Measurement Scales and Types of Data • Establishing the • Reliability and Validity of Instrumentation, or • Precision and Accuracy of Instrumentation

  3. Measurement Scales and Data

  4. Measurement Scales and Data

  5. Measurement as a Process • As in any process, regardless of the nature of data collected or generated, measurement systems must demonstrate • Stability through time, or control • Minimal variation as a proportion of specifications, or capability • Minimal variation as a proportion of process variation

  6. Standard MEASUREMENTPROCESS Product or Process to be Measured Procedure Equipment Measurement Operator Ambient EnvironmentalCharacteristics Measurement as a Process

  7. Definition of Terms • Reference Value • The theoretically or agreed upon correct value of the characteristic being measured, traceable to some standard • Resolution • The smallest increment, or unit of measure, available from a measurement process • Generally at least 1/10th of the specification range

  8. Definition of Terms • Precision • The degree of agreement (or variability) between individual measurements or test results from measuring the same specimen(s) • Accuracy (Bias) • The difference between the average of the measurement error distribution and the reference value of the specimen measured

  9. Precision Accuracy Precision vs. Accuracy

  10. Definition of Terms • Repeatability • The variation in repeated measurements of the same items with a single measurement system • Within appraiser/system variation • Reproducibility • The variation in the average measurements by different appraisers or systems measuring the same items • Between appraiser/system variation

  11. Measurement Error Distribution of repeatedmeasures on a single specimen or part Precision - Repeatability - Reproducibility Accuracy (Bias) Reference Value

  12. Terms • Linearity • The degree to which bias changes with changes in the magnitude of the characteristic measured • Stability • The dependability, or consistency of the measurement process over time

  13. MeasurementSystems Capability • The variability resulting from measurement error must not exceed a significant proportion of the intended specifications said to be capable • In addition, it is not desirable for measurement error to exceed a significant proportion of the total process variability • Capability is not the same as acceptability, acceptability must be determined on a case by case basis

  14. Measurement Error Distribution LSL USL 5.15E (USL - LSL) Measurement Systems Capability

  15. Measurement System Studies • Potential Studies • Assess potential of a measurement system to be capable over the long term • 10 parts measured 2–3 times by one or more appraisers • A “quick and dirty” study to find out if you are in the ballpark • Assesses repeatability and reproducibility • Often called an R&R study

  16. Measurement System Studies • Potential Studies • Error Through Time • Bias Through Time

  17. Measurement System Studies Potential Studies (continued)

  18. Results & Conclusions: Evaluating the Precision & Accuracy of the Measurement System • This result of the previous analysis allows us to calculate the average variance of the repeated measures, which when we take its square root gives us the estimate of the standard deviation due to measurement error: 2 = 66.39  = 8.15

  19. Results & Conclusions: Evaluating the Precision & Accuracy of the Measurement System • Using the estimate of measurement error, we can calculate the Precision-Tolerance ratio, which in the case of short-term studies, should be less than 10%. Assuming the engineering tolerance for this process is 470 (USL) – 450(LSL) = 20: 2 = 66.39  = 8.15 P/T = Precision-Tolerance Ratio = = 6() / USL-LSL = 6(8.15) / 470 – 450 = 2.44 = 244% > 10% Requirement (S-T)

  20. Results & Conclusions: Evaluating the Precision & Accuracy of the Measurement System • Likewise, we can estimate the Accuracy (amount of Bias) in the scale by calculating the average of the differences between the Means of the Repeated Measures and the True Values for the associated specimens:  = -3.09 Estimate Bias at 3.09 Grams; as compared to the Precision estimate, this is arguably an inconsequential value.

  21. Measurement System Studies • Short-term Studies • 25 parts measured 5-8 times by one or more appraisers • A more thorough short-term assessment • Long-term Studies • 8-10 parts measured 25+ times by one or more appraisers • Assesses through time stability

  22. Measurement System Studies • Long-term Studies

  23. Measurement System Studies • Long-term Studies

  24. Measurement Systems Requirements • Summary • Regardless of thetype of data gathered by an instrument, and the assessment methodology employed, the instrument or device utilized to obtain criterion data must meet three requirements before the experiment should proceed: • The instrument must be precise or reliable; • The instrument must be accurate or valid; and • The instrument should be / must be operating in a state of statistical control.

  25. Sources and References • The material used in the PowerPoint presentations associated with this course was drawn from a number of sources. Specifically, much of the content included was adopted or adapted from the following previously-published material: • Luftig, J. A Quality Improvement Strategy for Critical Product and Process Characteristics. Luftig & Associates, Inc. Farmington Hills, MI, 1991 • Spooner-Jordan, V. Understanding Variation. Luftig & Warren International, Southfield, MI 1996 • Luftig, J. and Petrovich, M. Quality with Confidence in Manufacturing. SPSS, Inc. Chicago, IL 1997 • Littlejohn, R., Ouellette, S., & Petrovich, M. Black Belt Business Improvement SpecialistTraining, Luftig & Warren International, 2000 • Ouellette, S. Six Sigma Champion Training, ROI Alliance, LLC & Luftig & Warren, International, Southfield, MI 2005 • Luftig, J. An Overview of Total Quality Management, Luftig & Warren, International, 1992 • Luftig, J. Dr. Deming’s Theory of Profound Knowledge as a Foundation for Strategic Planning and Policy Deployment, Luftig & Warren, International, 1997 • Luftig, J. and Jordan, V. Design of Experiments in Quality Engineering, McGraw-Hill/Irwin Publishing Company, 1998

More Related