1 / 50

Design and Analysis of Experiments Lecture 4.2

Design and Analysis of Experiments Lecture 4.2. Part 1: Components of Variation identifying sources of variation hierarchical design for variance component estimation hierarchical ANOVA Part 2: Measurement System Analysis Accuracy and Precision Repeatability and Reproducibility

gafna
Download Presentation

Design and Analysis of Experiments Lecture 4.2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design and Analysis of ExperimentsLecture 4.2 Part 1: Components of Variation • identifying sources of variation • hierarchical design for variance component estimation • hierarchical ANOVA Part 2: Measurement System Analysis • Accuracy and Precision • Repeatability and Reproducibility • Components of measurement variation • Analysis of Variance • Case study: the MicroMeter Diploma in Statistics Design and Analysis of Experiments

  2. An invalid comparison Comparing standard process, A, with modified process, B A B 58.3 63.2 57.1 64.1 59.7 62.4 59.0 62.7 58.6 63.6 Means: 58.54 63.20 St. Devs: 0.96 0.68 Diploma in Statistics Design and Analysis of Experiments

  3. An invalid comparison Process: batch manufacture of pigment paste Key variable: moisture content Sampling plan: single sample from single batch Measurements: 5 repetitions s measures measurement error no measure of variation within batch no measure of variation between batches Diploma in Statistics Design and Analysis of Experiments

  4. Sources of variation in moisture content • Batch, subject to Process variation • Sample from batch, subject to within batch variation • Measurement, subject to Test variation Model for variation in moisture content: Y = m + eP + eS + eT Diploma in Statistics Design and Analysis of Experiments

  5. Sources of variation in moisture content Process variation sP • eP m Diploma in Statistics Design and Analysis of Experiments

  6. Sources of variation in moisture content Process variation sP • eP Sampling variation sS • eS m Diploma in Statistics Design and Analysis of Experiments

  7. Sources of variation in moisture content Process variation sP • eP Sampling variation sS • eS Testing variation sT • eT e = eP + eP + eP m y Diploma in Statistics Design and Analysis of Experiments

  8. Components of Variance Recall basic model: Y = m + eP + eS + eT Components of variance: Diploma in Statistics Design and Analysis of Experiments

  9. Conclusions for process testing • Process measurements are subject to a hierarchy of variation sources. • Several measurements on a single sample from a single batch do not reflect overall variation. • Several batches and several samples from each batch are necessary to capture overall variation. • Comparison of process methods must be referred to the relevant level of variation Diploma in Statistics Design and Analysis of Experiments

  10. Hierarchical Design forVariance Component Estimation • A batch of pigment paste consists of 80 drums of material. • 15 batches were available for testing • 2 drums were selected at random from each batch and a sample was taken from each drum. • 2 tests for moisture content were run on each sample. • The results follow Diploma in Statistics Design and Analysis of Experiments

  11. Hierarchical Design forVariance Component Estimation Diploma in Statistics Design and Analysis of Experiments

  12. Nested ANOVA: Test versus Batch, Sample Analysis of Variance for Test Source DF SS MS F P Batch 14 1216.2333 86.8738 1.495 0.224 Sample 15 871.5000 58.1000 64.556 0.000 Error 30 27.0000 0.9000 Total 59 2114.7333 Variance Components % of Source Var Comp. Total StDev Batch 7.193 19.60 2.682 Sample 28.600 77.94 5.348 Error 0.900 2.45 0.949 Total 36.693 6.058 Diploma in Statistics Design and Analysis of Experiments

  13. Nested ANOVA: Test versus Batch, Sample Expected Mean Squares 1 Batch 1.00(3) + 2.00(2) + 4.00(1) 2 Sample 1.00(3) + 2.00(2) 3 Error 1.00(3) Translation: EMS(Batch) = EMS(Sample) = EMS(Error) = Diploma in Statistics Design and Analysis of Experiments

  14. Calculation = EMS(Error) = ½[EMS(Sample) – EMS(Error)] = ¼[EMS(Batch) – EMS(Sample)] Diploma in Statistics Design and Analysis of Experiments

  15. Theory Model: Yijk = m + ai + bi(j) + eijk Yij. = m + ai + bi(j) + eij. Yi.. = m + ai + bi(.) + ei.. Decomposition: (Yijk – Y... ) = (Yi..– Y... ) + (Yij.– Yi.. ) + (Yijk – Yij. ) (Yij.– Yi.. ) = (bi(j)– bi(.) ) + (eij.– ei.. ) EMS involves and Diploma in Statistics Design and Analysis of Experiments

  16. Conclusions fromVariance Components Analysis Variance Components % of Source Var Comp. Total StDev Batch 7.193 19.60 2.682 Sample 28.600 77.94 5.348 Error 0.900 2.45 0.949 Total 36.693 6.058 Sampling variation dominates, testing variation is relatively small. Investigate sampling procedure. Diploma in Statistics Design and Analysis of Experiments

  17. Sampling procedure Standard: • select 5 drums from batch at random, • sample all levels of each drum using a specially constructed sampling tube • thoroughly mix all samples • take a sample from the mixture for analysis Actual • take a sample from a drum for analysis Diploma in Statistics Design and Analysis of Experiments

  18. Another Example Testing drug treatments for pregnant women 22 women, 10 treatment A, 7 treatment B, 5 "control". Placentas examined for "irregularities": 5 locations, 10 slices, on microscope slides, 5 measurements (counts of "irregularities") per slide, 5,500 measurements in all. No significant treatment effect (10 vs 7) Diploma in Statistics Design and Analysis of Experiments

  19. Yet Another Example Comparing schools on student performance Schools Classes within schools Students within classes Diploma in Statistics Design and Analysis of Experiments

  20. Design and Analysis of ExperimentsLecture 4.2 Part 2: Measurement System Analysis • Accuracy and Precision • Repeatability and Reproducibility • Components of measurement variation • Analysis of Variance • Case study: the MicroMeter Diploma in Statistics Design and Analysis of Experiments

  21. The MicroMeter optical comparator Diploma in Statistics Design and Analysis of Experiments

  22. The MicroMeter optical comparator • Place object on stage of travel table • Align cross-hair with one edge • Move and re-align cross-hair with other edge • Read the change in alignment • Sources of variation: • instrument error • operator error • parts (manufacturing process) variation Diploma in Statistics Design and Analysis of Experiments

  23. Characterising measurement variation;Accuracy and Precision Precise Biased Accurate Imprecise Diploma in Statistics Design and Analysis of Experiments

  24. Characterising measurement variation;Accuracy and PrecisionCentre and Spread • Accurate means centre of spread is on target; • Precise means extent of spread is small; • Averaging repeated measurements improves precision, SE = s/√n • but not accuracy; seek assignable cause. Diploma in Statistics Design and Analysis of Experiments

  25. Accuracy and Precision: Example Each of four technicians made six measurements of a standard (the 'true' measurement was 20.1), resulting in the following data: Technician Data 1 20.2 19.9 20.1 20.4 20.2 20.4 2 19.9 20.2 19.5 20.4 20.6 19.4 3 20.6 20.5 20.7 20.6 20.8 21.0 4 20.1 19.9 20.2 19.9 21.1 20.0 Exercise: Make dotplots of the data. Assess the technicians for accuracy and precision Diploma in Statistics Design and Analysis of Experiments

  26. Accuracy and Precision: Example Diploma in Statistics Design and Analysis of Experiments

  27. Repeatability and Reproducability Factors affecting measurement accuracy and precision may include: • instrument • material • operator • environment • laboratory • parts (manufacturing) Diploma in Statistics Design and Analysis of Experiments

  28. Repeatability: precision achievable under constant conditions: same instrument same material same operator same environment same laboratory Reproducibility: precision achievable under varying conditions: different instruments different material different operators changing environment different laboratories Repeatability and Reproducibility Diploma in Statistics Design and Analysis of Experiments

  29. Measurement Capability of the MicroMeter 4 operators measured each of 8 parts twice, with random ordering of parts, separately for each operator. Three sources of variation: • instrument error • operator variation • parts(manufacturing process) variation. Data follow Diploma in Statistics Design and Analysis of Experiments

  30. Measurement Capability of the MicroMeter Diploma in Statistics Design and Analysis of Experiments

  31. Quantifying the variation Each measurement incorporates components of variation from • Operator error • Parts variation • Instrument error and also • Operator by Parts Interaction Diploma in Statistics Design and Analysis of Experiments

  32. Measurement Differences Diploma in Statistics Design and Analysis of Experiments

  33. Graphical Analysis of Measurement Differences Diploma in Statistics Design and Analysis of Experiments

  34. Average measurementsby Operators and Parts Diploma in Statistics Design and Analysis of Experiments

  35. Graphical Analysis of Operators & Parts Diploma in Statistics Design and Analysis of Experiments

  36. Graphical Analysis of Operators & Ordered Parts Diploma in Statistics Design and Analysis of Experiments

  37. Quantifying the variation Notation: sE: SD of instrument error variation sP: SD of parts (manufacturing process) variation sO: SD of operator variation sOP: SD of operator by parts interaction variation sT: SD of total measurement variation N.B.: so Diploma in Statistics Design and Analysis of Experiments

  38. Calculating sE sum = 18.6 sum = 7.0 s2 = (18.6 + 7.0)/32 = 0.8 sE = 0.89 Diploma in Statistics Design and Analysis of Experiments

  39. Analysis of Variance Analysis of Variance for Diameter Source DF SS MS F P Operator 3 32.403 10.801 6.34 0.003 Part 7 1193.189 170.456 100.02 0.000 Operator*Part 21 35.787 1.704 2.13 0.026 Error 32 25.600 0.800 Total 63 1286.979 S = 0.894427 Diploma in Statistics Design and Analysis of Experiments

  40. Basis for Random Effects ANOVA F-ratios in ANOVA are ratios of Mean Squares Check: F(O) = MS(O) / MS(O*P) F(P) = MS(P) / MS(O*P) F(OP) = MS(OP) / MS(E) Why? MS(O) estimates sE2 + 2sOP2 + 16sO2 MS(P) estimates sE2 + 2sOP2 + 8sP2 MS(OP) estimates sE2 + 2sOP2 MS(E) estimates sE2 Diploma in Statistics Design and Analysis of Experiments

  41. Variance Components Estimated Standard Source Value Deviation Operator 0.5686 0.75 Part 21.0939 4.59 Operator*Part 0.4521 0.67 Error 0.8000 0.89 Diploma in Statistics Design and Analysis of Experiments

  42. Diagnostic Analysis Diploma in Statistics Design and Analysis of Experiments

  43. Diagnostic Analysis Diploma in Statistics Design and Analysis of Experiments

  44. Measurement system capability sE sP means measurement system cannot distinguish between different parts. Need sE<< sP . Define sTP = sqrt(sE2 + sP2). Capability ratio = sTP / sE should exceed 5 Diploma in Statistics Design and Analysis of Experiments

  45. Repeatability and Reproducibility Repeatabilty SD = sE Reproducibility SD = sqrt(sO2 + sOP2) Total R&R = sqrt(sO2 + sOP2 + sE2) Diploma in Statistics Design and Analysis of Experiments

  46. Laboratory 1, Part 2A four factor process improvement study Low (–)High (+) A: catalyst concentration (%), 5 7, B: concentration of NaOH (%), 40 45, C: agitation speed (rpm), 10 20, D: temperature (°F), 150 180. The current levels are 5%, 40%, 10rpm and 180°F, respectively. Diploma in Statistics Design and Analysis of Experiments

  47. Design and Results Diploma in Statistics Design and Analysis of Experiments

  48. Pros and Consof omitting "insignificant" terms pro: • the model is simplified • the error term has more degrees of freedom so that s is more precisely estimated • in small samples, comparisons are more precisely made Diploma in Statistics Design and Analysis of Experiments

  49. Pros and Consof omitting "insignificant" terms con: • statistical insignificance does not imply substantive insignificance, so that • when the excluded term has some effect below statistically significant level, the residual standard deviation is likely to increase, giving less precise comparisons, • (although this may be a pro if conservative conclusions are valued) • predictions may be slightly biased. Diploma in Statistics Design and Analysis of Experiments

  50. Reading EM §1.5.3, §7.5, §8.2.1 MS Introduction to Measurement Systems Analysis BHH, §9.3 Diploma in Statistics Design and Analysis of Experiments

More Related