Design and Analysis of Experiments Lecture 1.2

1. Design and Analysis of ExperimentsLecture 1.2 • Review of Lecture 1.1 • Application; a simple comparative experiment • Comparing several means • Randomised block design • Randomised block analysis Diploma in Statistics Design and Analysis of Experiments

2. Examinations Timetable Please note that the definitive versions of the timetables are displayed on the College webpages at www.tcd.ie/Examinations/Timetables/PDF/1291000.pdf Diploma in Statistics Design and Analysis of Experiments

3. Experiments To find out what happens when you change something, it is necessary to change it. Experiment and you'll see! (BHH) Diploma in Statistics Design and Analysis of Experiments

4. Recall Example 2: walking babies • How long does it take a baby to walk? • Can this be affected by special training programs? 4 "training" programs: • special exercises • normal daily exercise • weekly check • end of study check each of 24 babies allocated at random to groups of 6 in each program. Diploma in Statistics Design and Analysis of Experiments

5. Characteristics of an experiment Experimental units: entities on which observations are made e.g., babies Experimental Factor: controllable input variable e.g., Training Factor Levels / Treatments: values of the factor e.g., training programmes Response: output variable measured on the units e.g., walking age Diploma in Statistics Design and Analysis of Experiments

6. Exercise 1.2.1 Recall comparison of standard (old) and new processes for manufacture of electronic components, 50 components sampled per day, 6 days per week, for 8 weeks, What were the experimental units factor factor levels response Diploma in Statistics Design and Analysis of Experiments

7. Two design principles • Blocking • identify homogeneous blocks of experimental units • assess effects of experimental change within homogeneous blocks • average effects across blocks • Randomisation • allocate experimental conditions to units at random • minimise chances of "lurking variable" pattern coinciding with factor level allocation pattern Diploma in Statistics Design and Analysis of Experiments

8. Homework 1.1.1 Assess the statistical significance of the difference in defect rates, %, between the first period and second period for the new process. Diploma in Statistics Design and Analysis of Experiments

9. Illustration of a full factorial design,with 12 experimental runs High Pressure Low High Low Temperature Diploma in Statistics Design and Analysis of Experiments

10. Exercise 1.2.2 Recall yield optimisation experiment. What were the experimental units factors factor levels response Diploma in Statistics Design and Analysis of Experiments

11. 2 ApplicationA simple comparitive experiment Wear of shoe soles made of two materials, A and B, worn on opposite feet by each of 10 boys, with randomallocation of materials to feet. Diploma in Statistics Design and Analysis of Experiments

12. Exercise 1.2.3 What were the experimental units factor factor levels response blocks randomisation procedure Diploma in Statistics Design and Analysis of Experiments

13. Analysis Wear of shoe soles made of two materials, A and B, worn on opposite feet by each of 10 boys Diploma in Statistics Design and Analysis of Experiments

14. Analysis t test of differences highly significant Diploma in Statistics Design and Analysis of Experiments

15. Diploma in Statistics Design and Analysis of Experiments

16. A simpler test Sign test: count the "+" signs; what are the chances of getting that number or more? MINITAB Diploma in Statistics Design and Analysis of Experiments

17. Was the blocking effective? Diploma in Statistics Design and Analysis of Experiments

18. Was the blocking effective? Ignore blocking and use a two sample t-test of the effectiveness of the new material. Diploma in Statistics Design and Analysis of Experiments

19. Effect of pairing Paired T for Material B - Material A N Mean StDev SE Mean Material B 10 11.0400 2.5185 0.7964 Material A 10 10.6300 2.4513 0.7752 Difference 10 0.410000 0.387155 0.122429 95% CI for mean difference: (0.133046, 0.686954) T-Test of mean difference = 0 (vs not = 0): T-Value = 3.35 P-Value =0.009 Two-sample T for Material B vs Material A N Mean StDev SE Mean Material B 10 11.04 2.52 0.80 Material A 10 10.63 2.45 0.78 Difference = mu (Material B) - mu (Material A) Estimate for difference: 0.410000 95% CI for difference: (-1.924924, 2.744924) T-Test of difference = 0 (vs not =): T-Value = 0.37 P-Value = 0.716 Diploma in Statistics Design and Analysis of Experiments

20. Effect of pairing Paired t: df = 9, t9,.05 = 2.26 Two-sample t: df = 17, t17,.05 = 2.11 Diploma in Statistics Design and Analysis of Experiments

21. Diagnostic analysis Diploma in Statistics Design and Analysis of Experiments

22. Homework 1.2.1 A maintenance manager tested a new method for repairing machines by recording the time since the previous repair prior to using the new method and the time to next failure after using the new method for each of 10 machines, with the following results: Formally test the effect of changing to the new method. Criticise the design. Was the blocking effective? Diploma in Statistics Design and Analysis of Experiments

23. 3 Comparing several means(Observational study) Burst strength (kPa) of 10 samples of each of four filter membrane types Exercise 1.2.4 Make dotplots of the breaking strengths Diploma in Statistics Design and Analysis of Experiments

24. Comparing several means Burst strength (kPa) of 10 samples of each of four filter membrane types Variable Membrane N Mean StDev Minimum Maximum Range Strength A 10 93 4.8 85 103 19 B 10 96 3.4 91 101 11 C 10 85 4.3 77 93 16 D 10 90 2.8 86 96 9 Diploma in Statistics Design and Analysis of Experiments

25. Comparing several means One-way ANOVA: Strength versus Membrane Source DF SS MS F P Membrane 3 709.2 236.4 15.54 0.000 Error 36 547.8 15.2 Total 39 1257.0 S = 3.901 R-Sq = 56.42% R-Sq(adj) = 52.79% Diploma in Statistics Design and Analysis of Experiments

26. Comparing several means One-way ANOVA: Strength versus Membrane Source DF SS MS F P Membrane 3 709.2 236.4 15.54 0.000 Error 36 547.8 15.2 Total 39 1257.0 S = 3.901 R-Sq = 56.42% R-Sq(adj) = 52.79% Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev ------+---------+---------+---------+--- A 10 92.840 4.831 (----*----) B 10 96.080 3.391 (----*----) C 10 84.630 4.287 (----*----) D 10 89.890 2.764 (----*----) ------+---------+---------+---------+--- 85.0 90.0 95.0 100.0 Diploma in Statistics Design and Analysis of Experiments

27. Comparing several means Tukey 95% Simultaneous Confidence Intervals All Pairwise Comparisons among Levels of Membrane Membrane = A subtracted from: Membrane Lower Center Upper ------+---------+---------+---------+- B -1.46 3.24 7.94 (---*----) C -12.91 -8.21 -3.51 (----*---) D -7.65 -2.95 1.75 (----*----) ------+---------+---------+---------+- -10 0 10 20 Membrane = B subtracted from: Membrane Lower Center Upper ------+---------+---------+---------+--- C -16.15 -11.45 -6.75 (----*---) D -10.89 -6.19 -1.49 (----*----) ------+---------+---------+---------+--- -10 0 10 20 Membrane = C subtracted from: Membrane Lower Center Upper ------+---------+---------+---------+--- D 0.560 5.260 9.960 (---*----) ------+---------+---------+---------+--- -10 0 10 20 Diploma in Statistics Design and Analysis of Experiments

28. Diagnostic analysis Diploma in Statistics Design and Analysis of Experiments

29. 4 Randomised block design Example 1: treating crops with fertiliser to improve yield. Four fertilisers being tested: divide a single field into four plots (treatment units) to form one block, assign treatments at random to the four plots, repeat with several other fields to form several blocks, choose blocks in varying locations, for generalising. Diploma in Statistics Design and Analysis of Experiments

30. Randomised block design Example 2: treating large sheets of rubber to improve abrasion resistance. Four treatments being tested: cut a single piece into four treatment units to form one block, assign treatments at random to the four units, repeat with several other pieces to form several blocks. Diploma in Statistics Design and Analysis of Experiments

31. Randomised block design Block 1 Diploma in Statistics Design and Analysis of Experiments

32. Randomised block design Block 2 Diploma in Statistics Design and Analysis of Experiments

33. Randomised block design Block 3 Diploma in Statistics Design and Analysis of Experiments

34. Randomised block design Block 1 Block 2 Block 3 Diploma in Statistics Design and Analysis of Experiments

35. Randomised block design Example 3: assessing new laboratory test methods. Four methods being assessed: assess each method in a single laboratory, randomise the time order in which the methods are run, repeat in several other laboratories to form several blocks. Diploma in Statistics Design and Analysis of Experiments

36. Randomised block design Example 4: assessing process changes. Five versions of the process being assessed: assess each version on five successive days, randomise the time order in which the versions are used, repeat over several weeks to form several blocks. Diploma in Statistics Design and Analysis of Experiments

37. 5 Randomised block designIllustration Manufacture of an organic chemical using a filtration process • Three step process: • input chemical blended from different stocks • chemical reaction results in an intermediate liquid product • liquid filtered to recover end product. Diploma in Statistics Design and Analysis of Experiments

38. Randomised block designIllustration • Problem: yield loss at filtration stage • Proposal: adjust initial blend to reduce yield loss • Plan: • prepare five different blends • use each blend in successive process runs, in random order • repeat at later times (blocks) Diploma in Statistics Design and Analysis of Experiments

39. Results Diploma in Statistics Design and Analysis of Experiments

40. General Linear Model ANOVA General Linear Model: Loss, per cent versus Blend, Block Analysis of Variance for Loss,%, using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P Blend 4 11.5560 11.5560 2.8890 3.31 0.071 Block 2 1.6480 1.6480 0.8240 0.94 0.429 Error 8 6.9920 6.9920 0.8740 Total 14 20.1960 S = 0.934880 R-Sq = 65.38% R-Sq(adj) = 39.41% Unusual Observations for Loss, per cent Loss, per Obs cent Fit SE Fit Residual St Resid 12 17.1000 18.5267 0.6386 -1.4267 -2.09 R Diploma in Statistics Design and Analysis of Experiments

41. Diploma in Statistics Design and Analysis of Experiments

42. Conclusions (prelim.) F(Blends) is almost statistically significant, p = 0.07 F(Blocks) is not statistically significant, p = 0.4 Prediction standard deviation: S = 0.93 Diploma in Statistics Design and Analysis of Experiments

43. Deleted diagnostics Diploma in Statistics Design and Analysis of Experiments

44. Iterated analysis:delete Case 12 General Linear Model: Loss versus Blend, Block Analysis of Variance for Loss Source DF Seq SS Adj SS Adj MS F P Blend 4 13.0552 14.5723 3.6431 8.03 0.009 Block 2 3.7577 3.7577 1.8788 4.14 0.065 Error 7 3.1757 3.1757 0.4537 Total 13 19.9886 S = 0.673548 Diploma in Statistics Design and Analysis of Experiments

45. Deleted diagnostics Diploma in Statistics Design and Analysis of Experiments

46. Conclusions (prelim.) F(Blends) is almost statistically significant, p = 0.01 F(Blocks) is not statistically significant, p = 0.65 Prediction standard deviation: S = 0.67 Diploma in Statistics Design and Analysis of Experiments

47. Explaining ANOVA ANOVA depends on a decompostion of "Total variation" into components: Total Variation = Blend effect + Block effect + chance variation; Diploma in Statistics Design and Analysis of Experiments

48. Decomposition of results Diploma in Statistics Design and Analysis of Experiments

49. Decomposition of results Diploma in Statistics Design and Analysis of Experiments

50. Decomposition of results Diploma in Statistics Design and Analysis of Experiments