Lecture 13

1. Lecture 13 • Multiple comparisons for one-way ANOVA (Chapter 15.7) • Analysis of Variance Experimental Designs (Chapter 15.3)

2. 15.7 Multiple Comparisons • When the null hypothesis is rejected, it may be desirable to find which mean(s) is (are) different, and how they rank. • Three statistical inference procedures, geared at doing this, are presented: • Fisher’s least significant difference (LSD) method • Bonferroni adjustment to Fisher’s LSD • Tukey’s multiple comparison method

3. Example 15.1 • Sample means: Does the quality strategy have a higher mean sales than the other two strategies? Do the quality and price strategies have a higher mean than the convenience strategy? Does the price strategy have a smaller mean sales than quality but a higher mean than convenience? • Pairwise comparison: Are two population means different?

4. Fisher Least Significant Different (LSD) Method • This method builds on the equal variances t-test of the difference between two means. • The test statistic is improved by using MSE rather than sp2. • We conclude that mi and mj differ (at a% significance level if > LSD, where

5. Multiple Comparisons Problem • A hypothetical study of the effect of birth control pills is done. • Two groups of women (one taking birth controls, the other not) are followed and 20 variables are recorded for each subject such as blood pressure, psychological and medical problems. • After the study, two-sample t-tests are performed for each variable and it is found that one null hypothesis is rejected. Women taking birth pills have higher incidences of depression at the 5% significance level (the p-value equals .02). • Does this provide strong evidence that women taking birth control pills are more likely to be depressed?

6. Experimentwise Type I error rate (aE) versus Comparisonwise Type I error rate • The comparisonwise Type I error rate is the probability of committing a Type I error for one pairwise comparison. • The experimentwise Type I error rate ( ) is the probability of committing at least one Type I error when C tests are done and all null hypotheses are true. • For a one-way ANOVA, there are k(k-1)/2 pairwise comparisons (k=number of populations) • If the comparisons are not planned in advance and chosen after looking at the data, the experimentwise Type I error rate is the more appropriate one to look at.

7. Experimentwise Error Rate • The expected number of Type I errors if C tests are done at significance level each is • If C independent tests are done, aE = 1-(1 – a)C • The Bonferroni adjustment determines the required Type I error probability per test (a) ,to secure a pre-determined overall aE.

8. Bonferroni Adjustment • Suppose we carry out C tests at significance level • If the null hypothesis for each test is true, the probability that we will falsely reject at least one hypothesis is at most • Thus, if we carry out C tests at significance level , the experimentwise Type I error rate is at most

9. Bonferroni Adjustment for ANOVA • The procedure: • Compute the number of pairwise comparisons (C)[all: C=k(k-1)/2], where k is the number of populations. • Set a = aE/C, where aE is the true probability of making at least one Type I error (called experimentwise Type I error). • We conclude that mi and mj differ at a/C% significance level (experimentwise error rate at most ) if

10. Fisher and Bonferroni Methods • Example 15.1 - continued • Rank the effectiveness of the marketing strategies(based on mean weekly sales). • Use the Fisher’s method, and the Bonferroni adjustment method • Solution (the Fisher’s method) • The sample mean sales were 577.55, 653.0, 608.65. • Then,

11. Fisher and Bonferroni Methods • Solution (the Bonferroni adjustment) • We calculate C=k(k-1)/2 to be 3(2)/2 = 3. • We set a = .05/3 = .0167, thus t.0167/2, 60-3 = 2.467 (Excel). Again, the significant difference is between m1 and m2.

12. Tukey Multiple Comparisons • The test procedure: • Assumes equal number of obs. per populations. • Find a critical number w as follows: k = the number of populations n =degrees of freedom = n - k ng = number of observations per population a = significance level qa(k,n) = a critical value obtained from the studentized range table (app. B17/18)

13. If the sample sizes are not extremely different, we can use the above procedure with ng calculated as theharmonic mean of the sample sizes. Tukey Multiple Comparisons • Select a pair of means. Calculate the difference between the larger and the smaller mean. • If there is sufficient evidence to conclude that mmax > mmin . • Repeat this procedure for each pair of samples. Rank the means if possible.

14. Tukey Multiple Comparisons • Example 15.1 - continued We had three populations (three marketing strategies).K = 3, Sample sizes were equal. n1 = n2 = n3 = 20,n = n-k = 60-3 = 57,MSE = 8894. Take q.05(3,60) from the table: 3.40. Population Sales - City 1 Sales - City 2 Sales - City 3 Mean 577.55 653 698.65 City 1 vs. City 2: 653 - 577.55 = 75.45 City 1 vs. City 3: 608.65 - 577.55 = 31.1 City 2 vs. City 3: 653 - 608.65 = 44.35

15. 15.3 Analysis of Variance Experimental Designs • Several elements may distinguish between one experimental design and another: • The number of factors (1-way, 2-way, 3-way,… ANOVA). • The number of factor levels. • Independent samples vs. randomized blocks • Fixed vs. random effects These concepts will be explained in this lecture.

16. Number of factors, levels • Example: 15.1, modified • Methods of marketing: price, convenience, quality => first factor with 3 levels • Medium: advertise on TV vs. in newspapers => second factor with 2 levels • This is a factorial experiment with two “crossed factors” if all 6 possibilities are sampled or experimented with. • It will be analyzed with a “2-way ANOVA”. (The book got this term wrong.)

17. One - way ANOVA Single factor Two - way ANOVA Two factors Response Response Treatment 3 (level 1) Treatment 2 (level 2) Treatment 1 (level 3) Level 3 Level2 Factor A Level 1 Level2 Level 1 Factor B

18. Randomized blocks • This is something between 1-way and 2-way ANOVA: a generalization of matched pairs when there are more than 2 levels. • Groups of matched observations are collected in blocks, in order to remove the effects of unwanted variability. => We improve the chances of detecting the variability of interest. • Blocks are like a second factor => 2-way ANOVA is used for analysis • Ideally, assignment to levels within blocks is randomized, to permit causal inference.

19. Randomized blocks (cont.) • Example: expand 13.03 • Starting salaries of marketing and finance MBAs: add accounting MBAs to the investigation. • If 3 independent samples of each specialty are collected (samples possibly of different sizes), we have a 1-way ANOVA situation with 3 levels. • If GPA brackets are formed, and if one samples 3 MBAs per bracket, one from each specialty, then one has a blocked design. (Note: the 3 samples will be of equal size due to blocking.) • Randomization is not possible here: one can’t assign each student to a specialty, and one doesn’t know the GPA beforehand for matching. => No causal inference.

20. Models offixed and random effects • Fixed effects • If all possible levels of a factor are included in our analysis or the levels are chosen in a nonrandom way, we have a fixed effect ANOVA. • The conclusion of a fixed effect ANOVA applies only to the levels studied. • Random effects • If the levels included in our analysis represent a random sample of all the possible levels, we have a random-effect ANOVA. • The conclusion of the random-effect ANOVA applies to all the levels (not only those studied).

21. Models offixed and random effects (cont.) Fixed and random effects - examples • Fixed effects - The advertisement Example (15.1): All the levels of the marketing strategies considered were included. Inferences don’t apply to other possible strategies such as emphasizing nutritional value. • Random effects - To determine if there is a difference in the production rate of 50 machines in a large factory, four machines are randomly selected and the number of units each produces per day for 10 days is recorded.

22. 15.4 Randomized Blocks Analysis of Variance • The purpose of designing a randomized block experiment is to reduce the within-treatments variation, thus increasing the relative amount of between treatment variation. • This helps in detecting differences between the treatment means more easily.

23. Examples of Randomized Block Designs

24. Randomized Blocks Block all the observations with some commonality across treatments Treatment 4 Treatment 3 Treatment 2 Treatment 1 Block3 Block2 Block 1

25. Randomized Blocks Block all the observations with some commonality across treatments

26. Sum of square for treatments Sum of square for blocks Sum of square for error Partitioning the total variability • The sum of square total is partitioned into three sources of variation • Treatments • Blocks • Within samples (Error) Recall. For the independent samples design we have: SS(Total) = SST + SSE SS(Total) = SST + SSB + SSE

27. Sums of Squares Decomposition • = observation in ith block, jth treatment • = mean of ith block • = mean of jth treatment

28. SSB= SST = Calculating the sums of squares • Formulas for the calculation of the sums of squares

29. SSB= SST = Calculating the sums of squares • Formulas for the calculation of the sums of squares

30. Mean Squares • To perform hypothesis tests for treatments and blocks we need • Mean square for treatments • Mean square for blocks • Mean square for error

31. Test statistic for treatments Test statistic for blocks Test statistics for the randomized block design ANOVA df-T: k-1 df-B: b-1 df-E: n-k-b+1

32. The F test rejection regions • Testing the mean responses for treatments F > Fa,k-1,n-k-b+1 • Testing the mean response for blocks F> Fa,b-1,n-k-b+1

33. Randomized Blocks ANOVA - Example • Example 15.2 • Are there differences in the effectiveness of cholesterol reduction drugs? • To answer this question the following experiment was organized: • 25 groups of men with high cholesterol were matched by age and weight. Each group consisted of 4 men. • Each person in a group received a different drug. • The cholesterol level reduction in two months was recorded. • Can we infer from the data in Xm15-02that there are differences in mean cholesterol reduction among the four drugs?

34. Randomized Blocks ANOVA - Example • Solution • Each drug can be considered a treatment. • Each 4 records (per group) can be blocked, because they are matched by age and weight. • This procedure eliminates the variability in cholesterol reductionrelated to different combinations of age and weight. • This helps detect differences in the mean cholesterol reduction attributed to the different drugs.

35. Conclusion: At 5% significance level there is sufficient evidence to infer that the mean “cholesterol reduction” gained by at least two drugs are different. Randomized Blocks ANOVA - Example Treatments Blocks b-1 K-1 MST / MSE MSB / MSE