1 / 12

Two Way ANOVA

two way anova approach in statistical design

Charlton1
Download Presentation

Two Way ANOVA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Week 7 Lecture: Two-way Analysis of Variance (Chapter 12) We can extend the idea of a one-way ANOVA, which tests the effects of one factor on a response variable, to a two-way ANOVA which tests the effects of two factors and their interaction on a response variable. Thus, each experimental unit is classified by two factors; e.g., treatment group and sex, forest location and soil groups, type of thinning and site quality class, etc. Each factor can have different levels; e.g., two levels for sex: male and female, four levels of thinning prescriptions: low, medium, high, control. The two-way ANOVA tests the null hypotheses of equal means for each factor, as we will see in the upcoming example. Two-way ANOVA with Equal Replication (see Zar’s section 12.1) This is the most common (and simple) type of two-way ANOVA. The two factors in this design are said to be crossed, meaning that each level of one factor is found in combination with the other factor. When there are equal number of replicated samples or experimental units (or simply replicates) in each factorial combination (also called a cell), then we have a balanced or orthogonal experimental design. We can still perform an ANOVA when we have an unbalanced design (i.e., unequal replicates per cell), though we will discuss this later. First, we will discuss the different sources of variation in the two-way ANOVA and present the ANOVA table with these different sources of variation with their corresponding degrees of freedom. Sources of Variation: As with the t-test and the one-way ANOVA, we assume that the variances are homogeneous. In a manner analogous to the one-way ANOVA’s within-group sums of squares, we can calculate a within-cells sums of squares and corresponding degrees of freedom, 1

  2. which under the assumption of constant variance, can be used to obtain a pooled variance common to all cells: ⎡ ⎤ 2 ( ) a b n ∑∑ ∑ = = 1 i 1 j − X X within-cells SS = ⎢ ⎥ ijl ij ⎢ ⎣ ⎥ ⎦ = l 1 within-cells DF = ab(n – 1), where: a = number of levels in factor A b = number of levels in factor B n = number of replicates. The pooled variance( ) , which is the best estimate of σ2, is found by: 2 p s − within cells SS error SS = = MSE . − within cells DF error DF We also need an estimate of the variability among the cells. This is analogous to “groups” in the one-way ANOVA: ( ) a b ∑∑ = 1 i 2 = − , cells SS n X X ij = j 1 cells DF = ab – 1. Finally, we need to know the variability among all the data, N, which is also analogous to that of the one-way ANOVA: ( ) a b n ∑∑∑ = = 1 i j 2 − total SS = , X X ijl = 1 l 1 total DF = N – 1. In a two-way ANOVA, we typically want to know about the differences between the two factors when considered independently. Thus, we want to examine the specific components of the cells 2

  3. SS and cells DF. We can examine these differences by only considering one factor at a time in the ANOVA. So, for factor A: a ( ) ∑ = i 2 − SS(A) = bn X X , . i 1 DF(A) = a – 1, and for factor B, ( ) b ∑ = j 2 − SS(B) = , an X X j . 1 DF(B) = b – 1. However, the sums of squares and degrees of freedom for factor A and factor B will not exactly sum to the cells SS because of interaction between the factors. The relationship is additive and it can be expressed as: SS(AxB) = SS(Cells) – SS(A) – SS(B), DF(A x B) = DF(Cells) – DF(A) – DF(B) = (a – 1)(b – 1). An interaction between the two factors implies that they are not independent of each other. We will test interaction to determine if it is actually significant in an upcoming example. As with the one-way ANOVA, you can divide the different SS by their corresponding DF to obtain a variance, called a mean square: SS ( A ) = = MS ( A ) , DF ( A ) SS ( B ) = = MS ( B ) , DF ( B ) SS ( AxB ) = = MS ( AxB ) , DF ( AxB ) 3

  4. SS ( Error ) = = MS ( Error ) . DF ( Error ) Error MS is usually called the mean square error (MSE). As with the one-way ANOVA, these ratios are F-distributed. So, we can calculate corresponding F-statistics and compare to a one- tailed critical value from the F-distribution for our hypothesis tests (for a Model I ANOVA – see discussion later in the notes): MS ( A ) F = = , for Ho: no effect of factor A on response variable, MSE MS ( B ) F = = , for Ho: no effect of factor B on response variable, MSE MS ( AxB ) F = = , for Ho: no interaction between factor A and factor B. MSE We reject any Ho if F ≥ Fcritical; otherwise, we do not reject Ho. At this point, we can now construct our theoretical ANOVA table: Source of Variation Degrees of Freedom (DF) Sums of Squares (SS) Mean Square (MS) F-statistic ( ) a b ∑∑ = 1 i 2 − n X X Cells ab - 1 ij = j ∑ = i ∑ = j 1 a MS ( A ) SS ( A ) ( ) 2 − F = = bn X X a – 1 Factor A . i MSE MS DF SS ( ( A B ) 1 ( ) ( B ) b ) 2 F = = − an X X Factor B b - 1 j . MSE MS DF ( SS ( B ) ) 1 AxB ( AxB ) cells SS – factor A SS – factor B SS ( ∑∑ ∑ = = = ⎢ ⎣ ( ∑∑∑ = = = 1 i 1 j 1 l F = = A x B (a – 1)(b –1) DF ( AxB ) MSE ⎡ ⎤ 2 ) SS ( Error ) a b n − X X ⎢ ⎥ Error ab(n – 1) ijl ij DF ( Error ) ⎥ ⎦ i 1 j 1 l 1 ) a b n 2 − X X Total N – 1 ijl 4

  5. Two-way ANOVA Example (from Zar, example 12.1 and 12.2) We want to test if plasma calcium concentration (mg/100 ml) of male and female birds is affected by hormone treatment. This is a balanced Model I ANOVA. We will test the three hypotheses: Ho1: There is no effect of hormone treatment on the mean plasma calcium concentration of birds (µno hormone = µhormone). Ha1: There is an effect of hormone treatment on the mean plasma calcium concentration of birds (µno hormone≠ µhormone). Ho2: There is no difference in the mean plasma calcium concentration between male and female birds (µfemale = µmale). Ha2: There is a difference in the mean plasma calcium concentration between male and female birds (µfemale≠ µmale). Ho3: There is no interaction of sex and hormone treatment on the mean plasma calcium concentration of birds. Ha3: There is an interaction of sex and hormone treatment on the mean plasma calcium concentration of birds. α = 0.05. I used the example SAS program, “TwoWayAnova.sas”, with the data presented in Zar to perform the ANOVA. I obtained the following ANOVA table: Source of Variation Cells Sums of Squares (SS) 1461.3255 Degrees of Freedom (DF) 3 Mean Square (MS) 1386.1125 70.3125 F-statistic p-value Treatment Gender 1386.1125 70.3125 4.9005 301.3920 1762.2175 1 1 1 73.585 3.733 0.260 < 0.0001 0.0713 0.6170 Treatment x Gender Error Total 4.9005 16 19 18.8370 5

  6. As you can see from the table, the F-statistic for the null hypotheses of no interaction (Ho3) is not significant (p-value = 0.6170 > 0.05). The F-statistic for the null hypothesis of no difference between sexes (Ho2) is also not significant (p = 0.0713 > 0.05). However, the F-statistic for the null hypothesis of no treatment effect (Ho1) is highly significant ( p < 0.0001 << 0.05). Thus, we conclude that hormone treatment affects the plasma calcium concentration in this sample of birds. If we had more than two levels in the treatment effect (e.g., two treatments and one control = 3 levels), we could use multiple comparison tests to determine where differences exist between treatments. The procedure is the same as that for multiple comparisons in a one-way ANOVA. If both factors are significant and they have more than two levels, we can perform multiple comparison tests for both factors separately. Assumptions of ANOVA The assumptions in the two-way ANOVA are the same as in the t-tests and one-way ANOVA: homogeneity of variance and normally distributed data. You can test these assumptions using the procedures we have already discussed. Remember that ANOVA is very robust and can thus handle all but the most extreme violations of these assumptions. If you are convinced that the assumptions are violated, you can resort to some non-parametric analogs of two-way ANOVA. However, many of these non-parametric procedures are not fully developed or widely available in computer programs. You should also know that there are three types of two-way (and n-way) ANOVA’s: 6

  7. 1.Model I or Fixed-effects ANOVA: In this type of two-way ANOVA, the factor levels are fixed by the experimenter. Thus, the factors are said to be “crossed”, as we previously mentioned. The example we just completed is an example of a Model I ANOVA. Note: This is the same Model I ANOVA we discussed for the one-way ANOVA. Another thing that differs from the one-way ANOVA is the interaction term. In our example, we found that the interaction effect was not significant. Thus, we could then test for individual factor effects. If the interaction had been significant, we could not say anything meaningful about factor effects because the difference between the levels of one factor is not constant at all levels of the other factor (see Zar’s Figure 12.2, page 260). 2.Model II or Random-effects ANOVA: This rare type of two-way ANOVA occurs when the levels of the factor are selected randomly. The F-statistics are calculated differently than the Model I ANOVA (see Zar’s Table 12.3, page 262). Note: This is the same Model II ANOVA we discussed for the one-way ANOVA. 3.Model III or Mixed-effects ANOVA: This type of factorial design has both a fixed and random effect factor. The F-statistics are again calculated differently than either the Model I or Model II ANOVA (see Zar’s Table 12.3, page 262). For example, if factor A is fixed and factor B is random, the F-statistics are calculated as: MS ( A ) F = = Factor A (fixed): , MS ( AxB ) MS ( B ) F = = Factor B (random): , MSE 7

  8. MS ( AxB ) F = = . Interaction: MSE Unbalanced two-way ANOVA (unequal replication) ANOVA is most powerful when replication is equal for the different levels of each factor. However, you can still perform an ANOVA if you have unequal replication. Continuing with our previous example, let’s say we had unequal replication. Then, you would enter the data as before, though some levels of the factors would be unequal. SAS will process these data in the appropriate manner, but you will need to use the “Type III” sums of squares to get the appropriate F-statistics for hypothesis testing. Look at the example output (located in the pdf file) from our balanced design example. Notice in the SAS output that there is “Type I” and “Type III” sums of squares. I don’t want to get into SAS “details”, but you need to know that most simply stated, the Type III SS are the ones to use in most cases. When the design is unbalanced, you will always use the Type III SS. Type I and Type III SS will be the same for balanced designs, so you can use either one in this case. Two-way ANOVA without replication You may use a two-way ANOVA where you only have one observation per cell. The computations are different in this case versus that with replication (see Zar’s Table 12.4, page 268). In this case, you are unable to test for interaction or error, which makes any inference testing of factors tenuous at best. See Zar’s Table 12.5, page 269, for the procedure to perform this type of test. It is advisable to always try to get some replication to avoid this situation. 8

  9. Randomized Block Design The randomized block design is an extension of the paired t-test to k samples. In this experimental design, an experimental unit in one treatment group is related to an experimental unit in another treatment group. Thus, experimental units are not assumed to be independent as in the completely randomized design. In this case, the researcher can group experimental units into blocks and create a more powerful test. The idea is to group experimental units (e.g., cows, fish, corn plots) into the blocks. Then, treatments would be assigned randomly to each experimental unit within a block Notice that there is only one replication per cell, which also occurs in a two-way ANOVA with no replication (thus, we will use the “remainder error” rather than the “error MS” to get our F-statistic (see Zar, page 268)). The data from this experimental design is analyzed as a Model III ANOVA, where blocks are the random effect and treatments are the fixed effects. Typically, the researcher is not interested in testing if block effects are significant, but rather the treatments only. The theoretical ANOVA table for a randomized block design is: Degrees of Freedom (DF) Source of Variation Sums of Squares (SS) Mean Square (MS) F-statistic 2 Treatment SS Treatment MS a ( ) b∑ F = − X X Treatment (i) a - 1 Treatment DF Re mainder MS . i = i 1 2 ( ) b a∑ − X X Blocks (j) b – 1 j . = j 1 Re mainder SS Total SS – Treatment SS – Block SS ( ∑∑ = = 1 i 1 j Remainder (a – 1)(b-1) Re mainder DF ) a b 2 − X X Total ab – 1 = N -1 ij 9

  10. Example: A researcher is interested in the effect of several herbicides on the spike weight (ounces) of gladiolus. Here’s the data: TREATMENT Control A B C Block 1 1.25 2.05 1.95 1.75 Block 2 1.73 1.56 2.00 1.93 Block 3 1.82 1.68 1.83 1.70 Block 4 1.31 1.69 1.81 1.59 We will test the following hypothesis: Ho: The mean spike weight is the same for each of the four herbicide treatments. Ha: not Ho. α = 0.05. After running SAS with these data, we get the ANOVA table: Source of Variation Blocks Treatment Remainder Total >>>Note: I did not include all the output from SAS in this table, notably the block MS. Degrees of Freedom (DF) 3 3 9 15 Mean Square (MS) Sums of Squares (SS) F-statistic 0.0947 0.2777 0.3765 0.7489 2.2153 0.0926 0.0418 Fcritical = F0.05(1), 3, 9 = 3.86 Decision Rule: If F ≥ 3.86, then reject Ho; otherwise do not reject Ho. Conclusion: Since 2.2153 < 3.86, do not reject Ho and conclude that none of the four herbicide treatments affect gladiolus spike weight in this sample (p = 0.1561). 10

  11. Note: You can perform multiple comparisons/contrasts for the fixed effects factor (i.e., treatments) in the same manner as the two-way ANOVA, though it was unnecessary in this case since we did not reject Ho (see Zar, page 274, Section 12.5). Friedman’s Test This is the non-parametric analog of the randomized block ANOVA. It is useful when your data for this design do not meet the assumptions of the parametric ANOVA. Like other non- parametric techniques, the analysis is performed on ranks rather than the original data. To carry out this test: 1.Rank the treatments within each block. 2.Calculate the rank sums, Ri, for each treatment. a 12 ( ) ∑ − i χ = − + 2 r 2 i 3.Find chi-square statistic: R 3 b a 1 . ( ) + ba a 1 1 4.Compare the chi-square statistic to a critical value from Table B.14 in appendix. If the critical value is not in the table, compare to a chi-square value with (a – 1) degrees of freedom, which approximates the critical value (though it tends to be conservative – more likely to commit a Type II error – see Zar, page 277 to 279, for a suggested solution). 11

  12. Example: Let’s continue with the gladiolus example from before. First, rank the data within each block and get the rank sums: TREATMENT Control A B C Block 1 1 4 3 2 Block 2 2 1 4 3 Block 3 3 1 4 2 Block 4 1 3 4 2 Rank Sums 7 9 15 9 )( ) 12 ( ) χ = + + + − ∗ + = 2 r 2 2 2 2 Test statistic: 7 9 15 9 3 4 4 1 4 . 5 . ( ∗ + 4 4 4 1 χ 2 r≥ Decision Rule: If . 7 80 (value from Table B.14), then reject Ho; otherwise do not reject Ho. Conclusion: Since 5.4 < 7.8 (0.1 < P < 0.2), do not reject Ho – same conclusion as before. Note: I also provided an example SAS file (FriedmansTest.sas) on the webpage. Use the second test statistic, “Row Mean Scores Differ”, for hypothesis testing. Also, you can perform non- parametric multiple comparisons/contrasts as for the parametric randomized block ANOVA, though again it was unnecessary since we did not reject Ho (see Zar, page 280). However, SAS does not provide non-parametric multiple comparisons so you will have to calculate them by hand/programming. 12

More Related