1 / 20

Class Handout #9 (Chapter 6)

Class Handout #9 (Chapter 6). Definitions. Two-Way Analysis of Variance (ANOVA).

austin
Download Presentation

Class Handout #9 (Chapter 6)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Class Handout #9 (Chapter 6) Definitions Two-Way Analysis of Variance (ANOVA) Recall that determining whether a relationship exists between a qualitative-nominal variable and a quantitative variable is essentially equivalent to determining whether there is a difference in the distribution of the quantitative variable for each of the categories of the qualitative-nominal variable. Also, recall that when looking for a difference in the distribution of a quantitative variable between categories of a qualitative-nominal variable, it is common to focus on the mean of the distribution. We can think of the quantitative variable as the dependent variable and the qualitative-nominal variable as the independent variable, that is, we can think of predicting the (mean of the) quantitative variable from one qualitative-nominal variable; this is when a one-way ANOVA might be used. When predicting from two or three qualitative-nominal variables, a two-way ANOVA or three-way ANOVA respectively could be used. We shall now consider the two-way ANOVA Recall that in a one-way ANOVA, we let 1 , 2 , … , k represent the means of the quantitative variable for the k categories of the qualitative-nominal variable. If in a two-way ANOVA, one qualitative-nominal variable has r categories, and the other qualitative-nominal variable has c categories, we let 11 , … , rc represent the means of the quantitative variable for the various possible combinations of categories of the two qualitative-nominal variables; if we label the two qualitative-nominal variables as the row variable and the column variable, these means can be displayed as follows:

  2. Category #1 for Column Variable Category #2 for Column Variable Category #c for Column Variable Category #1 for Row Variable 11 12 1c 1* Category #2 for Row Variable 21 22 2c 2* Category #r for Row Variable r1 r2 rc r* *1 *2 *c ** The data for a two-way ANOVA consists of random observations of the quantitative variable for each possible combination of the categories of the two qualitative-nominal variables. Each of these possible combinations is called a cell of the table, and the corresponding means are called cell means. The double subscript indicates a specific combination of the categories, where the first subscript indicates the row variable category and the second subscript indicates the column variable category.

  3. The means displayed outside the table at the end of each row represent row means, each one of which is the mean for the corresponding category of the row variable; the first subscript indicates the row variable category and the second subscript is an asterisk. The means displayed outside the table at the end of each column represent column means, each one of which is the mean for the corresponding category of the column variable; the first subscript is an asterisk and the second subscript indicates the column variable category. The mean displayed in the lower right corner outside the table represents the grand (overall) mean, which is the mean when all categories for both the row and column variable are combined together; both subscripts are asterisks. The following three questions are of interest in a two-way ANOVA: (1) (2) (3) Is there any statistically significant difference in the mean of the quantitative variable between categories of the row variable? Is there any statistically significant difference in the mean of the quantitative variable between categories of the column variable? Is there any statistically significant difference in the mean of the quantitative variable as a result of interaction between the row variable and column variable?

  4. The first two questions can be addressed with two one-way ANOVAs, one for the row variable and one for the column variable. However, the third question, which concerns interaction, can only be addressed with a two-way ANOVA. An interaction is said to exist among the means for a row variable and a column variable when the difference between a pair of row means is not the same for each column category, and equivalently, the difference between a pair of column means is not the same for each row category. The differences in means between categories of the row variable and between categories of the column variable are called main effects; the differences in means resulting from interaction are interaction effects. Recall that a regression where a quantitative dependent (response) variable is predicted from dummy variables which represent an independent qualitative variable models a one-way ANOVA. A two-way ANOVA can be modeled by a regression where a quantitative dependent (response) variable is predicted from dummy variables which represent two independent qualitative variables.

  5. To illustrate the different types of differences among cell means that are possible, suppose the mean height (inches) of a plant is compared for two different types of soil (the row variable) labeled A and B, and for two different climates (the column variable) labeled Cool and Warm. The following tables of (population) means illustrate various combinations of effects which might exist: A Row Main Effect, No Column Main Effect, and No Interaction A Column Main Effect, No Row Main Effect, and No Interaction No Main Effects and No Interaction Cool Warm Cool Warm Cool Warm A B A B A B 20 20 20 20 20 30 20 20 25 25 20 30 Both Row and Column Main Effects and No Interaction Both Row and Column Main Effects and Interaction No Main Effects and Interaction Cool Warm Cool Warm Cool Warm A B A B A B 20 30 20 30 20 30 25 35 25 45 30 20

  6. 1. Suppose the height (inches) of a plant is a dependent variable to be predicted from type of soil and type of climate. There are two types of soil (the row variable) labeled A and B, and two types of climates (the column variable) labeled Cool and Warm. Define dummy variables to represent the qualitative independent variables. (a) A qualitative variable with k categories can be represented in a regression model with k 1 dummy variables. The variable soil type can be represented by the following dummy variable: 1 for soil type A R1= 0 for soil type B The variable climate type can be represented by the following dummy variable: 1 for Cool C1= 0 for Warm

  7. Write a description of the least squares regression equation with only the dummy variables defined in part (a) as independent variables, and explain why this regression equation does not allow for interaction. (b) height = a + b1R1 + b2C1 The predicted heights (estimated means) for each soil type and climate type combination are as follows: Cool Warm A B a + b1+ b2 a + b1 a + b2 a This regression equation does not allow for interaction, because the difference between soil type means is the same for each climate (i.e., this difference is b1), and equivalently, the difference between climate type means is the same for each soil type (i.e., this difference is b2).

  8. 1.-continued Write a description of a least squares regression equation which does allow for interaction, and demonstrate that it indeed does. (c) height = a + b1R1 + b2C1 + b3R1C1 The predicted heights (estimated means) for each soil type and climate type combination are as follows: Cool Warm A B a + b1+ b2+ b3 a + b1 a + b2 a This regression equation does allow for interaction, because the difference between soil type means is b1 + b3 for the Cool climate and b1 for the Warm climate. Also, the difference between climate type means is b2+ b3 for soil type A and b2for soil type B.

  9. 2. A 0.05 significance level is chosen for a two-way ANOVA to study the height to which wheat grows for two types of wheat, labeled D and E, and four different types of soil, labeled C, G, H, and T. A random sample of heights is recorded in inches for each possible combination of wheat type and soil type with the following results: Soil Type C Soil Type G Soil Type H Soil Type T Wheat Type D 37.4 35.1 41.8 44.1 46.4 40.0 42.2 38.8 47.4 33.0 31.1 27.1 Wheat Type E 31.8 28.5 36.6 35.6 42.6 38.5 27.9 22.9 26.0 30.8 36.9 34.3 The data has been stored in the SPSS data file wheat_height. Identify the dependent (response) variable and the (explanatory) independent variables. (a) The dependent (response) variable is wheat height, and the independent (explanatory) variables are wheat type and soil type.

  10. 2.-continued Explain why only one define dummy variable is sufficient to represent the row independent variable, and only three dummy variables are sufficient to represent the column independent variable. Then define dummy variables sufficient to represent each independent variable. (b) A qualitative variable with k categories can be represented in a regression model with k 1 dummy variables. The variable wheat type can be represented by the following dummy variable: 1 for wheat type D R1= 0 for wheat type E The variable soil type can be represented by the following three dummy variables: 1 for soil type C C1= 0 otherwise 1 for soil type G C2= 0 otherwise 1 for soil type H C3= 0 otherwise

  11. To answer each of the three questions of interest in a two-way ANOVA, a hypothesis test based on an f statistic is available. In each case the null hypothesis states that there is no difference in means from the corresponding effect. Each f statistic is a ratio of mean squares, similar to the one-way ANOVA f statistic. The sums of squares, degrees of freedom, mean squares, and f statistics can all be summarized into a two way ANOVA table which is often organized as follows, where n represents the total sample size: rc– 1 r– 1 c– 1 (r– 1)(c– 1) n–rc n– 1

  12. The assumptions on which f-tests in one-way or two-way ANOVA are based include a normal distribution within each group/cell (which can be checked by examination of residuals) and equal variance across groups/cells (which can be checked with Levene’s test). These assumptions are discussed on pages 160 and 161 of the textbook. The two-way ANOVA table is structured similar to the one-way ANOVA table and has a column for the different sources of variation, a column for degrees of freedom (df), a column for sums of squares (SS), and a column for mean squares (MS); also included are four f statistics with corresponding p-values. Each of the four f statistics is calculated by dividing the mean square in the corresponding row by the mean square in the Error row. The Overall Model row focuses on the overall variation among cell means, and the f statistic in this row could be used in a hypothesis test to decide if there is at least one difference among the cell means (analogous to the hypothesis test in a one-way ANOVA); however, this hypothesis test is generally not of interest, since it does not focus on either the individual row and column variables or on the interaction between these variables. The degrees of freedom in this row is the number of cells rc minus one (1).

  13. The Rows row focuses on the variation in cell means resulting from the row variable, and the f statistic in this row could be used in a hypothesis test to decide if there is at least one difference among the row means. The degrees of freedom in this row is the number of categories for the row variable r minus one (1). The Columns row focuses on the variation in cell means resulting from the column variable, and the f statistic in this row could be used in a hypothesis test to decide if there is at least one difference among the column means. The degrees of freedom in this row is the number of categories for the column variable c minus one (1). The Interaction row focuses on the variation in cell means resulting from the interaction of the row variable and column variable, and the f statistic in this row could be used in a hypothesis test to decide if there is any statistically significant difference among the cell means resulting from interaction. The degrees of freedom in this row is the product of the degrees of freedom in the Rows row and Columns row. The Error row focuses on the variation within cells, that is, random variation; the mean square in this row is the denominator for each f statistic. The degrees of freedom in this row is the difference between the total sample size and the number of cells. The Total row focuses on the total variation in the observed data when all cells are combined into one sample. The degrees of freedom in this row is the total sample size n minus one (1).

  14. The degrees of freedom in the Overall Model row and in the Error row both add up to the degrees of freedom in the Total row. Also, the sum of squares in the Overall Model row and in the Error row both add up to the sum of squares in the Total row. The degrees of freedom in the Rows row, in the Columns row, and in the Interaction row all add up to the degrees of freedom in the Overall Model row. When the cell sizes in the data are all equal, then the sum of squares in the Rows row, in the Columns row, and in the Interaction row all add up to the sum of squares in the Overall Model row; however this may not be true, if the cell sizes are not all equal. The reason for this is because when the cell sizes in the data are all equal, then there is only one possible technique for computing the sums of squares in the Rows row, the Columns row, and the Interaction row; but when the cell sizes are not all equal, then there may be more than one technique for computing the sums of squares in the Rows row, the Columns row, and the Interaction row. In SPSS, the default technique for computing these sums of squares is called Type III sums of squares. The null hypotheses corresponding to the f tests in the Rows row, Columns row, and Interaction row of the two-way ANOVA table state respectively that there are no Row main effects, there are no Column main effects, and there are no Interaction effects. Generally speaking, if a researcher finds significant interaction, then describing the interaction effects is of primary interest, and whether or not there are main effects is of lesser interest. If a researcher finds no significant interaction, then whether or not there are main effects becomes of primary interest.

  15. Use the dummy variables defined in part (b) to complete the description of the least squares regression equation below the two-way ANOVA table, and identify the terms that correspond to the f-tests in the ANOVA table. Then complete the Source column and the df column in the two-way ANOVA table. (c) (2)(4)  1 = 7 1 Wheat Type 3 Soil Type Interaction (1)(3) = 3 24 –(2)(4) = 16 24 –1 = 23 corresponds to the Soil Type f-test ^ Y = a + b1 R1 + b2C1+ b3C2+ b4C3+ b5R1C1+ b6R1C2+ b7R1C3 corresponds to the Wheat Type f-test corresponds to the Interaction f-test

  16. 2.-continued In the document titled Using SPSS Version 19.0, use SPSS with the section titled Creating new variables by recoding existing variables to add the dummy variables defined in part (b) to the data file; then use SPSS with the section Creating new variables with transformation of existing variables to add the products of dummy variables which are necessary to allow for interaction. (d) In the document titled Using SPSS Version 19.0, look at the section titled Performing a multiple linear regression with checks of linearity, homoscedasticity, and normality assumptions, and notice the following: steps 2 to 6 are not applicable, since the linearity assumption need only be checked for quantitative independent variables; step 7 was already done in part (d); step 10 is not applicable, since Levene’s test can be used to assess the uniform variance assumption (just as is done in a one-way ANOVA); step 12 is unnecessary, since much of the output from this step will be available in a better format from the SPSS option designed specifically for two-way ANOVA. (e)

  17. In view of this, follow the instructions in steps 1, 8, 9, 11, 13, 14, and 15 to create graphs for assessing whether or not the normality assumption appears to be satisfied; then, decide whether or not this assumption appears to be satisfied. This histogram does not look bell-shaped, and the points on the normal probability plot show some departure from the diagonal line.

  18. 2.-continued (f) Based on the histogram and normal probability plot for the standardized residuals in part (e), explain why we might want to look at the skewness coefficient, the kurtosis coefficient, and the results of the Shapiro-Wilk test. Then use SPSS with the section titled Data Diagnostics to make a statement about whether or not non-normality needs to be a concern. Since there appears to be some possible evidence of non-normality in part (e), we want to know if non-normality needs to be a concern. Since the skewness and kurtosis coefficients are each well within two standard errors of zero, and the p = 0.047 is not less than 0.001 in the Shapiro-Wilk test, non-normality need not be a concern.

  19. There are essentially three possible scenarios for the results in a two-way ANOVA: (1) Interaction effects and main effects are all found not to be statistically significant. In this scenario, the researcher would conclude that there is no difference in cell means, or in other words, the row variable and column variable are not significant in predicting the quantitative dependent variable. Hence, no further analysis would be necessary. (2) Interaction effects are found not to be statistically significant, but main effects are found to be statistically significant. In this scenario, the researcher would conclude that there is at least one difference in either row means, column means or both. Hence, further analysis would be necessary. When only two categories are compared, then describing the direction of the difference is necessary; when more than two categories are compared, then a multiple comparison procedure, such as those used when the f test in a one-way ANOVA is statistically significant, can be employed: Tukey’s Honestly Significant Difference (HSD) method, the Least Significant Difference (LSD) method, Bonferroni’s method, and Scheffe’s method are available when equal variances can be assumed, and Tamhane’s T2 method is available when unequal variances are assumed.

  20. (3) Interaction effects are found to be statistically significant. Hence, further analysis would be necessary to describe the interaction effects. The researcher may or may not choose to consider main effects (using the procedures discussed in (2)), since these would now only be of secondary interest. To describe interaction, one possible method is Scheffe’s Multiple Comparison Procedure for Contrasts, which is performed in three steps: Next class, we shall finish Exercise #2: (g) In the document titled Using SPSS Version 19.0, use SPSS with the section titled Performing a two‑way ANOVA (analysis of variance)to obtain the output for a two-way ANOVA.

More Related