Download
1 / 30
300 likes | 479 Views
Psych 5510/6510. Chapter 13: ANCOVA: Models with Continuous and Categorical Predictors Part 1 : Increasing Power in True Experimental Designs. Spring, 2009. ANCOVA: “The analysis of Covariance”.
E N D
Psych 5510/6510 Chapter 13: ANCOVA: Models with Continuous and Categorical Predictors Part 1: Increasing Power in True Experimental Designs Spring, 2009
ANCOVA: “The analysis of Covariance” Originated in the analysis of experimental designs. The goal was to investigate the effects of the categorical variables while controlling some continuously measured variable, called a covariate.
Model Comparison Approach In the model comparison approach we are simply putting both continuous and categorical variables into our model. Usually this will involve using categorical variables to code our independent variable (i.e. which experimental group the subject belongs in) and continuous variables to measure some other aspect of each subject (something that is not being manipulated by the experimenter, e.g. height or age).
Contexts We will be looking at three contexts in which this will be useful: • Within a ‘true experimental’ design, where we can use this approach to increase the power of the design and to add sophistication to our model. • Within a ‘quasi-experimental’ or ‘static group’ design, where we can use this approach to control a confounding variable. • Within a correlational design, where we can introduce a categorical variable to better understand a continuous variable.
Context 1: True Experimental Designs True experimental design: subjects are randomly divided into groups, the independent variable is then manipulated by the experimenter. As the subjects are randomly assigned to groups, it is assumed that the group means start off being fairly equal. If, after the independent variable has been applied, a statistically significant difference between the group means is found it is interpreted as being the result of the independent variable.
‘Priming’ Example 60 subjects are randomly divided into two groups. Each subject is shown two words, a ‘priming’ word for 2 seconds, followed by a ‘test’ word. The perceptual threshold of the test word is measured. For Group 1, the priming word is similar in shape to the test word, for Group 2, the priming word is similar in meaning to the test word. • IV: Type of prime (shape or meaning) • DV: perceptual threshold For the analysis contrast coding (1 and –1) was used to code the independent variable.
Results: t Test for Independent Groups Mean Group 1: 31.13 Mean Group 2: 29.42 t(58)=1.942, p=.057
Interpretation Well, the results were not quite statistically significant, we cannot conclude that the independent variable had an effect. Perhaps if the experiment just had a little more power we could have rejected the null hypothesis. In the previous semester we examined what influences power, in this case we will focus on the variance within the groups. If we can reduce the variance of the scores within the groups then we can increase the power of the experiment.
Within-Group Variance Group 1: Standard deviation=3.07 Group 2: Standard deviation=3.71 Is that a lot? Well, it’s hard to say, but we can think about reducing it. To do that we can ask the question, ‘why do the scores differ within each group’? For our purposes in this chapter we will refine the question to ‘what measurable attribute of the subjects might be correlated to the dependent variable (perceptual threshold)’. Age comes to mind. There was a wide variety of ages within each group, if age is correlated with perceptual threshold, and the ages of the participants varies within each group, then it could account for some of the within-group variance.
Is Age Correlated with Perceptual Threshold? MODEL C: Ŷi = β0 MODEL A: Ŷi = β0 + β1Agei
Adding Age to the Experiment Up to now our approach has been: MODEL C: Ŷi = β0 MODEL A: Ŷi = β0 + β1Groupi H0: β1= 0 Ha: β1 0 Now we are going to move to: MODEL C: Ŷi = β0 + β2Agei MODEL A: Ŷi = β0+ β2Agei + β1Groupi H0: β1= 0 Ha: β1 0
Adding Age to the Exp. (cont.) MODEL A: Ŷi = β0+ β2Agei + β1Groupi The mechanics of this are simple, we are going to measure the subject’s age and add that variable to the model along with the variable that uses a contrast code to indicate which group they are in.
Why This Helps (ANOVA) If we think about this in terms of the t test for independent groups, what we are accomplishing is to remove from each group the variance that can be accounted for by knowing the subject’s age. If that reduces the variance in each group then the power of the experiment should increase.
Why This Helps (Regression) But let’s think about this in terms of the model comparison approach. Previous: MODEL C: Ŷi = β0 MODEL A: Ŷi = β0 + β1Groupi Now: MODEL C: Ŷi = β0 + β2Agei MODEL A: Ŷi = β0+ β2Agei + β1Groupi If we are interested in the worthwhileness of adding variable ‘Group’ to the model, why would adding it to a model that already contains ‘Age’ be better then adding it to a model that didn’t contain ‘Age’?
Why This Helps (Regression) To understand the explanation we need to note that variables ‘Age’ and ‘Group’ are likely to be fairly non-redundant. Remember that redundancy can be thought of as how much you can use one variable to predict the other. We have randomly divided people into groups, so the groups probably have a fairly similar distribution of ages, consequently we shouldn’t be able to use what group a person is in to predict their age, or vice versa. If the mean age in each group is the same then age and group are completely independent (non-redundant). The mean age in the two groups, however, will probably not be exactly the same so there could be a small amount of redundancy.
Why This Helps (Regression) In the following diagrams I show how adding X1 to a model of Y that already contains X2 is more powerful than without X2, but only if the predictor variables are not very redundant.
Note while the amount of Y that can be explained by X1 is the same in both cases, the PRE is greater below:
The situation would be different if X1 and X2 were quite redundant:
Redundancy of Age and Group The correlation between Age and Group is r=.007 (very low). If we square that to get the value of R² we get a value very close to zero, which would make the tolerance of Age and Group essentially 1. The redundancy doesn’t need to be this low for the covariate to add power, but I’m not complaining.
Results with Covariate (Age) Included Overall Analysis
Results with Covariate (Age) Included Both Group (p=.0367) and Age (p=.001) are worthwhile when added last to a model that contains the other predictor. Without Age in the model (our previous analysis) Group was not significant. The tolerances (not shown above) are very close to 1.00, indicating that Age and Group have very little redundancy.
Summary Without Age in the model the t value (from both the t test for independent groups as well as from the regression analysis) for the effect of Group (the independent variable) was 1.942, p=.057. With Age in the model the t value for the effect of Group was 2.135, p=.037. What happened? In terms of the t test for independent groups, including Age in the model removed the variance that could be accounted for by Age before the evaluation of the effect of Group, thus power was increased. In terms of model comparison, including Age in Model C lowered SSE(C) in a manner that was not redundant with Group, so the proportional reduction of error by adding Group was greater.
With the tools we now have we can add covariates to any type of experimental design. For example, if we have two independent variables, ‘A’, and ‘B’, and A has two levels (contrast X1 can handle that), and B has three levels (contrast X2 and X3 can handle that), and we also are looking at the interaction of A and B (contrasts X4 and X5 can handle that), and we want to add two covariates ‘Age’ (X6) and ‘Height’ (X7) to gain power then we regress Y on: Ŷi = β0+β1X1 +β2X2 +β3X3 +β4X4 +β5X5 +β6X6 +β7X7 If Age and Height are not very redundant with the contrasts that code the independent variables then they will increase the power of the tests of those contrasts. Fancier Designs
A Powerful Tool This procedure provides a very simple tool for increasing the power of a true experimental design. • Think of some reason why scores will differ within groups (e.g. age, income level, height, gender). • Measure that. • Test to see if that measure is significantly correlated with the dependent variable (i.e. regress the dependent variable on the measure). • If it is, add it to the model to increase the power of the test for the independent variable(s).
Ways of Thinking About It If we approach our original example from the perspective of a t test for independent groups then our focus is on whether or not the independent variable (type of priming) had an effect on the dependent variable (perceptual threshold). Including the covariate of Age is simply a means of increasing the power the experiment, but our focus remains on the effect of the independent variable.
Ways of Thinking About It If we approach what we are doing from the model comparison approach, then our focus is on trying to model perceptual thresholds, and we are interested in whether it would be good to have both Age and Type of Priming be part of our model. Our analysis shows that both are worthwhile. I wonder if they interact? Wouldn’t it be interesting if the effect of ‘type of priming’ was different across ages? Let’s check it out. We’ll create another variable that is (Group)x(Age) to test for an interaction...
Interactive Model MODEL C: Ŷi = β0 + β1Groupi + β2Agei MODEL A: Ŷi = β0 + β1Groupi + β2Agei + β3GroupiAgei PRE = -1.50²=.02 p=0.260 Moving to an interactive model was not worthwhile, and we can see that including the interaction term made the effect of Group no longer statistically significant. Why? A look at the tolerances (not shown in this table) shows a great deal of redundancy between Group and the Group x Age interaction. So, let’s leave the interaction out of our model.
The Model MODEL: Ŷi = β0 + β1Groupi + β2Agei This is our best model of perceptual threshold so far, I wonder what variable to try next? We could think of another continuous variable that we could measure in our next study, or we might want to manipulate some independent variable and see if it adds to the model. The goal is to work towards a better and better model of perceptual threshold. This is the flavor of the model comparison approach.
