Psychology 340 Spring 2010

1. Statistics for the Social Sciences Analysis of Variance (ANOVA) Psychology 340 Spring 2010

2. Outline • Basics of ANOVA • Why • Computations • Post-hoc and planned comparisons • Power and effect size for ANOVA • Assumptions • SPSS • 1 factor between groups ANOVA • Post-hoc and planned comparisons

3. Outline • Basics of ANOVA • Why • Computations • Post-hoc and planned comparisons • Power and effect size for ANOVA • Assumptions • SPSS • 1 factor between groups ANOVA • Post-hoc and planned comparisons

4. Example • Effect of knowledge of prior behavior on jury decisions • Dependent variable: rate how innocent/guilty • Independent variable: 3 levels Compare the means of these three groups Guilt Rating Criminal record Guilt Rating Jurors Clean record No Information Guilt Rating

5. Observed variance F-ratio = Variance from chance XA XC XB Analysis of Variance Test statistic • Need a measure that describes several difference scores • Variance • More than two groups

6. Testing Hypotheses with ANOVA • Hypothesis testing: a five step program • Step 1: State your hypotheses • Step 2: Set your decision criteria • Step 3: Collect your data • Step 4: Compute your test statistics • Compute your estimated variances • Compute your F-ratio • Compute your degrees of freedom (there are several) • Step 5: Make a decision about your null hypothesis • Additional tests: Planned comparisons & Post hoc tests • Reconciling our multiple alternative hypotheses

7. The ANOVA tests this one!! XA XC XB Testing Hypotheses with ANOVA • Hypothesis testing: a five step program • Null hypothesis: H0: all the groups are equal • Step 1: State your hypotheses • Alternative hypotheses (HA) • Not all of the populations all have same mean Choosing between these requires additional test

8. XA XC XB 1 factor ANOVA • Planned contrasts and Post-hoc tests: • Further tests used to rule out the different alternative hypotheses • reject • reject • fail to reject • Alternative hypotheses (HA) • Not all of the populations all have same mean

9. Why do the ANOVA? • What’s the big deal? Why not just run a bunch of t-tests instead of doing an ANOVA? • Experiment-wise error (see pg 398, Box 13.1 for discussion) • The type I error rate of the family (the entire set) of comparisons • αEW = 1 - (1 - α)c where c = # of comparisons • e.g., If you conduct two t-tests, each with an alpha level of 0.05, the combined chance of making a type I error is nearly 10 in 100 (rather than 5 in 100) • Planned comparisons and post hoc tests are procedures designed to reduce experiment-wise error

10. Which follow-up test? • Planned comparisons • A set of specific comparisons that you “planned” to do in advance of conducting the overall ANOVA • Post-hoc tests • A set of comparisons that you decided to examine only after you find a significant (reject H0) ANOVA • Often end up looking at all possible pair-wise comparisons

11. Planned Comparisons • General Rule of Thumb • Don’t plan more contrasts than (# of conditions – 1) • Different types • Simple comparisons - testing two groups • Complex comparisons - testing combined groups • Bonferroni procedure (Dunn’s test) • Use more stringent significance level for each comparison • Divide your desired α-level by the number of planned contrasts

12. Planned Comparisons • Basic procedure: • Within-groups population variance estimate (denominator) • Between-groups population variance estimate of the two groups of interest (numerator) • Figure F in usual way

13. XA XC Planned Comparisons • Example: compare criminal record & no info grps XB 1) Within-groups population variance estimate (denominator) • 2) Between-groups population variance estimate of the two groups of interest (numerator)

14. XA XC Planned Comparisons • Example: compare criminal record & no info grps XB 1) Within-groups population variance estimate (denominator) • 2) Between-groups population variance estimate of the two groups of interest (numerator) • 3) Figure F in usual way • α = 0.05 • Fcrit (1,12) = 4.75 • Fail to reject H0: Criminal record and no info are not statistically different

15. Post-hoc tests • Generally, you are testing all of the possible comparisons (rather than just a specific few) • Different types • Tukey’s HSD test (only with equal sample sizes) • Scheffe test (unequal sample sizes okay, very conservative) • Others (Fisher’s LSD, Neuman-Keuls test, Duncan test) • Generally they differ with respect to how conservative they are.

16. Effect sizes in ANOVA Recall: • The effect size for ANOVA is r2 • Sometimes called η2 (“eta squared”) • The percent of the variance in the dependent variable that is accounted for by the independent variable

17. Effect sizes in ANOVA • The effect size for ANOVA is r2 • Sometimes called η2 (“eta squared”) • The percent of the variance in the dependent variable that is accounted for by the independent variable

18. ANOVA Assumptions • Basically the same as with T-tests • Assumes that the distributions are Normal • Assumes that the distributions have equal variances • In both cases ANOVA analyses are generally robust against violations of these assumptions

19. ANOVA in SPSS • Let’s see how to do a between groups 1-factor ANOVA in SPSS (and the other tests too) • Enter the data: similar to independent samples t-test, observations in one column, a second column for group assignment • Analyze: compare means, 1-way ANOVA • Observations -> Dependent list • Group assignment -> factor • specify any comparisons or post hocs at this time too • Planned Comparisons (contrasts): are entered with 1, 0, & -1 • Post-hoc tests: make sure that you enter your α-level

20. Analysis of Variance