Statistics for the Social Sciences

1. Statistics for the Social Sciences Psychology 340 Fall 2006 Analysis of Variance (ANOVA)

2. Outline • Brief review of last time ANOVA • Post-hoc and planned comparisons • Effect sizes in ANOVA • ANOVA in SPSS

3. Example • Effect of knowledge of prior behavior on jury decisions • Dependent variable: rate how innocent/guilty • Independent variable: 3 levels • Criminal record • Clean record • No information (no mention of a record)

4. Observed variance F-ratio = Variance from chance XA XC XB Analysis of Variance Test statistic • Need a measure that describes several difference scores • Variance • Variance is essentially an average squared difference • More than two groups

5. Testing Hypotheses with ANOVA • Hypothesis testing: a five step program • Step 1: State your hypotheses • Step 2: Set your decision criteria • Step 3: Collect your data • Step 4: Compute your test statistics • Compute your estimated variances • Compute your F-ratio • Compute your degrees of freedom (there are several) • Step 5: Make a decision about your null hypothesis • Additional tests: Planned comparisons & Post hoc tests • Reconciling our multiple alternative hypotheses

6. The ANOVA tests this one!! XA XC XB Testing Hypotheses with ANOVA • Hypothesis testing: a five step program • Null hypothesis: H0: all the groups are equal • Step 1: State your hypotheses • Alternative hypotheses (HA) • Not all of the populations all have same mean Choosing between these requires additional test

7. XA XC XB 1 factor ANOVA • Planned contrasts and Post-hoc tests: • Further tests used to rule out the different alternative hypotheses • reject • reject • fail to reject • Alternative hypotheses (HA) • Not all of the populations all have same mean

8. Why do the ANOVA? • What’s the big deal? Why not just run a bunch of t-tests instead of doing an ANOVA? • Experiment-wise error • The type I error rate of the family (the entire set) of comparisons • EW = 1 - (1 - )c where c = # of comparisons • e.g., If you conduct two t-tests, each with an alpha level of 0.05, the combined chance of making a type I error is nearly 10 in 100 (rather than 5 in 100) • Planned comparisons and post hoc tests are procedures designed to reduce experiment-wise error

9. Which follow-up test? • Planned comparisons • A set of specific comparisons that you “planned” to do in advance of conducting the overall ANOVA • General rule of thumb, don’t exceed the number of conditions that you have (or even stick with one fewer) • Post-hoc tests • A set of comparisons that you decided to examine only after you find a significant (reject H0) ANOVA

10. Planned Comparisons • Different types • Simple comparisons - testing two groups • Complex comparisons - testing combined groups • Bonferroni procedure (Dunn’s test) • Use more stringent significance level for each comparison • Basic procedure: • Within-groups population variance estimate (denominator) • Between-groups population variance estimate of the two groups of interest (numerator) • Figure F in usual way

11. Post-hoc tests • Generally, you are testing all of the possible comparisons (rather than just a specific few) • Different types • Tukey’s HSD test (only with equal sample sizes) • Scheffe test (unequal sample sizes okay, very conservative) • Others (Fisher’s LSD, Neuman-Keuls test, Duncan test) • Generally they differ with respect to how conservative they are.

12. Effect sizes in ANOVA • The effect size for ANOVA is r2 • Sometimes called 2 (“eta squared”) • The percent of the variance in the dependent variable that is accounted for by the independent variable • Size of effect depends, in part, on degrees of freedom • See table 9-7 in textbook for what is considered “small” “medium” and “large”

13. ANOVA in SPSS • Let’s see how to do a between groups 1-factor ANOVA in SPSS (and the other tests too)