PSYCH 240: Statistics in Psychology

1. PSYCH 240: Statistics in Psychology Post Hoc Comparisons: Contrasting Means 

2. ANOVA as an Overall Analysis of the Means The Purpose of the Test In conducing the test, we are interested in assessing differences among the multiple groups This amounts to an omnibus test of group differences This provides our first glimpse into the data: “Is there something there?”

3. Notes on Interpretation If the F ratio is non-significant, we conclude that the differences are due to chance: “Why go further?” If the F ratio is significant: Some differences among the means are due to treatment We do not know which means in particular are different

4. Making Comparisons Comparison: The difference between two or more means in the context of a more general analysis Methods for Making a Comparison The independent samples t test Using an F ratio for two groups

5. Making Mean Comparisons Raw Effect Size for the comparison: The SE of this difference is:

6. SPSS ANOVA Output

7. HSD Post Hoc Tests for the Example

8. PSYCH 240: Statistics in Psychology Post Hoc Comparisons: Controlling Error Rates 

9. Problems with Multiple Tests The paradox of omnibus testing Often conducted to eliminate multiple tests Don’t tell you much of true interest Must follow-up by doing multiple tests Consequences of multiple tests Overlap in the information provided Inflates Type I Error

10. Determining Error Rates Two Important Error Rates (Alpha): Family-Wise: Overall Type I Error probability Per-Comparison: Probability of Type I Error for each specific comparison Type I Error Rate for Multiple Tests: where c = # of comparisons

11. Inflated Type I Error and Pairwise Comparisons

12. Controlling Type I Error What not to do: Conduct multiple tests but don’t worry about inflated Type I Error How to Control Type I Error Only conduct one test Only conduct post-hoc tests if have a significant omnibus F ratio Adjust alpha / critical value per comparison

13. PSYCH 240: Statistics in Psychology Post Hoc Comparisons: Post Hoc Procedures 

14. Making Comparisons Types of Comparisons Planned: Comparisons that you planned to make before the omnibus analysis Post Hoc: Comparisons you decide to make following the omnibus analysis The type of comparison is important because it affects your evaluation of significance

15. Significance of a Planned Comparison For a planned comparison You are allowed to use the “normal” critical value for the t test Note, however, that your degrees of freedom have changed (because of the use of MSWITHIN) For post hoc comparisons, the critical values will change for important reasons

16. Tukey’s HSD Procedure Purpose: Test all possible comparisons between pairs of treatment means (pairwise comparisons) Procedure: Adapt a t test for the comparison Obtain a more stringent critical value for evaluating significance

17. Tukey’s HSD Procedure In order to assess the significance of our comparison, we adapt the t test formula: Interpretation of the HSD Statistic The sampling distribution of HSD Critical values of HSD

18. HSD Post Hoc Tests for the Example