1 / 22

Analysis of Variance

Analysis of Variance. Mean Comparison Procedures. Learning Objectives. Understand what to do when we have a significant ANOVA Know the difference between Planned Comparisons Post-Hoc Comparisons Be aware of different approaches to controlling Type I error Be able to calculate Tukey’s HSD

stasia
Download Presentation

Analysis of Variance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis of Variance Mean Comparison Procedures

  2. Learning Objectives • Understand what to do when we have a significant ANOVA • Know the difference between • Planned Comparisons • Post-Hoc Comparisons • Be aware of different approaches to controlling Type I error • Be able to calculate Tukey’s HSD • Be able to calculate a linear contrast • Understand issue of controlling type 1 error versus power • Deeper understanding of what F tells us.

  3. What does F tell us? • If F is significant… • Will at least two means be significantly different? • What is F testing, exactly? • Do we always have to do an F test prior to conducting group comparisons? • Why or why not?

  4. When F is significant… • Typically want to know which groups are significantly different • May have planned some comparisons in advance • e.g., vs. control group • May have more complicated comparisons in mind • e.g., does group 1 & 2 combined differ significantly from group 3 & 4? • May wish to make comparisons we didn’t plan • Can potentially make lots of comparisons • Consider a 1-way with 6 groups!

  5. Decisions in ANOVA True World Status of Hypothesis Our Decision H0 True H0 False Reject H0 Don’t Reject H0

  6. Concerns over Type I Error • How it becomes inflated in post-hoc comparisons • Familywise / Experimentwise errors • Type I error can have undesirable consequences • Wasted additional research effort • Monetary costs in implementing a program that doesn’t work • Human costs in providing ineffective treatment • Other… • Type II error also has undesirable consequences • Closing down potentially fruitful lines of research • Loss costs, for not implementing a program that does work

  7. Beginning at the end, post-hocs… • Fisher’s solution – a no-nonsense approach • The Bonferroni solution – keeping it simple • The Dunn–Šidák solution – one-upping Bonferroni • Scheffé’s solution – keeping it very tight • Tukey’s solution – keeping a balance • Newman-Keuls Solution – keeping it clever • Dunn’s test – what’s up with the control group?

  8. Recall our ongoing example…

  9. The Bonferroni Inequality • The multiplication rule in probability • For any two independent events, A and B… • the probability that A and B will occur is… • P(A&B)=P(A)xP(B) • Applying this to group comparisons… • The probability of a type 1 error = .05 • Therefore the probability of a correct decision = .95 • The probability of making three correct decisions = .953 Bonferroni’s solution: α/c

  10. Bonferroni t’ / Dunn’s Test • Appropriate when making a few comparisons • Attributes • Excellent control of type 1 error • Lower power, especially with high c • Can be used for comparison of groups, or more complex comparisons • Linear contrasts Dunn-Šidák test is a refined version of the Bonferroni test where alpha is controlled by taking into account the more precise estimate of type 1 error:

  11. Tukey’s approach 1) Determine r: number of groups (3 in our teaching method example) 2) Look up q from table (B.2): using r and dfW (3 & 21  rounding down to 20 = 3.578). 4) Determine HSD: 4) Check for significant differences

  12. Student Newman-Keuls Example from One-Way ANOVA where k=7 MSW = 0.80, dfW = 28, n=5

  13. S-N-K cont’d

  14. And, finally… Homogenous subsets of means… Problems with S-N-K and alternatives

  15. Post-Hoc Comparison Approaches • The Bonferroni Inequality • Flexible Approaches for Complex Contrasts • Simultaneous Interval Approach • Taking magnitude into account • If you have a control group

  16. What if we were clever enough to plan our comparisons? • Linear Contrasts • Simple Comparisons • To correct or not correct…

  17. Orthogonal Contrasts • What are they? • How many are there? • How do I know if my contrasts are orthogonal? • When would I use one? • What if my contrasts aren’t orthogonal?

  18. Simple Example • Three treatment levels • Wish to compare A & B with C In words, A & B combined aren’t significantly different from C 1. Next we need to derive contrast coefficients, thus we need to get coefficients that sum to zero. First, multiply both sides by 2… 2. Then, subtract 2C from both sides. 3. A & B have an implied coefficient of “1”

  19. SS for C1 • All contrasts have 1 df • Thus, SS = MS • Error term is common MSW, calculated before

  20. Why? • Imagine another outcome… Now, F = 2.60, p > .05 MSW = 3.87

  21. Other types of contrasts • Special • Helmert • Difference • Repeated • Deviation • Simple • Trend (Polynomial)

  22. Final thoughts/questions • Do we need to do an ANOVA / F test? • What is your strategy for determining group differences? • Which methods are best suited to your strategy / questions?

More Related