Supplemental regarding simple effects

1. Supplemental regarding simple effects

2. Simple effects* • Often you will see people say that one cannot test for simple effects in the presence of a non-significant interaction, which makes sense in that we see simple effects as a way to understand the specifics of an interaction • However… • There can be significant simple effects without a significant interaction • How can this be?

3. Simple effects • The simple effects of one factor over the multiple levels of another are not partitions of the interaction only • Specifically they test in a one-way ANOVA fashion whether an effect is different from zero • However in general terms of variance they are the breakdown of the variance of the SSinteractand the SSmain effect being tested • Furthermore if one goes ‘backward’ from simple effects to try and draw conclusions about the interaction, one will not find a straightforward path • The interaction can be seen as testing whether or not the simple effects are different from each other whereas the simple effects are testing whether their own effect is different from zero • The problem of ‘accepting the null’ in the face of non-sig simple effects comes into play

4. Simple effects • SSA at B1 + SSA at B2 = SSA + SSA x B • The point of doing the simple effects analysis is to see the specifics of how the levels of an effect are changing over the levels of another • For example, without any knowledge of the interaction, if we had a one sig and one nonsig simple effect we might assume a moderating effect of one variable for the other • However, this would be sketchy due to the presence of the main effect

5. Possible outcomes: All signigificant • A x B design • SSA sig • SSB sig • SSA x B sig • Significant main effects are ambiguous due to interaction. • They are not an effect in isolation • Proceed to simple effects to determine the nature of the interaction.

6. B1 B2 A1 A2 Factor B significant only • SSA non • SSB sig • SSA x B non • Simple effects may show presence of a moderated effect, but only one type of simple effect could allow us to assume an interaction • SSB at levels of A would not tell us that there is a moderated effect of B by A • We have a significant main effect of B • Would the effect be coming from B or the AxB interaction? • If we tested A at B, we could claim a moderated effect if one simple effect was significant and the other not • I.e. the effect is not coming from the main effect of A as it has been shown to be zero

7. B1 B2 A1 A2 Factor A significant only • SSA sig • SSB non • SSA x B non • Here we have the converse situation • In this we can test the simple effect of B at the levels of A for possible moderation effects

8. Non-significance throughout B1 • SSA non • SSB non • SSA x B non • Can test either simple effect without ambiguity B2 A1 A2

9. Significant main effects, no significant interaction • SSA sig • SSB sig • SSA x B non • One should not test for simple effects in this situation as they would be reflections of main effects only • But consider as an example, low n, ‘smallish’ p, noticeable effect size • Simple effects testing might show differential effects for one treatment across levels of another • i.e. all of this discussion focuses on a p-value and we must be concerned about that • Technically though, one should stick to main effects post hocs in such a situation • Possible capitalization of chance

10. Last example* • In this example we have a non-statistically significant interaction (p = .08) • Tests of simple effects reveal a sig effect for the addicted group ( p = .02) but not the non-addicted (p = .09) • Conclusion? • Effect of Treatment for the Addicted group but not for the Non-Addicted group? • Incorrect • Concluding a zero for the non-addicted group is something we cannot do based on the type of test we employ • Remember, we can reject the null, but there are many hypotheses (of varying small effects) that might be viable if we do not. • Again, relying on effect sizes can help alleviate confusions arising from NHST

11. Summary • Note, that in the face of a significant interaction, one should do simple effects • Otherwise one is essentially claiming something without showing exactly what’s going on • However, even in the presence of a non-significant interaction, there are cases where one might still be interested in simple effects, and situations in which one can proceed to test them • Just remember that simple effects are not just a breakdown of the interaction by itself, and this is why we can