kxkxk 3-way Factorial Designs

1. kxkxk 3-way Factorial Designs • kxkxk 3-way designs, effects & causality • 3-way interactions • describing effects and checking if they are . “descriptive” or “misleading” • an example

2. 3-way Factorial Designs The simplest factorial design is a 2x2, which can be expanded in two ways: 1) Adding conditions to one, the other, or both IVs

3. 2) Add a 3rd IV (making a 3-way factorial design) Learning Psyc Methods Learning Psyc Content Ugrads Grads Ugrads Grads Computer Instruction Lecture Instruction Identify the three IVs in this design . . . Specify the properties of each condition/cell of this design . . .

4. 3-Way Factorial Designs • There are 7 omnibus F-tests involved in a 3-way factorial • 3 main effects F-tests (one for each IV) • 3 2-way interaction F-tests (one for each pair of IVs) • 1 3-way interaction F-test • For the example, these F-tests are… • main effects • 2-way interactions • 3-way interaction 1. Topic 2. Instruction Method 3. Ed. level 1. Topic X Inst. Method 2 .Topic X Ed. Level 3. Inst. Method X Ed. Level Topic X Instruction Method X Ed. level

5. Causal Interpretations of 3-way Designs • When can a main effect be causally interpreted ? • When can a 2-way interaction be causally interpreted ? • When can a 3-way interaction be causally interpreted ? When the conditions of that IV are RA, Manip, Etc. • When MEs of both the involved IVs are causally interpretable • When we can causally interpret the differences between causally interpretable simple effects. • When MEs of all 3 IVs are causally interpretable • When we can causally interpret the differences between simple 2- way interactions which each involve causally interpretable differences between causally interpretable simple effects.

6. A 2-way interaction… • “emerges” when there are 2 IVs (categorical or quant) • is the non-additive/joint effect of the 2 IVs • usually describes as “when the effect of one IV is different for different values of the 2nd IV” • Expanding this to define 3-ways… • “emerges” when there are 3 IVS (categorical or quant) • is non-additive/joint effect of the 3 IVs • usually identified as the “when the 2-way interaction of 2 IVs is different for different values of the 3rd IV”

7. What does a 3-way interaction look like? • Remember that a 2-way interaction is, “when the effect of one IV is different for different levels of a 2nd IV” • Extending this to a design with 3 IVs, a 3-way interaction is, “when the interaction of two IVs is different for different levels of a 3rd IV” Practice Difficulty1 10 Easy 90 = 90 Hard 40 < 70 SE of Practice is different for Easy and Hard Tasks Familiar Task Practice Difficulty1 10 Easy 90 = 90 Hard 40 < 70 Unfamiliar Task Practice Difficulty1 10 Easy 60 < 70 Hard 20 < 60 The 2-way interaction of Practice and Difficulty is different for Familiar and Unfamiliar Tasks

8. Remember what the 3-way F-test tells you !!! Familiar Task Practice Difficulty1 10 Easy 80 < 90 Hard 60 = 60 Unfamiliar Task Practice Difficulty1 10 Easy 50 < 80 Hard 40 = 40 F(1, 152) = 4.234, p = .0235 The simple 2-way interaction pattern of Difficulty x Practice is the same for Familiar and Unfamiliar Tasks?!? Where is the interaction? 1st – the significant 3-way tells you that the two simple 2-ways are different – that’s what a 3-way interaction means !!! 2nd – notice the Easy Practice effect is smaller for Familiar than for Unfamiliar – sometimes that’s all it takes for a sig 3-way !!!

9. Basic Rules for Follow-up Analyses: #1 Any significant effect with k>2 conditions will require a follow-up significance test to compare the condition means of that effect to describe the pattern of that effect. #2 Any lower-order efffect involved in a significant higher order effect will require follow-up significance tests to compare the means pattern of that lower-order effect with the means pattern of the corresponding simple effects, to determine if the means pattern of the lower-order effect is descriptive or misleading.

10. Remember… • In a 3-way effect we have 6 lower-order effects • 3  2-way interactions • 3  Main effects • As we discussed… There are 2 situations when we will care! • #1  if the aggregates compared in the effect represent a target population that we care about! • #2  when the effect is descriptive, and can be used to generalize from the aggregate to all the separate conditions/values (in the current study/design) For each lower-order effect we have to decide… Do we care??? If the aggregates represent meaningful populations, but the effect is misleading  we’ll have to carefully describe the situation – when the pattern is descriptive and misleading, etc. If the effect is neither representative not a useful general description, then we may not care about the effect !!!

11. “Descriptive” effects in a 3-way The 3-way -- significant or not -- is always descriptive ! If the 3-way is significant, all 2-way & main effects are “suspect” If the 3-way is significant, a 2-way is only descriptive if that 2-way has the same pattern for each condition of the 3rd IV If the 3-way is significant, a main effect is only descriptive if that main effect has the same pattern for each combination of the other two IVs If the 3-way is non-significant, all three 2-ways are descriptive

12. “Descriptive” effects in a 3-way If a 2-way is significant, the main effects of those 2 IVs are “suspect” If a 2-way is significant, the main effect of an IV involved in that interaction is only descriptive if that main effect has the same pattern for each condition of the other IV The main effect of an IV that is not involved in any 2-way or 3-way interaction is always descriptive

13. kxkxk Stimulus Type Shape Texture #Practices #Practices 1 10 1 10 Modality Vision Touch Bimodal

14. kxkxk Effects for this Design 3-way interaction of x Modality x Practice 2-way interaction of Stimulus x Modality 2-way interaction of Stimulus x Practice 2-way interaction of Modality x Practice Main effect of Stimulus Main effect of Modality Main effect of Practice Simple interaction of Stimulus x Modality at dif values of Practice Simple interaction of Stimulus x Practice at dif values of Modality Simple interaction of Modality x Practice at dif values of Stimulus Simple effect of Stimulus at dif combos of Modality & Practice Simple effect of Modality at dif combos of Stimulus & Practice Simple effect of Practice at dif combos of Modality & Practice

15. kxkxk 100 90 80 70 60 50 Vision Touch Bimodal 10 Practices 1 Practice Shape Texture

16. kxkxk 100 90 80 70 60 50 P E R F O R M A N C E 1 Practices 10 Vision Touch Bimodal Shape Texture

17. kxkxk