1 / 52

Group comparisons: Intro to factorial design

Group comparisons: Intro to factorial design. 31:43/Josh Rodefer, Ph.D. . Previously (?). Experimental design Design: Between vs Within Compare 2 groups (t test; different types) Paired (within; repeated measures) Unpaired (between) Control & confounds

aurorette
Download Presentation

Group comparisons: Intro to factorial design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Group comparisons:Intro to factorial design 31:43/Josh Rodefer, Ph.D.

  2. Previously (?) • Experimental design • Design: Between vs Within • Compare 2 groups (t test; different types) • Paired (within; repeated measures) • Unpaired (between) • Control & confounds • Extraneous variables (random vs systematic) • How to control or minimize?

  3. Basic issues in experimental • What do you need for an experiment? • An independent variable (what the researcher varies; minimally 2 groups) • Equal assignment to groups (lots of ways to do this) • Controlling extraneous variables (alternative explanations; confounds)

  4. Hypothesis testing: 2 samples • Want to test relationship between the means • Use a t-test • Used when you have 2 levels of 1 IV • Dependent samples (1 sample, tested twice) • Independent samples (2 different samples) • aka Within- (dependent) and Between (independent) designs

  5. Hypothesis testing: 2 samples • Dependent samples (Within) • Eg, before and after treatment (or pre/post) • 20 people tested on attitude towards recycling after watching nature video • IV=viewing nature video • Level 1: test score pre-video • Level 2: test score post-video • DV score on test • 2 samples, same people in each sample

  6. Hypothesis testing: 2 samples • Independent samples (between) • 2 scores are taken, but from different groups; 2 independent samples (random) • Eg, study effects of age on swearing • 20 ten yr olds and 20 fifteen yr olds, record swearing in 2 hr period • IV=age • Level 1= 10 yrs old • Level 2= 15 yrs old

  7. More than 2 groups: ANOVA • Analysis of Variance • Tests the differences between treatment groups (conditions) to see if they are significant • Look at variance in DV, partition it into 2 components: • variance due to IV (good) • variance due to error (bad; extraneous variables) • Asks if the ratio (IV variance/error variance) between the two types of variance is greater than would be expected due to chance (or equal; or about =1.00) • F test examines this ratio

  8. ANOVA – when to use? • One IV: at least 3 conditions • Between subjects: One-way ANOVA • Eg, effect of psychiatric diagnosis (depression, panic, no disorder) on memory • Within subjects: Repeated measures ANOVA • Eg, experiment where subjects experience all conditions; example Stroop effect, colors/words • Two (or more) IVs • Factorial ANOVA • Effect of psychiatric diagnosis on Stroop effect; multiple IVs

  9. First: One way ANOVA • One IV: 3 (or more conditions) • Single test • Tests Null that all group means are equal • Alternative: all group means are not equal • Null must be non-directional (vs t test directional) • Between subjects ANOVA • Analogous to independent samples t-test • (stats trivia: In fact….if you have 2 independent groups, then F=t2)

  10. One way ANOVA • Why not just perform 3 t-tests? • Psychiatric diagnosis & memory example • Depression vs panic disorder • Depression vs no disorder • Panic disorder vs no disorder • Same thing, right?

  11. One way ANOVA • Multiple tests result in an inflated alpha • Alpha for each test =0.05 (risk for Type I) • If you do 3 tests, then each is at 0.05 and they are additive • Thus, doing 3 t-test increases the experiment-wise Type I error rate from 0.05 to 0.15 (not acceptable) • Must use ANOVA to test all combinations simultaneously, keeping alpha at 0.05

  12. One way ANOVA • So, we can calculate the total variance and determine how much is due to IV and how much is error • F ratio = variance due to IV /error variance • Variance due to IV is called MSB (Mean square between groups) • Variance due to extraneous variables or error is called MSW (Mean square within groups) • F test formula: F = MSB/MSW

  13. ANOVA • Most data sets have both error variance and variance due to the IV (or, both between and within group variability) • We want to know if the between group variance is due to a true effect of the IV (is this effect real?) • F=MSb/MSw • If F = about 1, then no effect of IV • If F > 1 then may have an effect of IV (need to look up value on F table, depends on critical F and df)

  14. Factorial ANOVA: 2 way designs • Have 2 (or more) IVs in the same experiment; eg, test effects of gender and age on humor: • IV gender (M/F) • IV age (10, 20, or 30 yrs old) • DV freq of laughing during comedy show • Why not conduct 2 separate experiments? • Gender on humor; age on humor

  15. Factorial ANOVA: 2 way designs • Why do you want to look at 2 IVs? • More efficient: study factor A then study factor B…or…you could study them together • More interesting: can see how things relate together • More representative of the real world

  16. Factorial designs • Have 2 (or more) IVs • Each IV is called a ‘factor’ • Each factor has at least 2 levels/conditions • The design must be complete; each level of one IV must occur with each level of the other IV(s) • Described as: 2x2 or 3x5 or…2x3x4 • Terminology: Factors/Main effects; levels; cells • Can be experiment or quasi-experiment (what’s quasi?)

  17. Factorial designs • Example: examine effect of sleep and caffeine on reaction time study • IV-1: Sleep: 2 hrs vs 8 hrs • IV-2: Caffeine: zero (water) vs 3 Cokes • DV: computer game reaction time • Both IVs are between subjects variables

  18. Factorial design • Randomly select subject for each condition; equal cell sizes • 2 IVs; test for 3 effects: • 2 Main effects (effects of sleep and caffeine) • 1 interaction (interaction of sleep & caffeine) • Thus, 3 null hypotheses, 3 alternative hypotheses, and 3 F tests

  19. Factorial designs • Main effect • The effect of one factor (IV) collapsed across levels of the other factor (IV) (ie, you ignore the other IV) • Test one main effect for each factor (IV) • Need to analyze statistics to check for significance

  20. Factorial design: Main effect graphic 8 hrs 2 hrs 0 Cokes 3 Cokes Did different amounts of sleep have an effect? Mean of 2 hrs Mean of 8 hrs

  21. Factorial design: Main effect graphic 8 hrs 2 hrs 0 Cokes Mean of 0 Cokes 3 Cokes Mean of 3 Cokes Did different quantities of Coca-cola have an effect?

  22. Post hoc tests • Post hoc means after the fact • Tests that are performed after finding a significant effect (sig F value) • ANOVAs tell you if there is a difference, but they don’t tell you where it is • Post hoc tests answer that question…

  23. Post hoc tests • Would you need a follow up post hoc test after a significant t-test? • Would you need to follow up a significant F-test where there are 3 levels of the IV with a post hoc test (say, for the sleep experiment you used 2, 4 or 8 hrs of sleep as three different levels of the IV).

  24. Post hoc tests • So, need to use post hoc tests when… • You have 3 or more means • You have a significant difference with a general/omnibus test (F test) • Post hoc tests (pairwise comparisons) examine all possible combinations of means

  25. Post hoc tests • Many types: Tukey’s HSD (honestly significant difference), Bonferroni, Scheffe’s, Kruskal-Wallis, Dunnett’s… • Each tests pairwise hypotheses • Each calculates a particular statistic that resembles a t-test (but it isn’t that simple)

  26. Post hoc tests • Consider the 3 level IV, where you find a significant F value • Next need to assess all possible pairwise comparison: • Mean 1 =/diff mean 2 • Mean 1 =/diff mean 3 • Mean 2 =/diff mean 3 • Compute pairwise comparison for each test…ascertain where the difference(s) are statistically significant

  27. Next time • More factorials & interactions

  28. Previously -- 2nd Lecture • Experimental design (Between vs Within) • Compare 2 groups (paired vs unpaired) • Control & confounds (random vs systematic) • ANOVAs (3 or more groups) • Do overall F tests • Pairwise compartions (post hoc tests) • (like t tests; done after significant F; tells you where the difference is; that is, which two means are significantly different from each other)

  29. ANOVA – when to use? • One IV: at least 3 conditions • Between subjects: One-way ANOVA • Eg, effect of psychiatric diagnosis (depression, panic, no disorder) on memory • Within subjects: Repeated measures ANOVA • Eg, experiment where subjects experience all conditions; example Stroop effect, colors/words • Two (or more) IVs • Factorial ANOVA • Effect of psychiatric diagnosis on Stroop effect; multiple IVs

  30. Factorial designs • Have 2 (or more) IVs • Each IV is called a ‘factor’ • Each factor has at least 2 levels/conditions • The design must be complete (sometimes called ‘completely crossed’); each level of one IV must occur with each level of the other IV(s) • Described as: 2x2 or 3x5 or…2x3x4 • Can be experiment or quasi-experiment

  31. Factorial designs • Example: examine effect of sleep and caffeine on reaction time study • IV-1: Sleep: 2 hrs vs 8 hrs • IV-2: Caffeine: zero (water) vs 3 Cokes • DV: computer game – number correct • Both IVs are between subjects variables

  32. Factorial design • Randomly select subject for each condition; equal cell sizes • 2 IVs; test for 3 effects: • 2 Main effects (effects of sleep and caffeine) • 1 interaction (interaction of sleep & caffeine) • Thus, 3 null hypotheses, 3 alternative hypotheses, and 3 F tests

  33. Factorial designs • Main effect • The effect of one factor (IV) collapsed across levels of the other factor (IV) (ie, you ignore the other IV) • Test one main effect for each factor (IV)

  34. Factorial designs • Interactions are a way to qualify findings • Revisit coke and sleep study 8 hrs 2 hrs 0 Cokes 3 Cokes

  35. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes

  36. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes What does the parallel shift suggest to you?

  37. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes

  38. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes Spreading interaction (aka ordinal); usually at least one main effect

  39. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes But what about if we plotted this differently?

  40. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes

  41. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes Cross-over interaction (aka disordinal); opposite effects of IV1 at different levels of IV2; usually no main effects, but still can have interaction (!)

  42. Interactions • So look at main effects • Look for interactions • Graphing things helps • parallel lines = no interactions • crossing or close to crossing lines = interaction • But ultimately you need stats and p values to make judgement about significance

  43. Factorials • So with 2 Independent variables you can have multiple types of designs • Both IV are within • Both IV are between • One IV is within and one IV is between • Also can have 1 true IV, and 1 quasi-IV (sometimes called ‘mixed’ or ‘experi-corr’) • Lets you consider unmanipulated factors like subject variables (sex, age, diagnoses, etc.) • The more correlational something is, the more external validity (and less internal validity) you have

  44. Within designs • Simple 2 group comparison or ANOVA, doesn’t matter…. • Limitations? • How to fix?

  45. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions?

  46. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD

  47. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC,

  48. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC, ACBD,

  49. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC, ACBD, ACDB, ADBC, ADCB • BCDA, BCAD, BDAC, BDCA, BACD, BADC • CDAB, CDBA, CABD, CADB, CBAD, CBDA • DABC, DACB, DBAC, DBCA, DCAB, DCBA

  50. Counterbalancing • Too confusing…more simple way is a Latin Square design • Number conditions = number of orders • Each condition appears at each position only once C D A B 1st condition 2nd condition 3rd condition 4th condition

More Related