1 / 29

Experimental Design

Experimental Design. Reminders . Graduate symposium deadline Friday CSBS student conference April 26 Thesis defenses Outline due next week References. Validity . What is internal validity? How can it be increased?

ellie
Download Presentation

Experimental Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experimental Design

  2. Reminders • Graduate symposium deadline Friday • CSBS student conference April 26 • Thesis defenses • Outline due next week • References

  3. Validity • What is internal validity? How can it be increased? • What are the relationships between internal, external, and construct validity? • What is needed to establish causality?

  4. Single-group threats to intval • History • Maturation • Testing • Instrumentation • Mortality • Regression to the mean • What are each of these (examples) and how can they be decreased/controlled?

  5. Regression to the mean • Why does it happen? • When will it happen more? • How can you estimate it?

  6. Multiple group threats • Selection • Selection-history • Selection-maturation • Selection-testing • Selection-instrumentation • Selection-mortality • Selection-regression • What are each of these (examples) and how can they be decreased/controlled?

  7. Social interaction threats • Diffusion of treatment • Compensatory rivalry • Resentful demoralization • Compensatory equalization of treatment • What are each of these (examples) and how can they be decreased/controlled?

  8. Design notation • X O • R O X O O O

  9. Importance of random assignment • vs. random selection • What is the strongest design in terms of internal validity threats? • Why would you want to add a pretest? Or not? • Are there times that you wouldn’t want to randomly assign?

  10. Basic design terms • Independent vs. dependent variables • Within vs. between-participants designs: advantages? • What are control variables? • What can you do to reduce confounds in research?

  11. Factorial designs • What are they and why do we care? • What are levels vs. variables? • What kind of FD is this? How many main effects are possible? How many interactions? How many groups? How many people at n=10? • IVs: Presence of an audience, difficulty of task (easy, medium, or hard). DV = nervousness • IVs: Gender, Drug vs. CBT vs. control, previous experience with treatment (yes or no). DV=reduction in symptoms

  12. Main effects and interactions • People who were alone helped more often than those who were with a confederate. Seminary students also helped more often than business students. • Men and women helped equally overall, but men were more likely to help women than to help men.

  13. Examples on the board

  14. Terms • Null results • Main effect • Interaction • Block design • Covariate • Solomon 4-group • Switching design • ANOVA vs. ANCOVA vs. regression

  15. Issues related to design • If a factorial design is good, is a bigger one better? • When would you want to include a variable as an IV vs. a covariate vs. something you match on vs. a control variable? • How do you decide which variables to include?

  16. Placebos

  17. Moderators • What is a moderator? • How is a moderator different from a/an • IV? • DV? • Confound? • Mediator? • Covariate? • How can you design a study to test a moderator (various ways)?

  18. Mediators • What is a mediator? • How is a mediator different from a/an • IV? • DV? • Confound? • Moderator? • Covariate? • How can you design a study to test a mediator (various ways)?

  19. Examples • A study finds that convicted criminals are more likely than noncriminals to score low (negatively) on the Attitudes toward Women Scale. Further research shows, however, that this is only true for violent criminals. There is no relationship between nonviolent criminal activity and negative attitudes toward women. Identify the mediator or moderator.  • 2. A researcher finds that by increasing self-focus in children, she can decrease their likelihood of cheating on a test. By decreasing cheating, in turn, she finds that academic self-efficacy increases. Identify the mediator or moderator. • 3. Dr. Laylor finds a relationship between physical attractiveness and self-confidence. He later determines that the primary cause of this relationship is the positive feedback physically attractive people receive from others. Identify the mediator or moderator. • 4. Boys who are popular with the same-sex tend to also be popular with the opposite-sex, whereas girls who are popular with the same-sex tend to be less popular with the opposite-sex. Identify the mediator or moderator. • 5. Identify and explain at least two potential mediators and two potential moderators for the following relationship: School size and academic achievement.

  20. Analyzing moderators • Case 1: Cat IV, Cat Mod: • 2 x 2 anova • Case 2: Cont IV, Cat Mod: • Could do correlations sep and compare BUT • Better to do regression and use unstandardized coefficients (not betas) • Or SEM software

  21. More moderators • Case 3: Cat IV, Cont Mod: • Figure out how level of moderator affects IV-DV relationship • If linear, do hierarchical regression, showing that XZ adds to the effects of X and Z on Y • Case 4: Both Cont: • Use Case 2 if step function or • Use Case 3 if linear, quadratic • How do you know which one is the moderator vs. IV?

  22. Analyzing mediators • Causal steps (Baron & Kenny) • IV related to M M = i + aX + e • IV related to DV Y = i + c1X + e • M and IV related to DV Y = i + c2X + bM + e • C1 greater than C2 (look at size and sig) • Limitations: • Not good for multilevel, probit, logistic, survival • Need to test for whether C1 > C2 (Sobel test) • Low power esp when IV and DVaren’t related • Overestimates effect of IV on DV if error in M

  23. Sobel test • Used to test whether C1 > C2 • Good for sample sizes of 50+ with 1 M • Or 100-200 for >1 M

  24. Alternatives to B & K • Use multiple measures of M and use SEM • Distribution of the product • PRODCLIN • Better Type I error rates, higher power • Computer-intensive methods • Aka resampling • Fewer assumptions

  25. Assumptions of mediational tests • Residuals are independent in equations 2 and 3 • No XM interaction in equation 3 • Direction is correct (DV doesn’t cause M) • Measurement is perfect, esp. in M • No unmeasured variables that cause X, Y, or M

  26. Other mediation issues • Complete vs. partial mediation • Use hierarchical regression • Test C2 significance • Inconsistent mediation • Multilevel mediation • Can increase T1 error if you ignore • Categorical mediation • Use logistic or probit regression • Multiple mediators

  27. Longitudinal mediation • Moderated mediation • Mediated moderation • Mediated baseline by treatment moderation

  28. Meds and Mods • How do you know which variable is the mediator? • How can a moderator lead you to a mediator? • How can a mediator lead you to a moderator? • Can a variable be both a mediator and a moderator at the same time? • Are there variables that are always going to be one or the other? • Does a mediator have to be correlational? • Remember that one study does not a mediator/moderator make

  29. Outlines • Think of sections as headings in your thesis • Keep headings parallel • Always have at least 2 in each group • Think about the best logical order for things, and keep the order consistent throughout the paper

More Related