More About Factorial Design

1. More About Factorial Design • Suppose experiment tests whether taking a game theory class causes a person to get more papers published • We cannot simply examine whether people who took a game theory class had more papers published due to possible effect of self-selection into game theory class. • Therefore, we should randomly assign people to game theory class.

2. Does Game Theory Lead to More Articles? • DV: Number of papers published 4 years later • IV: Took game theory class or not • Two groups: • Take game theory class • Did not take game theory class

3. Does Class on Experiments Lead to More Articles? • We also may want to determine whether taking a class on experiments helps a person get more articles published. • Now we randomize whether a person takes a class on experiments

4. 2 x 2 Factorial Design Now we Have 4 groups in the experiment, randomly assigned. Each cell should have equal numbers of subjects

5. 2 x 2 x 3 Factorial • Now we randomly assign whether a person studies international relations, public policy, or economics. • This is now a 2 x 2 x 3 factorial design with 12 groups

7. Power Analysis: How Many Subjects Needed? • Example: G*Power (free!) • Hypothesize the magnitude of the effect (such as one additional paper published) • Calculate sample size needed for two independent samples difference in means (average number of articles published for each of two groups) • Answer will be number of subjects needed in each group

8. Natural Experiments Exploiting Randomization When It Occurs

9. Natural Experiments • Experimenter does not pre-plan experiment • Experimenter finds a naturally-occurring random assignment mechanism that serves as a treatment

10. Why Randomization? • Randomization allows comparison across groups by controlling for differences in other variables. • The expected values of other variables are the same between across randomly-assigned treatment groups. • If other variables have equal expected values across treatments, then differences in outcomes must be due to treatment variable

11. Dartmouth Roommate “Experiment” • First-year roommates at Dartmouth College assigned randomly within “like-groups” based on gender and whether student • Smokes • Listen to music while studying • Stay up late • Are neat or messy • Assignment random within 25=32 different groups

12. Treatment=Roommate • In this case, the treatment variable is a student’s roommate • The experiment does not have a true “control group” but many different treatments in the form of roommates • Also in this case, the roommates are “treatments” for each other

13. First Step: Is Assignment Truly Random? • Are there any variables that determine one’s assigned roommate other than • Gender • Smoking • Late night habits • Cleanliness • Listening to music

15. DV’s and IV’s • The DV in this case is a student’s GPA • The most important IV is the roommate’s GPA • Other IV’s included as control variables are a student’s own academic index (entrance exam scores and high school grades) and roommate’s academic index

16. Statistical Model

22. Conclusions • Roommates influence each other’s performance in school • Top performers reinforce each other • Bottom performers reinforce each other • Roommates also influence choice of social group such as fraternity or sorority • Roommates do not influence each other’s choice of major

23. Other Examples of Natural Experiments