1 / 14

Introduction: Research design

Introduction: Research design. 17.871 Spring 2012. The Biggest Problem in Research: Establishing Causality. Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if.

sal
Download Presentation

Introduction: Research design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction: Research design 17.871 Spring 2012

  2. The Biggest Problem in Research: Establishing Causality • Return to the case of voting machine problems in Florida • After the election, we wanted to know: are some machines “better” than others? • For the policy choice, we want to know if

  3. The Biggest Problem in Research: Establishing Causality • Return to the case of voting machine problems in Florida • After the election, we wanted to know: are some machines “better” than others? • For the policy choice, we want to know if

  4. The Biggest Problem in Research: Establishing Causality • Return to the case of voting machine problems in Florida • After the election, we wanted to know: are some machines “better” than others? • For the policy choice, we want to know if

  5. The Biggest Problem in Research: Establishing Causality • Return to the case of voting machine problems in Florida • After the election, we wanted to know: are some machines “better” than others? • For the policy choice, we want to know if

  6. The problem • How do we make sure that quality differences observed among machine types are due to machine types per se • This is an issue of causality • We attend to “internal validity” so that when we observe differences between groups, we can assure ourselves that this is because of the “treatments” of interest

  7. Review of internal and external validity

  8. 1. Nonrandom selection into the treatment group (confounding variables) Comparing apples with apples or apples with oranges? Random assignment ensures apple to apple comparisons Regression, matching, difference-in-differences also attempt to compare apples with apples 2. Reverse causation The chicken and egg problem, which came first? Is your dependent variable influencing your treatment (your explanatory variable)? If you can address these problems, you almost always have an internally valid study Randomly assigned experiments address both Internal validity: the two problemsThe two primary threats to internal validity.

  9. Is your sample representative of the population? Make sure your study population is relevant to the general population Address by randomly sampling External validity

  10. Good research is about addressing Internal validity External validity

  11. Clarification • Randomly sampling cases gets you? • External validity • Randomly assigning to treatment group? • Internal validity • Controlling for variables with regression addresses? • Internal validity • What study design addresses both internal and external validity? • Field experiments

  12. What is gold standard research design? • Field experiment, e.g., Connecticut voting turnout • Why? Addresses • Internal validity • Nonrandom selection into the treatment • Reverse causation • External validity • What aspects of our lives are governed by gold standard research? • In this class, we mostly do observational studies, • But the key to a successful observational research is always keep in mind how one study differs from a field experiment

  13. Next class: STATA • Kohler & Kreuter,Data analysis (2nd edition) • Chapter 1 • Skip section 1.3.19 (linear regression) • Chapter 3 • Only read section 3.1 • Chapter 5 • Read section 5.1 but skip 5.1.3 and 5.1.4 • Read section 5.2 • Handout: “How to use the STATA infile and infix commands” (course website)

  14. If you want to play around with Stata Visit http://ist.mit.edu/services/software/athena/numerical for basic info about accessing Stata in Athena Look on pp. xxi-xxii of Kohler & Kreuter for info about download example data

More Related