1 / 14

Regression Discontinuity Design Can Be Your Friend: Developing Evidence in the Real World

Regression Discontinuity Design Can Be Your Friend: Developing Evidence in the Real World. Applied Research Seminar Public Policy Research Center. David Kimball Adriano Udani Department of Political Science, UMSL. Agenda. Purpose Scholarship and Contributions Design Application.

shaine-robinson
Download Presentation

Regression Discontinuity Design Can Be Your Friend: Developing Evidence in the Real World

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Regression Discontinuity Design Can Be Your Friend:Developing Evidence in the Real World Applied Research Seminar Public Policy Research Center David Kimball Adriano Udani Department of Political Science, UMSL RD Design and Applications

  2. Agenda • Purpose • Scholarship and Contributions • Design • Application RD Design and Applications

  3. Regression discontinuity design • Method to estimate treatment effects in natural setting • Observed continuous variable and causal variable of interest exhibit a discontinuous increase at a certain threshold • Address confounding factors influencing control and treatment • Empirically verify assumptions • strengthens internal validity • Applies to observations only near “cutoff point” • limits external validity RD Design and Applications

  4. Thistlewaite and campbell (1960) • Impact of scholarships on future academic outcomes • Awards based on test scores, measured against cutoff point (c) • If score > c, then individual receive an award • Estimated treatment effect applies to individuals near the cutoff point • Assume these individuals have similar characteristics • EXCEPT receipt of award RD Design and Applications

  5. Types of RD studies RD Design and Applications

  6. Studies that use rdd • Health • Low birth weight babies (Almond et al. 2010) • Young adults who lose health insurance (Anderson et al. 2012) • Education • U.S. School Bond Referenda (Cellini, Ferreira, and Rothstein 2010) • Management studies • Yelp.com ratings (Anderson and Magruder 2012; Lucas 2012) • Political Science • Split tickets in the Senate (Butler and Butler 2005) • Incumbency effect (Snyder 2005) * • Coattails of Members of Congress (Broockman 2009) • U.K. House of Commons (Eggers and Hainmueller 2009) • Close House Races (Caughey and Sekhon 2011) • U.S. mayoral races (Gerber and Hopkins 2011) RD Design and Applications

  7. RDD: Treatment EFFECT RD Design and Applications Source: Perraillon (2013): http://home.uchicago.edu/~mcoca/docs/hrs_rdd_slides_f.pdf

  8. Black Mayors hire more black police Source: Hopkins and McCabe 2012 RD Design and Applications

  9. Evaluate RD Assumption Theoretically • Be wary of RD design if there is strategic behavior or manipulation near threshold. • Information • Incentives • Capacity RD Design and Applications

  10. Test RD assumptions Empirically • Balance test: Plotting means of pre-treament covariates in control group vs. treatment group (difference of means). • Density test: Examine distribution of observations just above and just below threshold. • Test causal direction (outcome or treatment DOES NOT predict pre-treatment DV or other covariates) • Placebo test: Look for other discontinuities in the range of scores. RD Design and Applications

  11. Check Stability of RD REsults • Test different specifications. • Linear • Polynomial • Local regression • Test different “discontinuity samples” (different bandwidths). • Test sensitivity to inclusion of pretreatment covariates RD Design and Applications

  12. Missouri Schools Application RD Design and Applications

  13. Implications for Policy Analysis • RD design is an appealing form of natural experiment. • Weak assumptions compared to other empirical methods • In many cases the assumptions are plausible • Policymakers might consider a threshold for policy applications – this would favor empirical analysis of the policy’s impact. RD Design and Applications

  14. References • Joshua D. Angrist and Jörn-Steffen Pischke, Mostly Harmless Econometrics (Princeton University Press, 2008). • Thad Dunning, Natural Experiments in the Social Sciences (Cambridge University Press, 2012). • Andrew C. Eggers, Anthony Fowler, Jens Hainmueller, Andrew B. Hall, and James M. Snyder, Jr. 2014. “On the Validity of the Regression Discontinuity Design for Estimating Electoral Effects: New Evidence from over 40,000 Close Races.” American Journal of Political Science (May 2014). RD Design and Applications

More Related