1 / 16

Using A Regression Discontinuity Design (RDD) to Measure Educational Effectiveness:

Using A Regression Discontinuity Design (RDD) to Measure Educational Effectiveness:. Howard S. Bloom MDRC 12-11-02 Howard.bloom@mdrc.org. This Talk Will:. introduce the history and logic of RDD, consider conditions for its internal validity, considers its sample size requirements,

royal
Download Presentation

Using A Regression Discontinuity Design (RDD) to Measure Educational Effectiveness:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using A Regression Discontinuity Design (RDD) to Measure Educational Effectiveness: Howard S. Bloom MDRC 12-11-02 Howard.bloom@mdrc.org

  2. This Talk Will: • introduce the history and logic of RDD, • consider conditions for its internal validity, • considers its sample size requirements, • consider its dependence on functional form, • illustrate some specification tests for it, • consider limits to its external validity, • consider how to deal with noncompliance, • describe an application.

  3. RDD History • In the beginning there was Thislethwaite and Campbell (1960) • This was followed by a flurry of applications to Title I (Trochim, 1984) • Only a few economists were involved initially (Goldberger, 1972) • Then RDD went into hibernation • It recently experienced a renaissance among economists (e.g. Hahn, Todd and van der Klaauw, 2001; Jacob and Lefgren, 2002) • Tom Cook has written about this story

  4. RDD Logic • Selection on an observable (a rating) • A tie-breaking experiment • Modeling close to the cut-point • Modeling the full distribution of ratings

  5. RDD As A Linear Regression

  6. Conditions for Internal Validity • The outcome-by-rating regression is a continuous function (absent treatment). • The cut-point is determined independently of knowledge about ratings. • Ratings are determined independently of knowledge about the cut-point. • The functional form of the outcome-by-rating regression is specified properly.

  7. RDD Statistical Model where: Yi = outcome for subject i, Ti = one for subjects in the treatment group and zero otherwise, Ri = rating for subject i, ei = random error term for subject i, which is independently and identically distributed

  8. Variance of the Impact Estimator s2 = variance of mean outcomes across subjects in the treatment group or comparison group R12 = square of the correlation between outcomes and ratings within the treatment and comparison group R22 = square of the correlation between treatment status and the rating P = proportion of subjects in the treatment group, N = total number of subjects

  9. Sample Size Implications • Because of the substantial multi-collinearity that exists between its rating variable and treatment indicator, an RDD requires 3 to 4 times as many sample members as a corresponding randomized experiment

  10. Specification Tests • Using the RDD to compare baseline characteristics of the treatment and comparison groups • Re-estimating impacts and sequentially deleting subjects with the highest and lowest ratings • Re-estimating impacts and adding: • a treatment status/rating interaction • a quadratic rating term • interacting the quadratic with treatment status • Using non-parametric estimation

  11. External Validity • Estimating impacts at the cut-point • Extrapolating impacts beyond the cut-point with a simple linear model • Estimating varying impacts beyond the cut-point with more complex functional forms

  12. Dealing With Noncompliance • Sharp and fuzzy RDDs • No-shows and crossovers • The effect of intent to treat (ITT) • The local average treatment effect (LATE) • The effect of treatment on the treated (TOT) Where rT and rC = the proportion of the treatment and control groups receiving treatment, respectively

  13. Application of RDD To Reading First • Reading First (RF) is a cornerstone of No Child Left Behind • RF resources are allocated purposefully to schools that need it most and will benefit most • Some districts allocated RF resources based on quantitative indicators • We chose a sample of 251 schools near the cut-points for 17 such districts and 1 state

  14. References Cook, T. D. (in press) “Waiting for Life to Arrive: A History of the Regression-discontinuity Design in Psychology, Statistics and Economics” Journal of Econometrics. Goldberger, A. S. (1972) “Selection Bias in Evaluating Treatment Effects: Some Formal Illustrations” (Discussion Paper 129-72, Madison WI: University of Wisconsin, Institute for Research on Poverty, June). Hahn, H., P. Todd and W. van der Klaauw (2001) “Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design” Econometrica, 69(3): 201 – 209. Jacob, B. and L. Lefgren (2004) “Remedial Education and Student Achievement: A Regression-Discontinuity Analysis” Review of Economics and Statistics, LXXXVI.1: 226 -244. Thistlethwaite, D. L. and D. T. Campbell (1960) “Regression Discontinuity Analysis: An Alternative to the Ex Post Facto Experiment” Journal of Educational Psychology, 51(6): 309 – 317. Trochim, W. M. K. (1984) Research Designs for Program Evaluation: The Regression-Discontinuity Approach (Newbury Park, CA: Sage Publications).

More Related