1 / 12

Chapter 9 – Assessing Program Designs Alternative Design

Chapter 9 – Assessing Program Designs Alternative Design. Michael Wurdeman. Bias in Estimation of Program Effects.

africa
Download Presentation

Chapter 9 – Assessing Program Designs Alternative Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 9 –Assessing Program DesignsAlternative Design Michael Wurdeman

  2. Bias in Estimation of Program Effects • Bias comes when either the measurement of the outcome with the program exposure or the estimate of what the outcome would have been without program exposure is higher or lower the the corresponding “true” value. • Problems occur when it is not possible to view the target without the program.

  3. Bias

  4. Selection Bias • Comparing target groups to a control group. • Works when both groups would be the same had they both not received program, random assignment. • Not randomly assigned groups or Nonequivalent comparison design

  5. Selection Bias • Volunteer • Subtle differences • Attrition • Targets drop out of the intervention or control group and cannot be reached • Targets refuse to cooperate in outcome measurement.

  6. Other Sources of Bias • Secular Trends • Long-term trends • Interfering Events • Short-term trends • Maturation • Time

  7. Quasi-Experimental Impact Assessment • Quasi-Experimental are designs that do not involve randomly assigned intervention and control groups. • Use of quasi-experiments requires extreme care to ensure as much equivalence as possible between groups that will be compared.

  8. Constructing Control Groups by Matching • Choosing Variables to Match • Use research literature to match important variables • Matching Procedures • Constructed through either individual or aggregate matching.

  9. Equating Groups by Statistical Procedures • Multivariate Statistical Techniques • A statistical model that represents the overall set of relationships among the control variables and outcome variables. • Modeling the Determinants of Outcome • Multiple Regression based on dependent variable. • Modeling the Determinates of Selection • Dependent variable is selection criteria instead of outcome.

  10. Regression-Discontinuity Designs • A quasi-experimental design in which selection into the intervention or control group is based on the observed value on an appropriate quantitative scale, with targets scoring above a designated cutting point on that scale assigned to one group and those scoring below assigned to the other. Also called a cutting-point design.

  11. Reflexive Controls • Simple Pre-Post Studies • Before and after study • Time-Series Designs • Strongest reflexive control design • Consists of multiple observations over time. Not just pre and post.

  12. Some Cautions About Using Quasi-Experiments for Impact Assessment • It is appropriate for evaluators to use quasi-experimental designs for impact assessments when randomized designs are not feasible, but only with considered efforts to minimize their potential for bias and acknowledgement of their limitations.

More Related