1 / 10

Randomized Control Trials

Randomized Control Trials. Difficulties to Consider. Costs. Cost of intervention itself often not difficult to justify Providing goods/services Only including promising possibilities New data are expensive Quality of evaluation dependent on quality of data

dennisramirez
Download Presentation

Randomized Control Trials

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Randomized Control Trials Difficulties to Consider

  2. Costs • Cost of intervention itself often not difficult to justify • Providing goods/services • Only including promising possibilities • New data are expensive • Quality of evaluation dependent on quality of data • More money spent on data is less money spent on providing the intervention to more people

  3. Creating the Control Group • Is it politically feasible to deny treatment to some people? • How important is it to measure how well the intervention works? • Issue of trade-offs • Ethics less contested if: • Budget constraints would have prevented everyone from receiving the intervention anyway • Everyone eventually receives the intervention and the control group is only denied it initially (phased-in rollout)

  4. Does Everyone Benefit? • Necessary to deny control group intervention • But don’t want to actively hurt them • Can’t deceive • Can’t make them worse off than they’d otherwise be • Some sort of small gift/compensation typical – careful not to make this into a second treatment • Must honor promises (phased-in rollout)

  5. Internal Validity • Was the control group valid? • Randomization worked • Intended treatment and control groups balanced • Actual treatment and control groups same as intended • Contamination from spillovers • Was the intervention consistent in all treatment areas? • Easier to guarantee in some cases than in others • Do data exist to rule out alternate hypotheses?

  6. External Validity • Will the subjects in the experiment be representative of the entire population who will eventually receive the intervention? • Logistically, much easier to do data collection in restricted area • Less likely that experiment will generalize to entire country

  7. Data Quality • Sensitive questions • How can we encourage subjects to give honest and complete answers? • Subjective questions • Self-reported vs quantitative measures • “recall error” • Hawthorne effect - People behave differently when they know they’re being watched • Might be desirable to follow them closely for more data • But that might make biases worse

  8. Cost-Effectiveness Comparisons • Resources are scarce – need to pick most effective programs • Need to be able to convert impacts from various projects into one set of units • How to compare improvement in nutrition to reduction in malaria?

  9. Scaling Up • Can intervention be implemented identically at scale? • If not, is RCT still informative? • Will the economy at large respond to the intervention at scale? (“general equilibrium effects”) • Prices might go down – economies of scale • Prices might go up – insufficient supply • Spillover effects could set in

  10. Final Thoughts • RCT is gold-standard in terms of identifying causality • But many complications arise during implementation • Need to weigh theoretical advantages against practicalities – is it really the best method?

More Related