Randomized Control Trials for Agriculture

1. Randomized Control Trials forAgriculture Pace Phillips, Innovations for Poverty Action www.poverty-action.org

2. Outline • What we do • IPA Ghana • IPA’s Mission and Approach • Why evaluate? • How to evaluate impact? • Examples of evaluations

3. What does IPA do? • IPA was created in 2002 to determine what works and what does not work in poverty alleviation through rigorous evaluation. • Estimate magnitude of effects • Most cost-effective solutions; can be surprising • Learn how to improve programs through testing operational questions • Determine where to spend limited resources

4. IPA Ghana • In Ghana since 2005 • 16 projects nationwide • Permanent offices in Accra and Tamale • Current Agric Partners: • Ghana Insurers Association • GIZ • IFPRI • Presbyterian Agricultural Service • MoFA • SARI • ISSER • Trainings /Conferences

5. Why focus on impact evaluation? • Surprisingly little hard evidence on what works • Can do more with given budget with better evidence • If people knew money was going to programs that worked, could help increase pot for anti-poverty programs • Instead of asking “do aid/development programs work?” should be asking: • Which work best, why and when? • How can we scale up what works?

6. Process and impact evaluation Inputs Outputs Outcomes • Process evaluations • Monitoring In the control of the organization Impact Monitoring/Process evaluation: tracking development and outputs of a program as compared to stated objectives, targets and timelines Impact evaluation:how, and how much, does the organization affect the welfare of beneficiaries (and non beneficiaries)

7. How do you evaluate a program? • In Northern Ghana, a development organization undertakes a program to promote fertilizer use to improve yields. How do we know if the program was successful?

8. Impact: What is it? Intervention Impact Crop Yields Counterfactual Time

9. What is the impact? Intervention Crop Yields Impact Counterfactual Time

10. What is the impact? Counterfactual Impact Intervention Crop Yields Time

11. Counterfactual • Counterfactual:What would have happened in the absence of the program? • Problem: Counterfactual is not observable  the key goal of all impact evaluation methods is to construct or “mimic” the counterfactual. • Solution: Counterfactual is often constructed by selecting a group not affected by the program.

12. Why randomize? • Without Random Selection, the control group will have intrinsic differences that can bias the study. • Because members of the groups (treatment and control) do not differ systematically at the outset of the experiment • Any difference that subsequently arises between them can be attributedto the program rather than to other factors • If properly designed and conducted, randomized experiments provide the most credible method to estimate the impact of a program 13

13. Random sampling and random assignment Randomly sample from area of interest

14. Random sampling and random assignment Randomly sample from area of interest Randomly assign to treatment and control Randomly sample from both treatment and control

15. Which method we use matters! *: Statistically significant at the 5% level

16. Underinvestment in Agriculture • A study in Northern Ghanato understand if farmers do not invest in their farms because either capital constraint or risk aversion. Year 2: 1377 farmers in 74 villages Year 2 Randomization Control: 177 farmers Insurance: 729 farmers Capital: 363 farmers Both: 108 farmers Rainfall index insurance offered at varying prices GHS 350 provided Rainfall index insurance offered at varying prices, AND GHS 350 provided

17. Results • Capital only farmers • Used more inorganic fertilizer only • Capital and insurance farmers • Increased farm investment by 20 percent • Insurance farmers • Increased total farm expenditure by 13 percent • Increased inorganic fertilizer use by 25 percent • Increased cultivation area by 8 percent • Increased expenditures on land preparation by 12 percent • Increased total labor use on plots by 13 percent • Increased harvest output by 8 percent • Increase in investment but no increase in profitability

18. Demand for Agriculture Insurance Uptake

19. A Well Timed Nudge • A study in Western Kenya tested whether farmers purchases of fertilizer was dependent on the time of the year it was offered. Baseline: 1,125 Farmers in Suri Kenya Randomization Fertilizer offered AFTER HARVEST with free delivery Fertilizer offered just BEFORE GROWING SEASON with free delivery CHOICE of when they want the fertilizer offer with free delivery. 50% Subsidy Fertilizer offered just BEFORE GROWING SEASON with free delivery Control – No Offer

20. Results • Farmers had high demand and ability to purchase fertilizer in advance. • Impact of the After Harvest offers were comparable to that of a 50% subsidy at fertilizer application time. • Increase in Fertilizer Use • Control Group 28% • After Harvest 39% • Choice of Time 47% • Before Growing Season 33% • 50% Subsidy Before Growing Season 41% • Once the program stopped, fertilizer usage went back to what it had been.

21. Thank You www.poverty-action.org

22. IPA Ghana • Why do farmers under-invest in farms? • What is the impact of financial savings programs in schools? • What is the willingness to pay for clean water? • What is the impact of remedial education programs, smaller class size, teacher training and tracking on literacy and numeracy? • What is the impact of health insurance education on enrollment and reenrollment in NHIS? • Are mobile reminders an effective way to improve adherence to ACT regimens?

23. Problems Selecting a Control Group Without Random Selection, the control group will have intrinsic differences that can bias the study. Difference between Groups Selection Bias Intrinsic Differences Impact of the program Observables Non Observables

24. Our approach We generate insights on what works and what does not through randomized evaluations, and ensure that those findings will be useful to, and used by practitioners and policy makers Innovate Evaluate Replicate Communi-cate Scale • Understand market failures • Develop innovative solutions to poverty • Use frontier knowledge from economics, and psychology • Randomized Controlled Trials • Impact evaluations • Comparing variations of an intervention • Experiment with product designs • Replicate evaluations in various settings to : • - Generalize research findings • Tell practitioners what works (and not), when • Effectively communicate to practitioners: • Conferences • Workshops with policy makers and practitioners • Policy memos and focus notes • Facilitate scale-up of effective solutions : • Active policy outreach • Practitioners’ toolkits • - Hands-on technical assistance