Evaluating HIV Prevention and Treatment Programs

1. Evaluating HIV Prevention and Treatment Programs Damien de Walque Markus Goldstein

2. (Impact) Evaluating Health Programs is different • Norm in medicine to use randomized control trials to figure out what works • What we know less about (for example): • Socioeconomic effects of health interventions • How to get people to utilize treatment (e.g. vaccines) • What is the most cost effective mode of treatment

3. This is an issue for HIV/AIDS • Examples: • ART works biologically, but what impact does it have on patients socio-economic status? • Circumcision is effective in reducing HIV risk, how do we get people to volunteer? • We know different prevention interventions have some effect – which is the most cost effective?

4. This presentation • We will not focus on the medical findings • We know something about non-medical outcomes through evaluation, but actually not that much rigorous (in a comparison group sense) evidence • But, a lot in progress, and we will talk about some of that

5. Example 1 • Duflo, Dupas, Kremer & Sinei: Education and HIV AIDS Prevention • The interventions: • training teachers in the Kenyan government’s HIV/AIDS-education curriculum • organized debate and essay contest on the role of condoms in protecting teens against HIV/AIDS • reduced cost of education through the supply of school uniforms • information campaign for Kenyan teenagers to spread the awareness of high HIV prevalence among adult men (uses earlier evaluation work)

6. Evaluation design • Schools randomly assigned to treatment groups and control • Baseline and endline data collection to use a difference in difference approach • Measured the effect of the program on: • Teacher coverage of HIV/AIDS • Student HIV knowledge • Student attitudes • Self reported behavior • Child bearing (girls) and drop out rates

7. Main findings • Teacher training led to no significant reduction in teen pregnancy, but increased likelihood that pregnancies occur within marriage • Debate over condoms led to (some) increased report of condom use • Reductions in the cost of schooling led to reduction in drop out rates and reduction in teen pregnancies

8. Prevention of HIV/AIDS Source: Duflo, Dupas, Kremer, and Sinei (2006)

9. Example 2 • Thirumurthy, Graff Zivin, and Goldstein: The Economics of AIDS Treatment: Labor Supply in Western Kenya • The intervention • Provide ART to patients, rule is for a CD4<200, but initial rationing • Treatment for opportunistic infections • Nutrition supplementation, but not much until later • Data – two rounds of hh survey – random sample & patients

10. 30 20 Mean hours worked in past week 10 0 -50 0 50 100 150 200 250 300 350 400 Days on ARVs treated counterfactual Evaluation Design • We know what happens to counterfactual group • Medical evidence: continued decline in health and death • Allows estimation of upper bound of treatment impact • Zero labor supply in round 2

11. BMI Before and After ARV Therapy LFP before and after ARV Therapy Source (right): Household survey data. Source: AMPATH Medical Records System – data as of March 2005.

12. Main Findings • Large and rapid labor supply response in patients • Spillover benefits to other members of the household – young boys & women reduce their labor supply • Patient earnings (relative to zero counterfactual) are close to the cost of treatment

13. Some work in progress

14. Question 1: Socio-economic impact of reducing premature adult mortality: the case of ART • In addition to labor supply: • Schooling and welfare of children • How household coping mechanisms change with ART • Effects of ART on farming productivity • Using evaluation results for macro models of the economic effects of AIDS & treatment

15. Question 2: Possible effects of ART on HIV transmission and prevention

16. Other questions (more on the supply- facility side) • 3) Adherence to treatment • 4) How to avoid the development and spread of resistance? • 5) How are ART beneficiaries identified? How to encourage timely uptake? • 6) How to assure the quality of HIV/AIDS service delivery? • 7) How to encourage capacity building to reinforce the sustainability of ART delivery?

17. Methodology and data collection (Longitudinally) • Biomedical follow-up including data on treatment regimen and treatment success (CD4 counts) • Household surveys (HIV patients and general population) including health, schooling, labor force. • 7 countries: Burkina Faso, Ghana, Kenya, India, Mozambique, Rwanda and, South Africa • Surveys ongoing and scheduled. Data and preliminary analysis will be available by the end of 2008.

18. Impact On health system Training Quality of service delivery Equipment Socio- Economic benefits for households Staffing and Incentives Associations Selection / Recruiting Employers Socio- Economic benefits for firms Community variables Stigmatization Patient Adherence Information Prevention socio-economic variables Framework for Learning Agenda Determinants Patient and Provider Behavior Outcomes Impact on the entire country Treatment Outcome, Resistance Development

19. Methodological challenges: endogeneity • Given issues with randomization, in some countries, we will evaluate some experiments on the conditions of ARV delivery. • Rwanda: performance-based contracting for HIV/AIDS services in health facilities • South Africa: food and counseling intervention as adherence support. • Kenya: text messaging intervention as reminders for adherence

20. Some other common issues for HIV evaluations • Measurement. Dupas & co. raise the issue of what should be the impact variable, particularly in prevention • Issue of self reported behavior (Gersovitz) • Think about using biomarkers – HIV or STI?

21. Comdom use at the last intercourse with the spouse: discordant reportsSource: DHS 2003 and 2004

22. Issues in HIV evaluation, cont. • Control Group: Thirumurthy et. al. point to the problem of a medical intervention known to be effective and the problem of generating a counterfactual • Sequel paper looks at children, here the authors use a range of control group techniques • The shape & determinants of the epidemic vary across time and across countries, so results in one country may not apply to another…need to do multiple evaluations of the same intervention