Challenges and Opportunities to evaluate Combination Prevention

1. Challenges and Opportunities to evaluate Combination Prevention Marie Laga Institute of Tropical Medicine Antwerp Belgium

2. Challenges to evaluate combination Prevention • Measuring HIV incidence • The complexity of Prevention programs with long causal pathways • The scarcity of monitoring data and process evaluation of current Prevention programs • Interpreting National HIV trends and attributing changes to Prevention programs 5. The evaluation methods to establish program effectiveness

3. Measuring change in HIV incidence • Cohort studies • Lab methods ( BED, STARHS,..) • Proxy’s: prevalence in 15-24y • Estimating incidence from successive prevalence survey’s • Modelling: EPP, Asian Epidemic model, and more recent methods None ideal, most progress with modelling Urgent need to develop HIV incidence assay

4. The complexity of the combination prevention programsneed for Program impact Pathway How is the program expected to reduce HIV incidence? • Spelling out the different steps and make the connections explicit […if..then…] • Attention to include underlying contextual factors resulting in long and complex causal chains • Balance between simplifying reality and not being simplistic • Need for guidelines on methodology and more practical examples

5. Public Changes in awareness, knowledge and attitudes about SF Elected officials Demonstrations of support Organize and implement Smoke-free campaign Mgrs of public areas/events Coalition Time Money Partners including youth Research and best practices Increased commitment, support, demand for SF environments SF policies implemented, enforced Worksite contacts Residential owners, mgrs Increased use of cessation resources Reduction in tobacco use and exposure Organize and implement strategy for treating tobacco addiction Changes in attitudes and motivations • Tobacco users • Adults • Youth Increased # of quit attempts Increased knowledge of availability of cessation resources Influential others Increased # of prevention programs, policies adopted, enforcement Youth Organize and implement strategy to prevent youth tobacco use Change in knowledge, attitude, motivations Parents, schools, etc. Policy makers Change in behaviors Increased commitment to eliminate access Retailers Statewide Tobacco Control: Smoke-free environments OUTCOMES University of Wisconsin-Extension, Program Development and Evaluation

6. Poor monitoring of the current prevention response Is the program implemented according to plan? Coverage? Reach? Quality? • Data not collected • Data not made available or not used • Coverage estimates: definition of numerator and denominator (seize estimates of hard to reach population; Human rights issues ) • Definition of minimum quality standards • Missing baselines

7. Understanding National TrendsAdjusted HIV prevalence* among ANC women, by year, 1996-2006-CAMBODIA 7

8. Understanding National Trends Estimated number of new HIV infections in Thailand 1988-2008 Source: Analysis and Advocacy (A2) Thailand Team, Pattaya presentation to ASAP meeting, January 2006.

9. HIV surveillance is critical data source to evaluate the 100% condom program Source: Bureau of Epidemiology, Ministry of Public Health Thailand

10. Evidence of behavioral change in Thailand Source: Relationships of HIV and STD declines in Thailand to behavioral change: A synthesis of existing studies, UNAIDS 1998

11. Understanding National trendsGeneralised epidemics Source: UNAIDS Report on the global AIDS epidemic, 2008

12. Understanding National Trends: Impact of PreventionThe example of Zimbabwe Natural decline in incidence ~ 1990 Accelerated decline in incidence, due to behaviour change: ~ 2000 Source: Hallett, Gregson, Gonese, et al., Epidemics, 2009

13. Evaluation of Public Health programs Ref: Habicht et al

14. Tanzania Zimbabwe • Teacher-led, peer-assisted in- school sexual and reproductive health education • Youth-friendly sexual and reproductive health services • Community-based condom promotors and distributors • Community-wide activities, to create a supportive environment The evaluation results • no impact on HIV and STI rates after 3 and 8 years • In and out of school youth • Parents and community stakeholders improving their knowledge and changing their attitudes to adolescent reproductive health issues as well as improving their communications skills • Clinic staff to improve accessibility and acceptability of rural clinics to young people • The evaluation results • no impact on HIV or HSV-2 prevalence • a significant reduction in the number of reported current or past pregnancies Ross D et al, 2007; Cowan et al,2008

15. So far, ALL community-RCT of behavioural interventions with HIV as endpoints have shown flat results • The intervention too weak, low adherence? • The control group not enough different? • The outcome HIV too distal? ……

16. The evidence dilemma • Good quality RCT require “ tightly defined interventions” preferebly with a short impact pathway • Combination prevention including “Social movements” advocacy, education, social mobilisation , are impractical to evaluate with RCT because less-well-defined and longer more complex impact pathway

17. Considerations about RCT in HIV prevention • RCT essential for biomedical prevention tool evaluation such as microbicides • RCT usefull for evaluation of well defined program components using intermediate outcomes in the impact pathway • Balance cost of program implementation and learning while doing through “Convergence of evidence” versus cost of trying to prove the “unprovable”…?

18. Showing program impact by Convergence of Evidence Triangulation HIV trends-Behaviour/STI-Program data • ANC surveillance • DHS or other population based HIV prevalence data • IBBS • Special studies: context and determinants “documenting Social tranfsormations” • Attention to Baseline data

19. Criteria for effectiveness: “Plausible attribution’ • Time , place , person • Context and program factors • Causality considerations: Bradford Hill criteria • Modelling to simulate control groups and predict impact

21. IMPACT OF SEX WORKER PROGRAMS in BeninMODELLING OF HIV PREVALENCE in SW and General population [INTERVENTIONS OF PROJETS SIDA-1, SIDA-2, AND SIDA-3.] A B C Provided by M. Alary ICASA Report on West Africa, 2009

22. Combination prevention needs Combination evaluation The use of a diverse range of data (mixed methods of quantitative and qualitative) and information sources , will likely provide more illuminating, relevant and sensitive evidence of effects than a single “definitive” study

23. Some conclusions • Need for better Incidence measurement tools • Need for Program Impact Pathways • Back to the basics: collect and use program Monitoring data in HIV Prevention evaluation • “Convergence of data sources” and modelling gets us a long way in Prevention evaluation • The role of C-RCT in documenting efficacy has been limited by complex nature of combination prevention programs

24. Some conclusions • Need to better document Combination Prevention successes to respond the current crisis of confidence • The glass is half full, but can be filled more