Monitoring and Evaluation of MARPs HIV Interventions in Thailand

1. Monitoring and Evaluation of MARPs HIV Interventions in Thailand Kimberley Fox, MD, MPH CDC/GAP Asia Regional Office USAID Regional Development Mission/Asia February 19, 2009 Chennai, India

2. Outline of presentation • Background • Approaches to M&E for HIV prevention in MARPs • IDU: complementary data from program monitoring and RDS survey • Prisoners: M&E in a sensitive context • MSM: M&E using national sentinel surveillance, challenges of rapid scale-up • FSW: evaluating the national program • M&E for a technical assistance-based program

3. HIV epidemic in Thailand • National success in slowing the spread of HIV in the 1990s • HIV prevalence rates in military conscripts and pregnant women were 0.5% and 0.9%, respectively, in 2006 • HIV rates increasing among MARPs • Among MSM in Bangkok, increases from 17% in 2003 to 30% in 2007 • Among IDU, HIV prevalence of 36%-50% • Among sex workers in Bangkok, 20% are HIV-positive and sex work is increasingly non-venue and street-based

4. USG strategy in Thailand • Technical-assistance based approach with focus on capacity building to ensure sustainability and facilitate integration of activities into routine systems • Four components: • Develop replicable models for prevention and care • Improve the quality of prevention and care programs • Increase the collection and use of strategic information • Share successful models and provide TA to other PEPFAR programs

5. Comprehensive Prevention Package • Minimum Package of Services • Behavior change communication • Condom distribution • STI screening and treatment • HIV counseling and testing • Substitution therapy and safer • injection practices for IDU • Linkages to care and treatment • Supportive interventions • Strategic Information • Capacity Building • Community Mobilization • Policy • Stigma and Discrimination • Income Generating Activities Interventions Most-at-risk Populations MSM (including TG, MSW) IDU FSW PLHA Clients of FSWs Other vulnerable populations

6. Example #1--Program monitoring to document trends in outputs and outcomes IDU outreach in Bangkok • Implemented by peer network organized by Bangkok city government • Promote risk reduction • Provide HIV risk reduction information • Demonstrate syringe cleaning and correct condom use • Discuss personal risk-reduction plan • Promote methadone treatment • Provide referrals to methadone clinics • Promote VCT • Provide referrals to VCT services • Refer to HIV, TB, and other medical treatment as needed • Collect program monitoring data • outreach records • referral cards

7. Referral and service access for IDU outreach clients (N=1,131) Referred Accessed service # of IDUs 300 250 200 133 150 84 78 100 43 18 36 50 22 7 0 Tuberculosis VCT Methadone HIV treatment services treatment Of those who accessed VCT services, 11 (13%) tested HIV-positive.

8. Increases in safe behaviors among IDUs with 6 outreach contacts (N=506) % reporting safe behaviors

9. Conclusions • Program monitoring data were able to give us important feedback on: • Positive behavior change associated with repeated outreach contacts • Access to methadone treatment, VCT, and health services as as result of outreach referrals • However, there were several limitations • Potential social desirability bias in reporting risk behavior to outreach workers, even peers • No information on non-participants • No information on program coverage

10. Solution: Periodic survey using respondent-driven sampling (RDS) • Advantages of periodic RDS survey • Reaches hidden populations • Anonymous survey on handheld computers; interviewer has no pre-existing relationship with participant (not outreach peer) • Can incorporate size estimation to provide denominator for coverage • RDS survey of 963 IDUs in Bangkok in 2004 • Estimated IDU population size: 3,600 • Information on risk behaviors and drug treatment experience • IDU RDS survey in Bangkok and Chiang Mai in 2009 • Follow trends in risk behaviors in Bangkok, get first measure in Chiang Mai • Repeat size estimation

11. Example #2--M&E in a sensitive context HIV prevention for prisoners • Peer education • Information on HIV, STI, TB • Promotion and distribution of condoms • Referrals to prison’s STI and VCT services • Improve VCT service and STI care • Monitoring data show: • Very high proportion of prisoners reached by outreach services • In VCT, 8.4% HIV+ and one-third of these have started antiretroviral treatment

12. It’s not a hidden population, but… Program monitoring data have limited variables Program monitoring data are not anonymous Solution: survey on handheld computers Challenges: protecting human rights and confidentiality Multiple ethical reviews Careful informed consent procedures NGO staff as recruiters and interviewers Limitations to program monitoring data in the prison context

13. Prison survey: Key findings • Prisoners with one risk behavior were more likely to have other risk behaviors • Perceived easy access to condoms was the only independent factor associated with condom use during anal sex in prison (AOR 13.0, 95% CI 4.0-43.2) • Access to condoms is a modifiable factor and should be a key component of HIV intervention in Thai prisons

14. MSM Model Health promotion HIV/STI services Peer outreach Within existing services Friendly MSM drop-in center/ clinic Special event MSM venue • STI screening • STI case management • HIV VCT • Prevention for positives • Basic & risk reduction counseling • Condom/ lubricant promotion • VCT promotion • STI screening promotion • Referral to HIV/STI services • Basic HIV/AIDS, STI knowledge • Risk reduction information • Counseling Example #3: M&E for model during scale-up Community

15. All the usual limitations—no data from non-participants, no anonymity Plus…Global Fund Round 8 will fund a rapid scale-up of MSM HIV interventions in 14 provinces Need to harmonize reporting systems across existing services (government, NGOs) to form national M&E system One part of the solution Exit interviews to monitor utilization of services and association with behavior change Coverage Plus surveys 2005 and 2007 Data collected: use of each component of services, intensity of intervention exposure (# of times), risk behaviors Limitations of program monitoring data

16. In 2007, more MSM program beneficiaries received the complete package of services… Source: Coverage Plus 2007, USAID/MEASURE Evaluation

17. …And MSM receiving the complete package at high intensity were more likely to use condoms Source: Coverage Plus 2007, USAID/MEASURE Evaluation

18. MSM sentinel venue-based integrated biological and behavioral surveillance Special survey in Bangkok in 2003, then 3 provinces in 2005 Expanded under MOPH to 5 provinces in 2007, planning 12 provinces in 2009-2010 Data collected: exposure to interventions, risk behaviors, HIV serology Another part of the solution: Use national sentinel surveillance to evaluate overall impact of interventions

19. Coverage of MPS interventions among MSM 2007 vs. 2005, Bangkok Source: Thai MoPH – US CDC Collaboration (2005) & Thai MoPH, BoE (2007)

20. Consistent condom use with casual male partner and receipt of the MPS, Bangkok 2007 Source: Thai MoPH, BoE, 2007

21. Example #4—Monitoring and evaluating a national program: HIV prevention in FSWs • Existing M&E data very limited, years after success reported • Sentinel surveillance reaches primarily venue-based FSW and uses interviewer-administered questionnaire • RDS used to measure the current status of HIV prevention among FSW

22. HIV prevalence among FSW, RDS 2007 compared to routine sentinel surveillance 2007 BangkokChiang Rai RDS 2007Sentinel 2007RDS 2007Sentinel 2007 Sex work type % (95% CI) % % (95% CI) % Direct --* 2.8 11.0 (0.8-41.0) 5.9 Indirect 20.6 (16.7-25.6) 2.4 9.1 (5.3-13.8) 2.1 Total 20.2 (16.4-24.7) 2.5 10.1 (6.6-14.0) 2.6 *RDSAT-adjusted could not be estimated because of small cell sizes; unadjusted prevalence 12.5%

23. Conclusions of RDS survey among FSW • High HIV prevalence among FSW in both cities • Bangkok 20%, Chiang Rai 10% • Sex workers obtain clients in many “places”: brothels, entertainment venues, telephone networks (including college students), internet • Older and lower-price sex workers in Bangkok at high risk of HIV • Mainly non-venue-based sex workers • Highest-risk FSW not reached by routine surveillance • Response • Need to reconsider surveillance approaches • Need to address prevention needs of non-venue-based and other high-risk FSW

24. Challenges and Lessons Learned • Program monitoring data provide critical feedback on program success, but have limitations: • Social desirability bias, limited variables, only reach participants • Anonymous, systematic surveys important to provide accurate data on broader population sample • Most-at-risk populations are dynamic and often hidden • To measure impact, evaluation methods must adapt to changes and must access hidden populations • Special surveys are complex and costly • But to use surveillance data as a tool for measuring impact, data quality must be high • M&E during rapid national scale-up requires harmonization of measures; surveillance may be a useful tool in this setting • Cost-effectiveness evaluation for MARPs interventions is needed as a tool for policy advocacy

25. Challenges to M&E of TA-based Programs • USG contributions to national systems or programs are a small proportion of total funding • Direct outputs attributable to USG support are small; indirect outputs are potentially national • But how do we measure the impact of USG support on the national outputs? • And how can we best measure the success of our TA-based program?

26. Indicator Domains for TA-Based Programs Phase I: Development and evaluation Phase II: Integration and expansion Phase III: Quality assurance • Number of sites • Number of staff trained • Tools, interventions, • materials developed • Monitoring and evaluation conducted • Cost • Feasibility for scale-up assessed: • - Stakeholders engaged • - Funding identified • - Policy support • - Target population • Number (% of target pop) of people reached as result of expansion (indirect results) • Geographic coverage • Non-USG dollars invested in • program scale up • Policies or benefits changed as result of program • Number of institutions with capacity to implement • Number of people trained (directly and ToT) • Quality assessment of expansion conducted • Health status indicators in target population (national) • Ongoing quality monitoring of implementation • Programmatic revisions as needed based on changes in policies or epidemiology

27. Conclusions: M&E of TA-Based Programs • For model development, the key outcome is the progression—from model development and evaluation, to scale-up using other national or donor funding • Indicators should measure not only outputs and outcomes, but also process measures such as the leveraging of funds, non-financial contributions of other partners, and ultimate impact of broad implementation of the model • South-to-south exchange other country-to-country technical assistance activities also need systematic approaches for measurement

28. Acknowledgements • CDC GAP/Asia Regional Office • Chomnad Manopaiboon • Prin Visavakum • Monsicha Poolsawat • Michelle McConnell • Dimitri Prybylski • USAID Regional Development Mission/Asia • Patchara Rumakom • Nithya Mani • Thailand Ministry of Public Health • Bangkok Metropolitan Administration