Monitoring and Evaluation: HIV/AIDS Programs

1. Monitoring and Evaluation: HIV/AIDS Programs

2. Learning Objectives At the end of this session, participants will be able to: • Identify M&E implications of the global HIV/AIDS program context • Identify M&E and information systems implications of the HIV/AIDS program environment • Apply basic M&E concepts to an HIV/AIDS program component • Explain the implications of broadened HIV/AIDS programs, particularly treatment scale up, for monitoring program impact

3. Context

4. Global Summary of the HIV/AIDS Epidemic December 2004 Number of people living with HIV/AIDS Total 39.4 million (35.9 - 44.3 million) Adults 37.2 million (33.8 - 41.7 million) Children under 15 years 2.2 million (2.0 - 2.6 million) People newly infected with HIV in 2003 Total 4.9 million (4.3 - 6.4 million) Adults 4.3 million (3.7 – 5.7 million) Children under 15 years 640 000 (570 000 – 750 000) AIDS deaths in 2003 Total 3.1 million (2.8 – 3.5 million) Adults 2.6 million (2.3 – 2.9 million) Children under 15 years 510 000 (460,000 – 600,000)

5. Global Action to Address the HIV/AIDS Epidemic Word Bank multi-sectoral AIDS Project (MAP) U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) 2000 2001 2003 2004 World Health Organization's call to provide treatment to 3 million people by 2005 (WHO 3x5) (Announced December 1, 2003) United Nations General Assembly Special Session on AIDS (UNGASS) Global Fund for AIDS, Malaria, and Tuberculosis

6. GOALS: Global • Millennium Development Goal6: Combat HIV/AIDS, malaria, and other diseases Target 7: Have halted by 2015 and begun to reverse the spread of HIV/AIDS • WHO 3 by 5 Goal: Universal access to antiretroviral therapy for all living with HIV/AIDS Target: Treating 3 million people by 2005

7. Goals: US Presidents Emergency Plan • Prevention of 7 million new infections • Treat 2 million HIV-infected people • Care for 10 million HIV-infected individuals and AIDS orphans

8. What Are the Goals of HIV/AIDS Programs? • Prevent new HIV infections • Extend and improve life for those already infected with HIV • Mitigate the social and economic impacts of the epidemic

9. Key HIV/AIDS Program Areas Prevention • Behavior Change and Communication • e.g. sexual behavior, condom use, injecting drug use (IDU) behaviors • Medical Interventions • e.g. PMTCT, VCT, blood safety, universal precautions, STI treatment etc. Care and Treatment • Care and support to PLWHA and their families • Prophylaxis and treatment of opportunistic infections (including tuberculosis) • Treatment with antiretroviral therapy (ART) Impact Mitigation • Support to Orphans and Vulnerable Children (OVC) • Reduction of stigma and discrimination • Addressing gender disparities

10. M&E Implications

11. M&E Implications • High emphasis on accountability • Mandatory’ reporting on international indicators (MDG and UNGASS) • Donor-reporting requirements linked to large influx of money • Drive toward standardization

12. The Three Ones: Principles for the coordination of national AIDS responses • One agreed-upon HIV/AIDS Action Framework that provides the basis for coordinating the work of all partners. • One national AIDS authority, with a broad-based multi-sectoral mandate. • One agreed-upon country-level monitoring and evaluation system Source: UNAIDS. 2004. Commitment to principles for concerted AIDS action at country level

13. M&E Challenges • Complex multi-sectoral M&E plans • Wide range of information needed • Different approaches needed in concentrated versus generalized epidemics • Each HIV/AIDS program component has specific M&E needs and challenges

14. M&E Challenges I ALL COMPONENTS • Rapid scale-up of new/routine systems • Denominators – identifying eligible people • Double counting in service statistics

15. M&E Challenges I PREVENTION • Quality of reporting of sensitive behaviors • Identifying size of most-at-risk population VCT & PMTCT • Measuring impact • Quality of services • Service cascade

16. M&E Challenges II CARE AND SUPPORT • Little M&E experience • Often community-based • Minimum care standards • Integration with TB-tracking referrals ARV • Patient-level tracking systems • Adherence

17. M&E Challenges III OVC • Little M&E experience • Often community-based • Minimum package of services • Psychosocial support measurement • Ethical & methodological issues in data collection STIGMA & DISCRIMINATION • Definitions • Measurement tools – validity in different contexts • Selection bias – only disclosed PLWHA observed

18. Applying General M&E Principles to HIV/AIDS Programs

19. General M&E Principles • Determine how data are to be used • Prepare M&E Plan • M&E Framework • Indicators • Data sources and data-collection schedule • Evaluation-design / targeted-evaluations needs • Data reporting and utilization plan • Implement M&E Plan • Use data for program decision-making and reporting

20. Information for Decision Making • Global Level: Are we achieving global goals? • National Level: What should be the national policy on PMTCT? • Program Level: Are we distributing services to meet the need? • Facility Level: Are we providing enough services to meet the need? • Provider Level: What is quality of a care for this patient? • Community Level: What are we doing as a community to meet the need? • Individual Level: Where should I get treatment?

21. Frameworks for HIV/AIDS Programs • Different types of frameworks can be used (e.g. Results framework, log frame) • Different, inter-related frameworks for different program areas (e.g. VCT, PMTCT, care and support) likely to be needed for a comprehensive program. • Output of one program activity may be the input to another • HIV/AIDS frameworks based on relevant documents such as a national AIDS strategy.

22. HIV/AIDS Indicators Program-level, National- and Global-level Indicators (Guides/Sources) First-generation • 2000: General HIV/AIDS programs Second-generation • 2004: HIV/AIDS Care and Support • 2004: Prevention of HIV in infants and young children (PMTCT) • 2004: HIV/AIDS programmes for young people • 2005 update: UNGASS (Millennium Development Goals) • 2005: ARV • 2000-2005: All UNAIDS and partner HIV/AIDS guides • To be released: OVC program guide (FHI), Concentrated Epidemics M&E Guide, and revised GFATM Toolkit (2005/06)

23. Information sources for HIV/AIDS M&E • Document review and key informant interviews • National Composite Policy Index • AIDS Program Effort Index • Routine program information • Annual condom sales • Providers trained in VCT etc. • Routine health information systems • No. client visits for VCT etc. • ARV drugs distributed etc. • Medical records/patient tracking systems • ART adherence • No. patients on ART • PLACE (Priorities for Local AIDS Control Efforts) • Identification and characteristics of sites where risk behaviors take place • Sexual partnership formation at sites

24. Information sources for HIV/AIDS M&E • Facility surveys • Coverage of HIV/AIDS services (facility-based) • Readiness to provide quality HIV/AIDS services • Appropriate STI management • General Population surveys • Sexual behavior • HIV seroprevalence • Targeted Population Surveys/Behavioral Surveillance Surveys • Sexual and other risk behaviors • HIV seroprevalence • Surveillance • HIV seroprevalence • Vital Registration • AIDS mortality

25. Information Storage: CRIS Country Response Information System • Purpose: To enable the systematic • storage • analysis • retrieval • dissemination of collected information on a country’s response to HIV/AIDS • Structure:Integrated system with 3 modules • Indicator • Project / resource tracking • Research Inventory

26. Reporting Schematic

27. EXAMPLE: VCT

28. Illustrative Questions for VCT programs • Are VCT services being provided as planned? • Do services meet minimum quality standards? • Is utilization of services increasing? • Are there reductions in riskier behavior (among those seeking services)? • Are there increases in use of care, support, and treatment services (among those seeking services)?

30. Illustrative Process and Output Indicators • Number of people trained in providing VCT according to national and international standards • Number of people who receive counseling (by sex) • Number of people who receive testing (by sex) • Number of clients who test positive for HIV • Number of HIV+ clients referred to treatment, care, and support services

31. Percentage of Facilities Providing VCT Services Source: 2002 Ghana HIV Service Provision Assessment and 2002 Uganda HIV Service Provision Assessment

32. Percentage of VCT Sites With Selected Inputs for Quality Counseling Percent Source: 2002 Ghana HIV Service Provision Assessment

33. Percentage of VCT Sites With Selected Inputs for Quality Counseling Percent Source: 2002 Uganda HIV Service Provision Assessment

34. Illustrative Outcome Indicators • Percentage of people in the community who know about the VCT services • Proportion of people counseled and tested who report positive behavior change to avoid HIV infection/transmission • Proportion of people tested and found to be HIV+ who report positive or negative reactions from others

35. HIV testing among women in Zambia Source: 2001 Zambia DHS

36. HIV testing among men and women in Zambia Source: 2001 Zambia DHS

37. HIV testing experience among sentinel groups in Zambia Source: BSS, Zambia 2000

38. Multi-centre study Design of the VCT Efficacy Study • Multi-center randomized trial: 1995-1997 • Three sites: Kenya (N=1515), Tanzania (N= 1427) & Trinidad (N=1357) • Randomized to receive VCT (N=2152) or Health Information (N=2141) • Traced and interviewed at 6 & 12 months • Cross-over at 6 months so that the original Health Information group now had access to VCT

39. 40 30 20 10 0 HI VCT HI VCT Males Males Females Females Baseline 6 Months 12 Months Multi-centre study Unprotected intercourse with non-primary partners decreased significantly more among VCT participants percent

40. Multi-centre study: Unprotected intercourse with commercial sexual partners decreased significantly more among VCT participants 14 12 10 8 Baseline 6 months 6 4 2 0 HI males VCT males HI VCT Overall HI Overall females females VCT

41. HIV Program Impact

42. Incidence vs. Prevalence • Incidence = number of new infections in a time period susceptible population (HIV negative) in the time period • Prevalence = number of infected people at a given point in time total population (HIV negative & HIV positive) at that point in time

43. Prevalence, the faucet and sink… New HIV Infections Number of HIV infected people Deaths

44. Impacts Indicators • Prevent new HIV infections • Percentage of young people aged 15-24 who are HIV-infected (UNGASS and Millennium Development Goal) • Percentage of HIV-infected infants born to HIV-infected mothers (UNGASS) • Extend and improve life for those already infected with HIV • AIDS incidence and prevalence • Quality of life measures • Case fatality rate for HIV/AIDS • Proportion of mortality attributed to AIDS • General population life expectancy • Mitigate the social and economic impacts of the epidemic • Gross national product or other economic indicators • Infant and child mortality rates

45. Impacts: HIV sero-prevalence among 20-24 year old antenatal women in Uganda: 1990-2001 Source: STD/AIDS Control Programme, Ministry of Health, Uganda 2002

46. Impacts: HIV prevalence estimates based on ANC sentinel-site surveillance vs. estimates based on population surveys Sources: Boerma, et. al. 2003; Central Bureau of Statistics 2004; Republic of Kenya 2001

47. SAVVY - Sample Vital Registration with Verbal Autopsy • A package: • “Sample vital registration” conducted in surveillance communities • “Verbal autopsy” conducted as an interview with the family of the deceased to ascertain the likely cause of death • Purpose is • to provide measurement of vital events when alternative sources (vital registration) are not available or not complete • To provide information on the cause of death when death certificates may not be accurate • May be used to monitor AIDS related deaths, use of health services, etc.