Surveys and Surveillance

1. Surveys and Surveillance Regional Workshop on the Monitoring and Evaluation of HIV/AIDS Programs February 14 – 24, 2011 New Delhi, India

2. Learning Objectives • Describe applications of different survey approaches used for monitoring and evaluation • Identify approaches to sampling populations for monitoring target populations and discuss their strengths and limitations • Identify issues related to planning and managing survey and surveillance activities

3. Types of Surveys • Facility Surveys • Household Surveys (i.e. AIDS Indicator Surveys (AIS), DHS) • Targeted Surveys (i.e. BSS, PLACE)

4. Role of Facility Surveys in M&E • Provide program level data on inputs, outputs and outcomes • Link to population level data to assess impact of program outputs in population level outcomes • Conduct every two to four years

5. Components of an HIV/AIDS Program Population level Program level Outcomes Inputs Processes Outputs Services Availability % facilities offering Service % communities with outreach # trained staff Utilization # new clients # treated Resources Staff Drugs, Supplies Equipment Intermediate Behavior Long-term Health Status Functions, Activities Training Outreach and Education Logistics Facility Surveys

6. Information Provided • Readiness to provide services (inventory) • Infrastructure, staffing, hours of operation, fees • Health worker knowledge • Provider interviews • Quality of Care • Client-provider observation • Client satisfaction • Exit interviews

7. HIV Service Provision Assessment (HSPA) • Measures the conditions and capacity of health facilities to provide HIV-related care and treatment services • inpatient and outpatient care and treatment • HIV Counseling and testing • PMTCT services • laboratory services • linkages to other HIV/AIDS-related services for patients and their families (i.e. home-based care, support groups, etc.), • availability of guidelines and protocols for HIV/AIDS-related care and support services • the availability of medicines and supplies • facility conditions • referral linkages between services.

8. Source: Barbados Caribbean Region HIV and AIDS Service Provision Assessment Survey 2005. MEASURE Evaluation 2007

9. Examples of Facility Survey Indicators • Percentage of health facilities that have the capacity and conditions to provide advanced HIV/AIDS clinical and psychosocial support services, including providing and monitoring antiretroviral combination therapy • Percentage of facilities that provide comprehensive care referrals HIV/AIDS care and support services • The number of public, missionary and workplace venues (FP and PHC clinics, ANC/MCH, and maternity hospitals) offering the minimum package of services for the prevention of HIV infection in infants and young children in the preceding 12 months Source: National AIDS Programmes: A Guide to Monitoring and Evaluating HIV/AIDS Care and Support. UNAIDS 2005. National Guide to Monitoring and Evaluating Programmes for the Prevention oh HIV in Infants and Young Children. UNAIDS/WHO 2004

10. Strengths of Facility Surveys • Can cover both public and private health facilities • More detailed information than is typically available in routine systems • Can be tailored to specific program needs • Timing can coincide with program implementation • Can combine with population survey for outcome monitoring and impact evaluation • Quality control may be easier than in routine systems

11. Limitations of Facility Surveys • Survey sampling design and analysis may be complex • Expensive, time-consuming • Stand-alone – sustainability concerns • Less connected to ongoing program decision-making • Information rapidly outdated, unless repeated – not available regularly • Coverage/sample size constraints

12. Role of Household Surveys in M&E • Provide data on indicators of program outputs, outcomes, and impact • Service utilization and coverage • Knowledge and attitudes • Health-related behaviors • Health status • Conducted every 2-5 years

13. AIDS Indicator Survey • Nationally representative household survey for HIV/AIDS • Standardized questionnaire that collects data on global indicators • Collects information on • Use of HIV related services (VCT, STI, etc) • Knowledge of HIV • HIV-related risk behaviors • HIV status

14. Components of an HIV/AIDS Program Population level Program level Outcomes Inputs Processes Outputs Services Availability % facilities offering Service % communities with outreach # trained staff Utilization # new clients # treated Resources Staff Drugs, Supplies Equipment Intermediate Behavior Long-term Health Status Functions, Activities Training Outreach and Education Logistics Population Surveys

15. Examples of Household Survey Indicators • Percentage of young women and men15–24 who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission • Percentage of women and men aged 15-49 who received an HIV test in the last 12 months and who know their results • Percentage of adults aged 15–49 who had more than one sexual partner in the past 12 months reporting the use of a condom during their last sexual intercourse Source: Monitoring the Declaration of Commitment on HIV/AIDS. Guidelines on Construction of Core Indicators, 2010 Reporting. UNAIDS 2009

16. Source: Cambodia DHS 2005

17. Strengths of household surveys • Representative of the general population • Wide range of outcome level indicators can be collected • Can provide estimates of program coverage • Well-tested instruments that provide internationally agreed upon indicators; often provide high quality data

18. Limitations of household surveys • Coverage; national versus sub-national – not suitable for district-level estimates • Frequency; typically only conducted every 2-5 years. • Cannot detect small changes or changes over short periods of time without large sample sizes (expensive) • Not suitable for some types of information (e.g. retrospective attitudes – recall bias) • Do not capture populations not living in households • Under estimate socially unacceptable behaviors

19. Role of Population and Target Group Surveys in M&E • Provide information on populations that are specifically targeted by interventions • Provide data on indicators of program outputs, outcomes, and impact • Service utilization and coverage • Knowledge and attitudes • Health-related behaviors • Health status • Conducted every 2-3 years

20. Sampling approaches for hard to reach populations • Network-based • i.e. Snowball sampling, Respondent Driven Sampling (RDS) • Venue-based • i.e. BSS with Time Location Sampling (TLS), PLACE • Institution-based • i.e. probability samples in prisons, drug treatment centers Source: A framework for monitoring and evaluating HIV prevention programmes for most-at-risk populations. UNAIDS 2007

21. Population Surveys: BSS • Track behaviors in key population sub-groups which are critical to reducing the spread of HIV • Sex workers • Intravenous drug users • Men who have sex with men • Other high-risk populations (mobile pops, truckers, etc) • May include biomarkers for assessment of disease status (STI, HIV) • Should include questions on exposure to/use of interventions for coverage estimates

22. Condom use with clients among FSWs is increasing steadily over time, terai highway districts, Nepal,1998 - 2002 Source: Family Healthy International, Bangkok

23. Strengths of BSS for M&E • Better than household surveys at collecting information on behaviors that are relatively rare in the general population • More valid estimates of behaviors that are highly stigmatized • Captures populations not in households

24. Examples of BSS Indicators • Percentage of most-at-risk populations (IDU, MSM, FSW) that have received an HIV test in the last 12 months and who know their results • Percentage of most-at-risk populations who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission • Percentage of injecting drug users reporting the use of sterile injecting equipment the last time they injected Source: Monitoring the Declaration of Commitment on HIV/AIDS. Guidelines on Construction of Core Indicators. 2008 Reporting. UNAIDS 2007

25. Limitations of BSS for M&E • Can be difficult to define risk groups • Stigma / Self-presentation bias • Dynamic and multiple memberships in risk groups may not be captured • Without stable denominator data, difficult to interpret changes

26. Surveillance • HIV sentinel surveillance systems (HSS) • Populations at risk of HIV infection are tested for HIV on a regular basis, usually annually • Integrated Behavioural Biological Surveillance Systems (iBBS) • Behavioural surveys that include HIV testing, usually conducted every 2-3 years Source: A framework for monitoring and evaluating HIV prevention programmes for most-at-risk populations. UNAIDS 2007

27. Why do we do surveillance (including BSS)? • To gain understanding about how HIV is spreading within countries • Who is infected with HIV where? (HIV surveillance) • Who is being exposed to HIV where?(Behavioral surveillance) • What is the source of new infections and how is it changing over time? (case reports, surveillance, models) • Using the information to guide the response (i.e. what interventions are needed when and where) 3. Using the information to inform policy and advocacy strategies (planning for maximum impact) 4. Using the information to monitor & evaluate progress in the interventions Note: Priorities may vary in different settings Source: Family Health International

28. Role of Surveillance in M&E • Provides a way to assess the collective response to HIV • Information may not be specific to, or attributable to, a specific program

29. Planning and Managing Surveys • What is the appropriate sampling approach for the population of interest? • Are instruments used in planned surveys adequate to collect the indicators as defined in the national M&E framework? • Are planned surveys/surveillance activities being conducted in the same geographic areas where interventions are being conducted? • Are the same populations that are being targeted for interventions included in planned surveys? Are they defined in the same way?

30. Planning and Managing Surveys (cont.) • Are standardized protocols developed including checks to ensure data quality? • What forms of data analysis and presentation are needed? • What mechanisms need to be put into place to allow for data sharing among different involved stakeholders? • If data are to be aggregated across survey populations, are population definitions, instruments and sampling comparable?

31. Group Work (1): Population Size Estimations Discussion Questions: • What methods and approaches have worked/not worked as a way to obtain data on the size of the population that your program targets? • If you are working with a hard to reach population (i.e. IDU, FSW, mobile populations), what are the challenges and proven approaches for reaching and collecting these data? • What are the gaps in data availability? • Are there issues with data quality? • How can data on population size estimates be used for your program? Task: • Include in the M&E plan that you are developing plans for filling gaps in data and/or plans for improving data quality in regards to population size estimations. 

32. Group Work (2): Biological and Behavioural Surveillance Discussion Questions: • Are sufficient trend data on HIV prevalence and risk behaviour available for the population targeted by your program? • Are these data collected in the same populations and geographic areas where programs are targeted? • If you are working with a hard to reach population (i.e. IDU, FSW, mobile populations), what are the challenges and proven approaches for reaching and collecting these data? • What are the gaps in data availability? • Are there issues with data quality? • How can data from surveys and surveillance be used for your program? Task: • Include in the M&E plan that you are developing plans for filling gaps in survey and surveillance data and/or plans for improving the quality of survey and surveillance data.

33. MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. Government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.