Population Size Estimation and coverage calculation for MARPs and MARA Dave Burrows, Director AIDS Projects Management G

1. Population Size Estimation and coverage calculation for MARPs and MARA Dave Burrows, Director AIDS Projects Management Group

2. “Coverage” • Perhaps the most mis-named, misused and least understood concept in HIV work • Coverage means whatever the person using it chooses to mean • Most common use: % of people ever reached (or reached in 1 year) with an intervention: this is an utterly useless statistic • If 100% of IDUs are reached once with education or a new needle & syringe, or if MSM or SW are reached once with education or a condom, it will have virtually no impact on a HIV epidemic

3. 1st problem is PSE • PSE increasingly needed for national HIV plans & GF projects: if do not know size of population, how can we estimate coverage after 5 or 6 years of programs + plan scale-up? • Whatever definition of coverage is used, it almost always begins with “% of X population (IDUs, MARA, etc)” • X population is the denominator for all further calculations related to coverage and its constituent parts: reach, regularity of reach, breadth of services, quality • To find X population, population size estimation (PSE) methods are used

4. Why is PSE so difficult? • Some populations difficult to count, especially hidden, stigmatised • Usual epidemiological methods such as national household or schools surveys usually do not work • Definition problems: eg, IDU has “ever injected”, “injected in past month”, injected in past year”? • Even more difficult for MARA and MARY as most epidemiological statistics & estimates are not disaggregated by age (or sex)

5. PSE Methods • Variety of methods available, but most include: • Consensus/ Delphi • Multiplier methods • Other potential methods • RDS: Respondent Driven Sampling • Social networks

6. Consensus/ Delphi • Asks key informants to agree on number of people in X population • Can be done at: • National level • All levels from local to national • Local to national seems to generate most accurate numbers • Should be triangulated with other methods

7. Multiplier methods • Recommended by UNAIDS for population size estimation, eg for reporting on UNGASS IDU indicator • Uses existing data source with survey data • Benchmark: Reliable, regularly collected data: IDUs accessing health services, drug treatment, overdose deaths • Multiplier: Survey of as broad a sample as possible (eg not just from treatment centres)

8. Multiplier formula X (population) = multiplier x benchmarkExample: 1000 IDUs entered drug treatment in 2007 (benchmark) • 10% of IDUs surveyed said they entered drug treatment in 2007 (multiplier) • X = 1000 x 10/100 (= 10) • X = 10,000 IDUs

9. Triangulation • Single multiplier exercises tend to be inaccurate • UNAIDS recommends using 3 at least separate processes, and averaging results to find a mean estimate: • Eg: Different processes may give 10,000; 8000; 11,000. Mean = 9670

10. RDS/ Social networks • RDS uses snowball sampling in specific methods to attempt to achieve highly representative sampling: was not developed as a PSE method! • Mexico AIDS Conference: meta-analysis of 200 RDS papers found no evidence that RDS is useful in PSE • Promoted by many agencies with little/no evidence of accuracy; costly, time-consuming • Social networks PSE: new method, currently promoted by UNAIDS PSE workshops. May have value but requires evaluation, and to date apears costly and time-consuming

11. APMG Tajikistan project • In Tajikistan, APMG is finalising a 5-month process for UNDP (GF PR) to: • Estimate national populations of IDUs and SW • Risk behaviour of IDUs & SW in 5 sites • Capacity of implementation agencies to scale up service delivery to IDUs & SW in these sites • In addition, APMG is trying to tie this process to ongoing PSE for IDUs and SW as numbers change (especially locally as IDUs & SW are chased from 1 area by police activity or attracted to an area by availability of drugs or SW clients)

12. Tajikistan PSE methods • Expert estimation (Delphi) at rayon level, combined at oblast and national levels • Survey for risk assessment included multiplier question re use of narcological services in 2008 • Benchmark: narcological statistics in 2008 • Results presented to national consensus meeting (September 21) to agree final numbers

13. Lessons learned from Tajikistan • Biggest error was carrying out risk assessment and PSE simultaneously: much larger sample sizes needed for risk assessment sampling meant expert estimation could not be carried out in all rayons nationally • PSE can be relatively cheap and quick if done as a stand-alone activity

14. Lessons learned from Tajikistan • Rayon-level estimation requires national/ oblast level staff to assist local officials to come to consensus • Time should be included to allow rayon estimates to be considered at oblast level, then national meeting based on oblast estimates • If this process used, could set up 6-monthly monitoring by asking rayons to consider increases/ decreases over the previous 6 months. Requires oblast/ national compilation

15. Tajikistan lessons re MARA • PSE of MARA in Tajikistan could be accomplished using the same methods (with lessons learned) BUT • Definition required • Definition would need to be agreed with officials from various departments • Definition to be explained at rayon level

16. Armenia • PSE of MARA in Armenia carried out by APMG and MoH staff working on GF RCC proposal (2008) • Had already estimated IDUs, MSM, SW, migrants, uniformed personnel • MoH wanted to include specific programs for MARA but this required a statement about projected coverage after 6 years • To calculate coverage figure, PSE was needed for MARA

17. Armenia Methods • No time available for MARA PSE study • Estimate figure was calculated as 5% of all adolescents in Armenia on the basis of household and school surveys that showed at least 5% of adolescents engaging in risky sexual behaviour or illicit (not necessarily injecting) drug use • Population estimate was used in RCC proposal with a note that a full PSE would be carried out as part of the grant activities • RCC was approved and will begin in late 09

18. Macedonia • PSE in Macedonia will be carried out by National Public Health Institute (NPHI) for MoH (GF PR) • NPHI has decided to combine PSE with risk behaviour survey and to use RDS (against our advice) • APMG’s role will be to examine all documents (methods, instruments, sampling frames, data analysis & reports) to recommend corrections • From this process, we will be able to learn lessons about use of RDS for PSE (probably by end 09)

19. Some further thoughts on Coverage • APMG accepts WHO Universal Access definition: % of those who need an intervention who receive that intervention • APMG sees 3 aspects: • Reach, including regularity of reach. What % of the total population participate? Is this a sufficient proportion to prevent/ reverse/ treat the epidemic? • Breadth: Spectrum of Services. Are interventions able to prevent/ reverse/ treat the epidemic? • Quality: Are interventions sufficiently attractive and effective to meet their objectives?

20. Coverage Calculation • APMG accepts WHO Universal Access definition: % of those who need an intervention who receive that intervention • E.g., for needle-syringe programs, it appears that a percentage of IDUs in a specified area need to access NSP of adequate quality ON A REGULAR BASIS to prevent/ reverse a HIV epidemic among IDUs. • WHO, UNODC and UNAIDS state that the % of IDUs who have been reached by NSP regularly (at least monthly for past 12 months) should be considered as: Low coverage: <20% Medium coverage : >20– <60% High coverage : >60%

21. Coverage questions • WHO, UNODC and UNAIDS Target Setting Guide for IDUs include: • Proportion of IDUs regularly reached by NSP • Number of pharmacies/ 1000 IDUs • NSP sites/ 1000 IDUs • Number of syringes distributed per IDU per year • % of IDUs who have been reached by NSP regularly (at least monthly for past 12 months) • % of IDUs who have been reached by NSP in the past month

22. Coverage questions 2 • Similar questions on proportion of IDUs in substitution treatment • Similar questions on proportion of IDUs in other drug dependence treatment • Similar questions on proportion of IDUs participating in VCT and know their results • Ratio of HIV+ IDUs receiving ART to non-IDU HIV+ receiving ART (relative to proportions of HIV+ population) • Questions on TB, hepatitis C, etc

24. Quality • Generally, view is that quality should be measured by adherence to guidelines, e.g. target setting guide asks: • Percentage of NSP sites adhering to WHO guidelines on NSP • Percentage of NSP sites adhering to UNAIDS best practice recommendations for HIV prevention among IDUs • Percentage of occasions when clients access an NSP and receive IEC • Percentage of occasions when clients access an NSP and receive condoms • In Russia, APMG is helping Russian Harm Reduction Network to develop NSP quality measurement and improvement processes based on the WHO/ UNAIDS/ UNODC Guide to Starting and Managing NSPs • Manual plus instruments should be available in English & Russian early 2010

25. Coverage for other MARPs • Similar processes now under way for MSM: • APMG working with Amfar, UNDP & WHO on coverage calculation, targets & breadth of services • WHO working on similar processes re SW • MARA and MARY not yet really included in these global processes