Measuring Adherence in Microbicide Trials

1. Centre For International Health Research Measuring Adherence in Microbicide Trials Robert Pool WHO-ICMR-CONRAD-IPM Meeting on Regulatory Issues in Microbicide Research New Delhi October 28-31 2007

2. THE ISSUE • Testing the efficacy of microbicides requires accurate measurement of product use – adherence • This also contributes to our understanding of acceptability

3. deliberate accidental The problem • Direct observation is impossible • Self-reporting can be unreliable • The topic is sensitive • There are different kinds of non-adherence • There are different reasons for over/under-reporting • This has consequences for the accessibility of data and the methods used to collect it

4. How can we collect information on behaviour and adherence? Self-reporting • questionnaires • open in-depth interviews • self-assessment • diaries • ballot boxes • ACASI (audio computer assisted self interview) • “Respondent-independent” • Applicator counts • Biological markers • Chemical traces in the body or on the applicator • “Smart” applicators & rings

5. Advantages and disadvantages of self-reporting • Standardised • Easy to do and to analyse • Large numbers & generalisation QUESTIONNAIRES OPEN INTERVIEWS SELF-ASSESSMENT • Flexible • Open interpretation • Rapport, personal • Avoids embarrassment • Avoids desirability bias • Immediate (diaries) • Inflexible • Hidden interpretation • Embarrassment & desirability bias • Time lapse, memory • Difficult to analyse • Small numbers & limited generalisation • Embarrassment & desirability • Time lapse, memory • Inflexible • Impersonal • Complementary • Main criticism: self-reporting is unreliable per se. Evidence of respondents not being truthful

6. Advantages and disadvantages of “respondent-independent” methods • Easy for researchers • More objective SMART APPLICATORS APPLICATOR COUNTS BIOMARKERS • Objective • Accurate • Objective • Accurate: “the chemical in the body” • Microbicide chemicals not absorbed (?) • Whose vagina? • Trust • Acceptability • Whose vagina • Trust • Acceptability • Inconvenient for participants • Participants may not return applicators • Doesn’t record other uses of product • More objective, though still not perfect • Problematic assumptions

7. Using different methods in microbicide research - some examples • Pop Council Phase II study in S. Africa: compared interviews and ACASI • Carraguard trial assessed product use by using a dye that stains vaginal mucous on used applicators • Pop Council/HPRU are comparing ACASI & interviews, and comparing these to the dye-based technique and a technique for identifying semen in the vagina • IPM is developing a “smart” applicator that could measure insertion • CAPRISA trial: ancillary study using a biomarker (tenofovir disoproxil fumarate (TDF) levels in the genital tract) and comparing to in-depth interviews • MDP: Triangulation between applicator returns and self-reporting through questionnaires, coital diaries and in-depth interviews

8. Independent Self-report - + QUAL QUANT The best methods • THREE POSSITIONS: • We already know what the best method is and should use/develop • We don’t know but should find the best method by comparison • Different methods will always be needed to cover different facets and we should try to develop the best combination • This combination must be rigorous, practical and acceptable • And should combine: • QUAL and QUANT • Self-report and respondent-independent • Self-assessment and face-to-face

9. The Microbicides Development Programme • MDP 301 • Objective:To determine the efficacy and safety of two concentrations of “PRO 2000/5 Gel” compared to placebo in preventing vaginally acquired HIV infection • Six sites • 9,673 HIV negative women randomised to 3 arms, followed for 12 months (24 in Uganda)

10. Measuring adherence in MDP 301 • Regular clinic interview, including questions on sexual behaviour and gel use • Applicator returns, including used/unused at home or lost, compared to number given • Memory aid • Social science & triangulation with randomly selected sub-sample of 600 women

11. Interview • Comparison & probing inconsistencies Comparison Form IDI Nvivo T&T FEEDBACK TO TRIAL Summaries Summaries • Interview • Comparison with women Male partners IDI Nvivo T&T FGDs Nvivo Women, community T&T Summaries THE MDP TRIANGULATION PROCESS Women Applicator returns Clinic interview Coital Diary4 weeks before CRF

12. Inconsistency between questionnaire and triangulated data • Tendency toward under-reporting of numbers of sex acts, gel and condom use in the clinical interviews • Though there is also some over-reporting of gel and condom use • Reasons for discrepancies: • desirability effect (over-reporting) • participant forgetting • participant making a mistake • participant misunderstanding • interviewer mistake  Inaccurate reporting is largely unintentional

13. Less returned than reported More returned than reported # gel used reported in last week - # returned applicators/weekWeek 4: for woman who reported having sex in last week (n=4110) 59% agree to ± 1 applicator 79% agree to ± 2 applicators

14. Consistency between reported gel use in clinic questionnaire and triangulated dataset compared to used applicator returns

15. Reasons for over/under-reporting gel use • Confusion about when the week starts/ends • Problems matching dates of interviews and applicator returns OVER-REPORTING • Desirability bias • UNDER-REPORTING • Forgetting, mistakes, misunderstandings • Double use (a couple of women) • Inserting but no sex (reported at about 8% of visits) • Use of product to “practice” (common at one site) • Other uses of gel – “drying” and cleansing vagina (some evidence at some sites) • Other uses of gel – hair gel, skin cream, etc. (anecdotal, rumour) • Sharing (limited evidence) • Squeezing out but not using (some evidence, particularly at some sites, but not widespread). Problematic because deliberate

16. Reasons for non-adherence(475 women in 852 data sets)

17. Conclusion: reasons for continuing to use mixed methods, including qualitative methods, to study adherence • Biological or electronic techniques are probably most accurate for reporting actual product use, but they are unlikely to be entirely foolproof • And even if we have an effective biological or electronic technique, we still need to know and understand the reasons for non-adherence so that we can take measures to reduce it • Adherence is also closely linked to acceptability, and understanding non-adherence is a necessary part of understanding acceptability • So some form of mixed method design, including qualitative methods, will always be needed