Matthew Lamb mrl2013@columbia ICAP-M&E NY

Using routinely-collected data to estimate patient retention in care and loss to follow-up ICAP Methodology Webinar January 19, 2012 Matthew Lamb mrl2013@columbia.eduICAP-M&E NY

Upcoming methodology webinars February 9 Overview of ICAP Geographic Information System Resources Charon Gwynn, Yingfeng Wu, and Mark Becker Future methodology webinar ideas? email the methodology webinar coordinator, Bill Reidy: wr2205@columbia.edu

Using routinely-collected data to estimate patient retention in care and loss to follow-up ICAP Methodology Webinar January 19, 2012 Matthew Lamb mrl2013@columbia.eduICAP-M&E NY

Outline • Defining retention • Why is retention a useful outcome, compared to mortality? • How does loss to follow-up impact our measures of patient survival? • How can we use retention to assess patient and programmatic outcomes? • How can we measure retention using routinely-collected data? • Patient-level • ART cohort • Aggregate

Enrollment into HIV care and treatment clinic ART ineligible at enrollment (window) ART eligible at enrollment Already on ART Unknown eligibility at enrollment (window) LTF Transfer Transfer Death Becomes ART eligible LTF Transfer Death LTF Transfer LTF Death Transfer Death Initiates ART LTF Transfer Death Follow-up after ART initiation Follow-up after ART initiation

Working definitions • Retained • Known to be alive and engaged in care • Retained on ART • Known to be alive, engaged in care, and on ART • Retained in pre-ART care • Known to be alive and engaged in care, but not yet on ART • Known dead • Death known to clinic and documented • Transferred out • Patient transfer to another clinic known and documented • Lost to follow-up • Patient not known to be dead or transferred, treatment status and whereabouts unknown Non-retained = Known dead + Lost to follow-up

Non-retention combines “bad” outcomes of death and loss to follow-up Treatment interruption/stoppage Lack of monitoring Lack of services Death LTF Unmeasured death Non-retained Measured death Loss to follow-up results in underestimates of patient mortality

LTF influences our measures of survival (example) • Suppose you live in a universe where HIV clinics have perfect documentation, and all patients who enroll into HIV care attend every one of their scheduled visits and take all of their medication.

Measuring risk of death in a cohort with no LTF Incidence proportion of death: 4/20 (20%) Retention proportion: 16/20 (80%) 1 2 3 4 5 6 7 8 9 10 11 12 Time (months) sinceART initiation

LTF  underestimating the risk of death Incidence proportion of death: 4/20 (20%) Retention proportion: 16/20 (80%) Incidence proportion of death: 3/20 (15%) Retention proportion: 12/20 (60%) 1 2 3 4 5 6 7 8 9 10 11 12 Time (months) sinceART initiation

Introducing incidence rates • To account for unequal follow-up time in the presence of loss to follow-up, we measure incidence using incidence rates instead of incidence proportions Incidence rate Incidence proportion

Estimating mortality rates in the presence of LTF Incidence proportion of death: 3/20 (15%) Retention proportion: 12/20 (60%) Incidence proportion of death: 4/20 (20%) Retention proportion: 16/20 (80%) 12 5.5 12 12 7.75 12 12 12 12 12 12 12 12 9 12 12 12 9 12 12 223.25 person-months 12 2.25 12 12 7.75 12 12 12 12 12 9 12 4 9 12 8 12 9 6 12 199 person-months Incidence rate of death 3/199 pm 18.1 per 100 py Non-retention rate 8/199 pm 48.2 per 100 py Incidence rate of death 4/223.25 pm 21.5 per 100 py Non-retention rate is the same here Months of observation for each patient 1 2 3 4 5 6 7 8 9 10 11 12 Total person-time Time (months) sinceART initiation

Incidence rates give us closer estimates to “the truth” in populations with loss to follow-up

Using incidence proportions when follow-up time is unequal biases our interpretation of outcome occurrence 3m 6m 3m 9m 6m 12m 9m 12m Incidence proportion: 4/(4+10) = 4/ 14 = 29% Incidence rate: 4/((4*1m) + (10*12m)) = 4/124 person-months = 39 per 100 person-years Incidence proportion: 4/(4+10) = 4/ 14 = 29% Incidence rate: 4/((4*11m) + (10*12m)) = 4/164 person-months = 29 per 100 person-years Risk Ratio = 29%/29% = 1 Rate Ratio = 39/29 = 1.3

Comparing retention between clinics allows us to assess factors that may improve retention *Among patients initiating ART April 2008-March 2010. Loss to follow-up is defined as patients not known to have died or transferred without a visit in the last 6 months of data collection (ART). Patients LTF are censored 15 days after their last visit (ART).

Comparing retention between clinics allows us to assess factors that may improve retention Source: Lambdin et al. JAIDS.Volume 57(3), 1 July 2011, pp e33-e39

Transforming patient-level data into follow-up cohorts • Working example: Non-retention one year after ART initiation • Select study population and time period for enrollment and follow-up of patients • Define loss to follow-up • No visit in the last 6 months of data collection • Therefore need to extend follow-up time for 6 months after enrollment period • Define “zero time” • Calculate person-time for each patient • Calculate the incidence rate for the outcome of interest

Calculating retention from patient-level data 13 Non-retained within 1 year 153.5 person-months Non-retention rate = 13/153.5 person-months Non-retention rate = 102 per 100 person-years Q1 FU1 FU2 Q2 Q3 Q4 • Study population: patients initiating ART between Q1 and Q4 • Exclude patients initiating prior to Q1 • Outcome = non-retention (LTF or death) • Make sure all patients have sufficient opportunity to meet definition of LTF • Extend follow-up period to 6 months after Q4 • Start following patients from their ART start date until they become non-retained or the study • period ends • Calculate person-time • Calculate non-retention rates 7.5 4 4.5 7.5 12 7.5 12 12 7.5 12 5 7.5 7.5 7.5 7.5 2.5 9 7 6.5 7 Limit follow-up to 1 year

Slight diversion: what to do about transfers • Transfers prevent us from knowing the true retention status of the patient after they transfer. • We say that patients who transfer are “censored,” meaning that we do not have complete information on their retention status, had they remained at their initiating clinic • We allow transfers to contribute person-time to the denominator until their transfer date

Summary: calculating non-retention from patient-level data • Clearly define • Study population • Study period of enrollment • Follow-up time • Loss to follow-up • Outcome of interest • Zero time • Calculate each person’s follow-up time from zero time until reaching outcome of interest, censoring, or end of study • Calculate incidence rate

Patient-level data is awesome, but… • Not everyone has it • It requires some “work” to analyze Thankfully, there is other information that we routinely collect

12 month aggregate ART cohort data (URS) Quarter of ART initiation: May-July 2010 Number initiating ART in May-July 2010: 269 Number on ART July-Sep 2011 214 Proportion retained = 214/269 = 80%

12 month aggregate ART cohort data Q1 FU1 FU2 Q2 Q3 Q4 Count patients initiating within the same quarter 5 Count how many of these are on ART 11-16 months later 1 Proportion retained = 1/5 20%

Weighted average retention measures from aggregate cohort data Weighted-average 12-month retention incidence proportion

12 month aggregate ART cohort data: caveats • Transfers should be excluded from numerator and denominator • Does not separate non-retention into LTF, death • Can not directly calculate incidence rates • Only collected at 12 months • Can combine several cohort measures of retention to obtain a clinic-specific average 12-month retention

HIV care and treatment clinics routinely report the following • Cumulative number of patients on ART at the end of the previous quarter • (which equals the cumulative number of patients on ART at the beginning of the current quarter) • Number newly initiating ART during the quarter • Cumulative number of Deaths, Transfer, LTF, ART discontinuation through the end of the quarter • From these, we can calculate overall retention estimates

Aggregate retention 4 1 3 2 7 9 13 14 Q1 FU1 FU2 Q2 Q3 Q4 5 5 5 5

Aggregate retention Assume 3 months follow-up time for each individual on ART at beginning of the quarter Assume 1.5 months follow-up time for each individual initiating during the quarter Subtract 1.5 months follow-up time for each individual exiting during the quarter 4 1 3 2*3 + 5*1.5 = 13.5 pm 9*3 + 5*1.5 – 1*1.5 = 33 pm 7*3 + 5*1.5 – 3*1.5 = 24 pm 13*3 + 5*1.5 – 4*1.5 = 40.5 pm 13 2 7 14 9 Q1 Q2 Q3 Q4 5 5 5 5

Aggregate retention Overall one-year non-retention rate: 7/(13.5+24+33+40.5) 7 per 111 person-months 76 per 100 person-years 4 1 3 2*3 + 5*1.5 = 13.5 pm 9*3 + 5*1.5 – 1*1.5 = 33 pm 7*3 + 5*1.5 – 3*1.5 = 24 pm 13*3 + 5*1.5 – 4*1.5 = 40.5 pm 13 7 14 9 2 Q1 Q2 Q3 Q4 5 5 5 5

Aggregate retention: important caveats • Aggregate retention does not estimate a patient’s risk of being non-retained in one year • It is a clinic-level estimate of the flow of patients into and out of the clinic • We can use the same procedure to calculate aggregate LTF and mortality rates • Comparing aggregate retention rates between clinics is very useful

Summary • Loss to follow-up results in underestimates of survival • In situations with high loss to follow-up, non-retention provides us with a more conservative estimate of undesirable patient outcomes • Incidence rates are the preferred measure of non-retention incidence, but are not always available • Routinely-collected Patient-level, ART cohort, and cumulative aggregate data can be used to estimate non-retention incidence • Comparing non-retention between populations and clinics can help us identify areas that affect patient outcomes…

Using non-retention: CROI 2012 Abstracts • Lamb et al. Factors Associated With High Loss To Follow-Up Among sub-Saharan African Youth 15-24 Years of Age Enrolled in HIV Care • Elul et al. Six- and 12-month Non-retention Over Time among 5,690 Cohorts with 316,762 Patients Initiating Antiretroviral Therapy (ART) in 9 Countries in Sub-Saharan Africa • Mcnairy et al. Retention of HIV-infected Children on ART in ICAP-supported HIV Care and Treatment Programs

Matthew Lamb mrl2013@columbia ICAP-M&E NY