1 / 28

“Maximising Return from Cohorts” Project: Prevention of Attrition Findings

“Maximising Return from Cohorts” Project: Prevention of Attrition Findings. Cara Booker, Ph.D. MRC SPHSU October 12, 2009. Overview. Systematic Review Survey of Retention Among MRC Cohort Studies Preliminary Findings Future Analyses. Systematic Review. Objectives

burt
Download Presentation

“Maximising Return from Cohorts” Project: Prevention of Attrition Findings

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. “Maximising Return from Cohorts” Project: Prevention of Attrition Findings Cara Booker, Ph.D. MRC SPHSU October 12, 2009

  2. Overview • Systematic Review • Survey of Retention Among MRC Cohort Studies • Preliminary Findings • Future Analyses

  3. Systematic Review • Objectives • To determine the effectiveness of retention strategies in improving response rates in prospective population-based cohort studies. • To identify possible characteristics (i.e. locality, age, etc.) of studies that may have affected the retention of cohort members.

  4. Population-based cohort study: “Any well-defined population defined by geographic boundaries, membership or occupation”.Szklo M. Population-based cohort studies. Epidemiologic Reviews. 1998;20(1):81-90. Exclusion Criteria: Clinical or non-clinical trials Non-population-based cohort studies Cohorts with record linkage as the only method of follow-up Studies which only looked at effectiveness of tracking methods Definition, Inclusion & Exclusion Criteria • Inclusion Criteria: • At least one wave of follow-up data collection in which the participant was personally contacted by the study • Retention rates were reported • Some description of the retention methods was available

  5. Review Methods • Electronic Database Search • 13 databases searched • 5 search terms • Manual searches of bibliographies • Internet and website searches for technical reports and internal documents • Contact with Principal Investigators and other study personnel for access to unpublished manuscripts and grey literature

  6. Total papers found from search (n=17,210) Excluded: Trials & non-population-based cohort studies (n=12,596) Excluded: Recruitment (n=3,701) Potentially relevant prospective population-based cohort studies (n=913) Excluded (n=623) No information on strategies (n=577) No information on follow-up (x=46) Papers retrieved for further evaluation (n=290) Working Papers /Handbooks/Users Guide (n=5) Unpublished papers (n=1) Book chapter (n=1) Excluded from review (n=265) No evaluation (n=245) Reviews (n=13) Tracing methods only (n=7) 28 Studies in 32 Papers Flowchart

  7. Results • 28 Studies were identified • Countries • 15 in the US • 5 in the UK • 3 in Canada • 2 in Sweden • 1 in Norway, Spain & Australia • 11 were randomized trials • Response rates ranges from 34% - 98%

  8. Retention Strategies by Data Collection Method

  9. Incentives • Randomized Trials (n=8) • Incentives associated with increased response rates • Average increase ranged from 4% to 27% • Type of incentive (i.e. gift, money, cash card) did not appear to have an effect on response rate • Use of cash cards and percentage of checks cashed varied • Previous round responders had higher rates than non-responders • Non-randomized studies (n=2) • Comparing monetary to gifts, monetary had increased response • Non-monetary and informational incentives compared against each other produced no difference in response • Greater value of incentive great response, less money spent overall

  10. Reminder Letters/Calls • Randomized Trials (n=1) • Higher response among participants with second reminder of phone call • Non-randomized studies (n=8) • Reminder letters increased response • However compared to a second questionnaire there was lower response from the reminder letters • Overall, use of multiple reminders or methods of data collection increased response • Average initial response rate was 61% • The average increase of response rate via • Reminder letters was 17% • Reminder calls was 11%

  11. Multiple Posting/Calls/Visits • 8 studies posted multiple questionnaires, 1 had multiple visits and 2 with multiple calls • Posting additional questionnaires increased response by an average of 15% • Costs increase with subsequent posting • Multiple visits to schools increased response by 34% • More call attempts appeared to increase response rates

  12. Alternative Data Collection Method • 10 studies offered alternative methods • Postal studies that offered telephone interviews increased response by an average of 5% • Face-to-face interviews which offered alternatives increased response by 25% • Two studies started in a central location, one started with clinic visits • Telephone studies that sent postal questionnaires increased response by 1% • Costs are higher for postal vs. in-school questionnaires (costs of mail & project coordinator) and telephone interviews are more expensive than post (due to personnel costs, and tracing costs)

  13. Other • 2 Randomized Trials • Length of questionnaire • Received higher response with short form of questionnaire (not significant) • Postal method • Randomized type of envelope and certified mail vs. UPS • Certified mail had better response • Handwritten envelopes had better response than other types of envelopes

  14. Study Characteristics • Different designs, sample sizes, reporting etc make it difficult to address our secondary aims. Broad pattern of: • Lowest response rates in US-based studies • Higher response rates in studies started in the 1980s and 1990s • In general, participants who were ethnic minorities, younger and of lower SES had lower response regardless of retention method • Higher education associated with earlier response • Postal studies with males only had higher response rates than mixed and female only studies • In mixed gender studies females had higher response rates in 8 of 11 studies. • Proportion of males increased with increasing contacts

  15. Conclusions • Incentives were the only strategy that had a clear and positive association with response rates • Multiple mailings of questionnaires and reminder letters also appear to increase response • Alternative data collection methods had minimal increases for postal questionnaire studies, but large increases for in-person interview studies • Use of multiple strategies increases overall response rate

  16. Discussion/Limitations • Very few cohort studies with analyses of retention methods • Reporting of attrition is not standardized • Often have some mention in various manuscripts detailing characteristics of attriters, which may vary by wave of data collection • More often don’t have technical reports, or detailed manuscript about the strategies used the characteristics of the attriters across the study • Calculation of response rates is also difficult due to eligibility criteria, definition of denominator and booster samples

  17. Recommendations • Pilot studies or sub-samples to evaluate retention strategies • Focus more expensive strategies to non-responders • Most initial responders will do so without the need for incentives or other more vigorous reminders • Explore effectiveness and ethical issues associated with internet searches and use of social networking sites in relation to participation not just for tracking

  18. Survey of Retention Among MRC Cohort Studies • Objectives • To examine what retention methods are associated with reductions in attrition • To explore what other study design features, if any, influence minimisation of attrition • Sample • Drawn from studies which are housed in one of the 12 MRC Population Health Research Network (PHSRN) units • Thirty-eight studies were identified

  19. Data collection Conducted between December 2007 and August 2008 Reminder methods 1 Reminder telephone call 3 Reminder emails Questionnaire Contents General study design issues Recruitment Consent issues Compensation/Incentives/Reimbursement Tracking of study members Attrition Methods of Survey

  20. Retention, Tracking, Tracing... • Retention Strategies • Used to increase response at time of data collection • Tracking/Tracing Strategies • Used to find participants between methods of data collection • Ambiguous Strategies • Often used between data collection but often used to increase loyalty & not specifically to track/trace

  21. General Study Design • 24 studies responded (75% response rate) • 7 were dropped from analyses • One study had two distinct samples, increasing analysed studies to 25 • 4 settings • Data presented here is from 18 studies

  22. Data Collection • How studies collected data • Face-to-face: Over 90% • Post: 70% • Telephone: 40% • Internet: 10% • Studies collected between four and eight different types of data

  23. Retention Strategies • 6 studies allocated funding for retention • 10 studies offered incentives • 1 study evaluated use of incentives • Unconditional gift voucher found to be the most successful • Retention Methods • 50% of the studies used 2-6 different retention methods • Most commonly used • Email, newsletters, newspapers/magazines • GPs/CCs, schools • Key Leaders/Gatekeepers, administrative/supervisory bodies, parent assessments

  24. Tracking Methods • Tracking/Tracking Methods • Minimum number of methods used was 1 & 14 was maximum • Most used: • Change of address cards, GP records, routinely registered events, stable address of close relative • Ambiguous methods • 80% provided findings • 40% provided holiday cards & 20% provided birthday cards

  25. Strategies: Numbers, Barriers & Successes • Successful Retention Strategies • Family/friend contact, change of address card and telephone/directory assistance • Barriers to retention • Residential mobility, disinterest in issues covered by study, incorrect address and people too busy to join study.

  26. Further Analyses • Analyze the impact of retention methods and different data collection methods on retention • What retention methods are the most significant in reducing attrition? • What is the impact on attrition when more invasive data collection methods are used (i.e. Vene-puncture, tissue collection, etc.) • Use a multi-level modelling approach • Waves nested within studies • Wave variables include: retention methods, data collection methods • Study level variables include: setting, average age of sample at baseline, gender of sample, year study started

  27. Future Analytic Methods • Multivariate modelling • Combine methods into larger categories • i.e. Radio + Internet = Media • Examine the effects of different sample population characteristics on subsequent retention rates • Data collection types • Age • Gender • Study setting

  28. Thank you!

More Related