Improving response rates to postal questionnaires in clinical trials

1. Improving response rates to postal questionnaires in clinical trials Rachel Nakash PhD, BSc(Hons), MCSP

2. Introduction • Patient based outcome measures – assess various aspects of health from patient’s perspective. • Usually take format of questionnaire • Telephone, face to face, post • Post: least expensive, prevents interviewer bias, patient can respond in own time • Disadvantages: NON-RESPONSE

3. Non-response in a clinical trial • Reduces effective sample size, introduces bias • What is acceptable loss to follow up? • Sackett et al (1997) ‘five and twenty’ rule of thumb • Some journals refusing to publish trials with losses > 20% • Obtaining losses to follow up < 20% in a clinical trial takes considerable effort

4. Previous research • Methods of improving response rates to postal questionnaires – mostly SURVEY research • No connection to health care • Most relevant Cochrane review: Edwards et al (2003): incentives, ‘user friendly’ and shorter questionnaires improve response – but... only 20% of included trials had a health care connection • Clinical trial participants ? different to survey participants – lack of literature

5. PhD aims and objectives • 1. Systematic literature review – refinement of previous reviews. Focussed just on health care literature • 2. Randomised controlled trial – to test a method of improving response within the context of an existing clinical trial • 3. Qualitative study of clinical trial participants • 4. Analysis of characteristics of responders and non-responders

6. Setting for the research • Collaborative Ankle Support Trial – CAST • A multi centre RCT funded by the NHS HTA • Designed to evaluate clinical and cost effectiveness of 3 methods of mechanical support compared to tubigrip in severe ankle sprain • Outcomes assessed at 4 weeks, 12 weeks and 9 months following injury using postal questionnaires

7. CAST pilot phase was used to: • Assess response rates • Clarify follow up procedures • Identify aspects of response to subject to deeper investigation • 24 patients recruited onto CAST pilot

8. A follow up protocol was needed to chase response • Developed based on available lit and time and resource constraints of CAST • Series of steps involving telephone contacts and repeat mailing of questionnaire • Time consuming (?soul destroying!) • Response categories developed – summary of how much prompting was required by each patient at each time point

10. Response categories grouped into response ‘types’

11. Response types at each time point

12. 1. Systematic Literature Review • Cannot assume methods of improving response in survey research will be successful in clinical trial setting • Refinement of previous reviews • Limited to ‘health care studies’ on ‘patient populations’ • More useful and relevant for health researchers using postal questionnaires

13. Results: • 13 trials identified evaluating 5 different methods of improving response • Questionnaire length, incentives, question order, reminder strategies, information brochure • Similar methods of improving response grouped for comparison • Random effects model meta-analysis conducted

14. Conclusions • Intense follow up strategies and shorter questionnaires improve response rates • The evidence is, however, limited

15. 2. Randomised Controlled Trial • Method of improving response nested into CAST • ‘Trial Calendar’ – a tool to prompt questionnaire return • Research question: Does the trial calendar improve response rates and reduce the need for prompting

16. 296 CAST participants randomly allocated to receive a trial calendar or not • Effect of calendar assessed using chi-squared test at each follow up point • No sig effect on response at any follow up point • Logistic regression performed to account for effect of covariates (eg age, sex) • Still no effect of trial calendar at any time point (except females with calendar sig better at responding at 9 months – data trawling!)

17. Conclusions: • Trial calendar not an effective tool for improving response rates in a clinical trial • Follow contact and prompting needs to be high profile and timely – trial calendar too subtle • Extra cost involved in implementing a tool such as a trial calendar not justified

18. 3. Qualitative Study • To gain the participant’s perspective of response issues in clinical trials • Semi-structured interviews of a purposive sample of CAST participants • 22 participants interviewed – responders and non-responders • Framework Analysis used to analyse data (similar to thematic content analysis but deeper).

19. Common themes: • Reasons for response: • Personal relevance • Agreed to take part – Compliance/trial understanding • Obliged to respond – Good service/reciprocation • Altruism • Important project • Reasons for non-response • Internal – due to aspects of trial • External – beyond control of trial • Personal blame – Forgetful, lazy, disorganised… • Life events • (Full recovery, treatment preference)

20. Typologies: • ‘Happy non-responders’ • ‘Unhappy non-responders’ • Serial non-responders – once a non-responder, always a non-responder • Conclusions: • Recruitment phase very important in appealing to psychological processes of response and ensuring participant fully understands trial follow up • ? Convert happy non-responders

21. 4. Characteristics of responders and non-responders • ? Any predictive characteristics of responders and non-responders • Survey researchers have attempted to establish common personal characteristics of responders and non-responders • Advantage of a clinical trial is the detailed background information collected at baseline • Despite this there is a lack of work looking specifically at CT participants

22. 555 participants recruited onto CAST (not inc pilot) • Either responder or non-responder at each follow up point • This info combined with CAST database background info to allow for comparisons between responders and non-responders in terms of certain socio-demo characteristics • Age, sex, CAST treatment, employment type, education level, recovery • Analysis: chi-squared, logistic regression

23. Conclusions: • Age the only variable to sig effect response – older people respond better • Females generally better at responding • Better recovery leads to lower likelihood of response • Active refusals to cooperate very low compared to surveys (refusal in a trial usually occurs at recruitment stage) • Non-contacts steadily increase at each follow up (incorrect contact details, contact details change, house movers)

24. Overall conclusions and recommendations • Consider carefully and a priori follow up protocols to chase reluctant responders • Keep non-response as a result of non-contact to minimum – get as much contact info as possible • No evidence incentives effective so don’t bother • Careful wording of written/verbal instructions to appeal to theoretical issues of response and understanding of follow up commitment • Exclude potential non contacts at baseline

25. Thank you for listening!

26. References • Lamb SE, Marsh JL, Hutton JL, Nakash R, Cooke MW; Mechanical supports for acute, severe ankle sprain: a pragmatic, multicentre, randomised controlled trial. Lancet 2009; 373(9663): 575-81. • Sackett D, Richardson W, Rosenberg W, Haynes R. Evidence-based medicine: how to practice and teach EBM. New York: Churchill Livingstone 1997.

27. Groves R, Singer E, Corning A. Leverage-Saliency theory of survey participation. Public Opinion Quarterly. 2000;64:299-308. • Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al. Methods to increase response rates to postal questionnaires. The Cochrane Database of Methodological Reviews. 2003(Issue 4):Art. No.: MR000008. DOI: