Improving Response Rates Lessons from Physician Surveys

1. Improving Response RatesLessons from Physician Surveys PMRS Ottawa Chapter February 26, 2004

2. Presentation Overview • Survey Response Rates: The state of the art • Particularities of Physician Surveys • Response rate boosting tactics: What works and what does not. • On the use of Monetary Incentives in Physician and Consumer Surveys • Q & A period

3. Current response rates • Academic Surveys published between 1961 and 1977: 71% • Academic surveys published in 1991: 54% • Academic surveys published between 1986 and 1995, sample size over 1,000 respondents: 52% • Commercial/marketing physician surveys (2002): 20% • RETICULUM surveys: 12% to 66%

4. Current response rates • Surveys of executives, published in 1991: 21% • PMRS Members surveys: 15.7%(1997);11.3% (2000); • One-time telephone surveys: 16% (1997); 13% (2002)

5. Physician Surveys Particularities • More homogenous populations • Highly-regulated professionals Better sampling frames Better record-keeping

6. Physician Surveys Particularities • Highly-solicited respondents • Highly-educated respondents • ‘Well-connected’ respondents

7. Physician Surveys Particularities • Surrounded by ‘tough’ gatekeepers • ‘Addicted’ to monetary incentives

8. Tactics that boost response by 50% or more • Monetary Incentives • Multiple contacts & multiple contact modes

9. Monetary Incentives • (Gallagher, 2001) 1st contact by mail, no incentive: 11% 2nd contact by phone, no incentive: 22% (cumul) 3rd contact by courier, $20 incentive: 57% (cumul) • (Malin, 2000) 1st mailing, no incentive: 17% 2nd mailing, no incentive: 13% 3rd mailing, $50 incentive: 66% Cumulative response rate: 76%

10. Multiple Contacts & Contact Modes • Typical response rates after multiple mailings: 1st mailing: X% 2nd mailing: X/2% 3rd mailing: X/4% • (CDC, 1997): 1st contact by First Class mail: 60% 2nd contact by Fedex: 72% (cumul) 3rd contact by phone: 96% (cumul)

11. Tactics that boost response by a few % points • Pre-notification by phone • Personalization • Advertising the survey • Choice of sponsors • Shortening the questionnaire • Instituting a draw

12. Pre-notification by phone • (Osborn, 1996) No Pre-notification: 64% Pre-notification: 77% • (Ward, 1994) No Pre-notification: 69% Pre-notification: 84%

13. Personalization • First Class mail • Commemorative stamps • Stamped return envelope • Name & address printed on the envelope • Personalized salutation • Full date on Cover Letter • Handwritten signature • Handwritten note

14. Personalization • (Maheux, 1989) Handwritten ‘thank you’ note: 30% No ‘thank you’ note: 22% • (Streiff, 1999) Stamped return envelope: 38% Business-reply envelope: 32%

15. Choice of sponsor • (Asch, 1994) Veteran Affairs return address pulled 20% more than a Hospital Department of Medicine • (RETICULUM, 2000) A joint study with IMS Health, Royal College, College of Family Physicians: 22% IMS Health alone: 12%

16. Tactics that don’t boost response • Pre-notification by mail • Offering non-monetary incentives (pens, mouse pads, candy, booklets, software..) • Mailing surveys on a specific day of the week • Promising anonymity • Gimmicks

17. On the use of Monetary Incentives Even symbolic sums will boost response • (Everett,1997) $0: 45% $1: 63% (one-dollar bill included in mailing) • (Donaldson, 1999) $0: 46% $5: 58% (five-dollar cheque included)

18. On the use of Monetary Incentives Larger incentives, Higher response rates • (Asch, 1998) $2 incentive: 46% $5 incentive: 63% • (Gunn, 1981) $0 incentive: 58% $25 incentive: 69% $50 incentive: 77%

19. On the use of Monetary Incentives Larger incentives, Higher response rates: UP TO A POINT • (VanGeest, 2001) $5 incentive: 60% ; $10 incentive: 68% $20 incentive: 67% • (RETICULUM/ IMS Health, 2000) $25 incentive: 22%; $50 incentive: 34% $75 incentive: 36%

20. On the use of Monetary Incentives Pre-paid incentives outperform Post-paid incentives • (Berry, 1987) $20 incentive, pre-paid: 78% $20 incentive, post-paid: 66%

21. Monetary Incentives in Lay Surveys • Sparse data • Controversial practice • Banned in certain jurisdictions

22. Monetary Incentives in Lay Surveys • (James & Bolstein, 1992) $0: 52% $2: 64% • (Dillman and al., 1999) in 5 different studies $2 incentive: boosted response by 19 to 31%

23. Monetary Incentives in Lay Surveys • Pre-paid incentives will outperform post-paid incentives • (Johnson & McLaughlin, 1990): $5 pre-paid: 83% $10 post-paid: 72% • (James & Bolstein, 1992): survey of small contractors $1 pre-paid: 64% $5 pre-paid: 72% $50 post-paid: 57%

24. On the use of Monetary Incentives • Why and how do they work???

25. On the use of Monetary Incentives • Respondent appreciated, not taken for granted • Value-creating • Attention grabbing: Secretary • Attention grabbing: Physician • Pre-paid incentives: create trust

26. On the use of Monetary Incentives The Pitfalls: • Point of no-return • Cost • Fraud • Ethical Issues

27. On the use of Monetary Incentives • Pre-paid incentives • (Gallagher, 2001) 46% replied 3% declined and returned the 20-dollar pre-paid incentive 51% declined, but pocketed the 20-dollar pre-paid incentive

28. PMRS Ottawa Chapter • Thank you very much • Q & A (in both official languages!)