Improving Response Rates Lessons from Physician Surveys

1. Improving Response RatesLessons from Physician Surveys PMRS Ottawa Chapter February 26, 2004

2. Presentation Overview Survey Response Rates: The state of the art Particularities of Physician Surveys Response rate boosting tactics: What works and what does not. On the use of Monetary Incentives in Physician and Consumer Surveys Q & A period

3. Current response rates Academic Surveys published between 1961 and 1977: 71% Academic surveys published in 1991: 54% Academic surveys published between 1986 and 1995, sample size over 1,000 respondents: 52% Commercial/marketing physician surveys (2002): 20% RETICULUM surveys: 12% to 66%

4. Current response rates Surveys of executives, published in 1991: 21% PMRS Members surveys: 15.7%(1997);11.3% (2000); One-time telephone surveys: 16% (1997); 13% (2002)

5. Physician Surveys Particularities More homogenous populations Highly-regulated professionals Better sampling frames Better record-keeping

6. Physician Surveys Particularities Highly-solicited respondents Highly-educated respondents ‘Well-connected’ respondents

7. Physician Surveys Particularities Surrounded by ‘tough’ gatekeepers ‘Addicted’ to monetary incentives

8. Tactics that boost response by 50% or more Monetary Incentives Multiple contacts & multiple contact modes

9. Monetary Incentives (Gallagher, 2001) 1st contact by mail, no incentive: 11% 2nd contact by phone, no incentive: 22% (cumul) 3rd contact by courier, $20 incentive: 57% (cumul) (Malin, 2000) 1st mailing, no incentive: 17% 2nd mailing, no incentive: 13% 3rd mailing, $50 incentive: 66% Cumulative response rate: 76%

10. Multiple Contacts & Contact Modes Typical response rates after multiple mailings: 1st mailing: X% 2nd mailing: X/2% 3rd mailing: X/4% (CDC, 1997): 1st contact by First Class mail: 60% 2nd contact by Fedex: 72% (cumul) 3rd contact by phone: 96% (cumul)

11. Tactics that boost response by a few % points Pre-notification by phone Personalization Advertising the survey Choice of sponsors Shortening the questionnaire Instituting a draw

12. Pre-notification by phone (Osborn, 1996) No Pre-notification: 64% Pre-notification: 77% (Ward, 1994) No Pre-notification: 69% Pre-notification: 84%

13. Personalization First Class mail Commemorative stamps Stamped return envelope Name & address printed on the envelope Personalized salutation Full date on Cover Letter Handwritten signature Handwritten note

14. Personalization (Maheux, 1989) Handwritten ‘thank you’ note: 30% No ‘thank you’ note: 22% (Streiff, 1999) Stamped return envelope: 38% Business-reply envelope: 32%

15. Choice of sponsor (Asch, 1994) Veteran Affairs return address pulled 20% more than a Hospital Department of Medicine (RETICULUM, 2000) A joint study with IMS Health, Royal College, College of Family Physicians: 22% IMS Health alone: 12%

16. Tactics that don’t boost response Pre-notification by mail Offering non-monetary incentives (pens, mouse pads, candy, booklets, software..) Mailing surveys on a specific day of the week Promising anonymity Gimmicks

17. On the use of Monetary Incentives Even symbolic sums will boost response (Everett,1997) $0: 45% $1: 63% (one-dollar bill included in mailing) (Donaldson, 1999) $0: 46% $5: 58% (five-dollar cheque included)

18. On the use of Monetary Incentives Larger incentives, Higher response rates (Asch, 1998) $2 incentive: 46% $5 incentive: 63% (Gunn, 1981) $0 incentive: 58% $25 incentive: 69% $50 incentive: 77%

19. On the use of Monetary Incentives Larger incentives, Higher response rates: UP TO A POINT (VanGeest, 2001) $5 incentive: 60% ; $10 incentive: 68% $20 incentive: 67% (RETICULUM/ IMS Health, 2000) $25 incentive: 22%; $50 incentive: 34% $75 incentive: 36%

20. On the use of Monetary Incentives Pre-paid incentives outperform Post-paid incentives (Berry, 1987) $20 incentive, pre-paid: 78% $20 incentive, post-paid: 66%

21. Monetary Incentives in Lay Surveys Sparse data Controversial practice Banned in certain jurisdictions

22. Monetary Incentives in Lay Surveys (James & Bolstein, 1992) $0: 52% $2: 64% (Dillman and al., 1999) in 5 different studies $2 incentive: boosted response by 19 to 31%

23. Monetary Incentives in Lay Surveys Pre-paid incentives will outperform post-paid incentives (Johnson & McLaughlin, 1990): $5 pre-paid: 83% $10 post-paid: 72% (James & Bolstein, 1992): survey of small contractors $1 pre-paid: 64% $5 pre-paid: 72% $50 post-paid: 57%

24. On the use of Monetary Incentives Why and how do they work???

25. On the use of Monetary Incentives Respondent appreciated, not taken for granted Value-creating Attention grabbing: Secretary Attention grabbing: Physician Pre-paid incentives: create trust

26. On the use of Monetary Incentives The Pitfalls: Point of no-return Cost Fraud Ethical Issues

27. On the use of Monetary Incentives Pre-paid incentives (Gallagher, 2001) 46% replied 3% declined and returned the 20-dollar pre-paid incentive 51% declined, but pocketed the 20-dollar pre-paid incentive

28. PMRS Ottawa Chapter Thank you very much Q & A (in both official languages!)