1 / 16

Understanding the Decision to Participate in a Survey and the Choice of the Response Mode

Understanding the Decision to Participate in a Survey and the Choice of the Response Mode. Anders Holmberg and Boris Lorenc. European Conference on Quality in Official Statistics Rome, Italy July 11, 2008. Content. Introduction to the problem Background The design of the study goals setup

alta
Download Presentation

Understanding the Decision to Participate in a Survey and the Choice of the Response Mode

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Understanding the Decision to Participate in a Survey and the Choice of the Response Mode Anders Holmberg and Boris Lorenc European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  2. Content • Introduction to the problem • Background • The design of the study • goals • setup • results • Summary and tentative conclusions European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  3. Mixed Mail-and-Web Surveys • Swedish individual and household surveys with Self-Administered Questionnaires (SAQ) • The problem: low use of the web alternative • belief in rationality of the web mode (editing, other quality aspects, cost, environment) • cannot publicly afford the Web mode only • possibly, a mode switch (more of which later) • goal: finding a solution with an increase in the proportion of web respondents and a cost reduction, without harming the response rate European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  4. Background • PAP (mail Q) as the standard mode for SAQ • cover letter, questionnaire, return envelope, in some cases separate instruction, incentives. • response rates – as elsewhere – fall gradually • Web as an alternative mode for SAQ • introduced in the 2000’s for ind/hh surveys in Sweden • with the expectations of; easier input editing, higher response rate, cost reduction, and greater acceptance • carried out by including a paper sheet in the above, with an URL and login data • Reminders European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  5. Background (cont’d) • Web as an alternative mode for SAQ (cont’d) • its use is optional, according to preference of external customer or internal statistical product • The expectations were not fulfilled • mostly due to the web’s low use: in mail-and-web mixed mode surveys around 10-15% of the responses are collected through the web • no previous study of cost done, but impression of survey managers is that the costs are higher in the mixed mode than in the single paper mode. (Development cost of single use SAQ:s) • Reasons? European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  6. The Parent Survey • Entry on the labour market after graduation. • Proportion of NR as well as web respondent were higher for men than women, (37% vs. 28%) and (23% vs. 11%) • The proportion of NR is highest in the lowest and highest income class. • No observed differences in choice of mode with regards to age and immigrant background. European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  7. Design of the Study • A study was initiated to understand the reasons for the low proportion of web responses • conducted as a telephone follow-up • a simple random sample among the three groups in the parent survey: web respondents (W), paper respondents (P) and nonrespondents (NR) • sample sizes of about 310 in each group; the response rates 77%, 83% and 61% respectively European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  8. Data collection results from interviews European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  9. Design of the Study, the Interview questionnaire • Specific questions to each group of respondents • Was the web alternative observed? • Why/why not use the web questionnaire? • Preferences regarding choice of response mode • A follow-up question to the NR-group on their status of occupation. European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  10. Results Web-mode visibility • The main results • 80% of paper respondents saw the web mode. Roughly 65% of paper respondents saw the web mode alt. in the first mail-out. Roughly 55% of the NR-group saw the web mode in the first mail-out. • 75% of the paper respondents who saw the web mode alt. chose Simplicity of PAP as the reason to hold onto the mail mode (as did 60% of those paper respondents who, on another question, ranked web as the mode of highest preference) • 25% of the paper respondents who saw the web alt. chose those alternatives that refer to unavailability of computer or Internet (permanent or temporal) as the reason European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  11. Results: Choice of paper or web • In an optional open-ended question, 20% of the paper respondents who saw the web mode alt., gave circumstantial reasons for using the PAP mode, namely immediate availability of PAP • security and privacy do not seem to be a concern • Web respondents • Over 90 % of the web responders had the simplicity of the web as their main reason for choosing the web mode. European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  12. Results: The Nonrespondents • > 60 % of nonrespondents declared specific circumstances as their main reason for not responding, i.e. lack of time, forgetfulness, etc • Approx. 8% decided not to take part in the parent survey as they had (fallaciously) interpreted themselves as being outside the target population. European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  13. Results: Question on mode of preference European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  14. Respondent Suggestions to increase web use • Send survey invitationsthrough e-mail • Discard the paper questionnaire • Make the web-option obvious and the only alternative • Provide incentives for web responses • Have functioning technical solutions European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  15. Summary • Observations • The web option is not visible enough • The mail mode is considered simple (easily move around, change entries, break and continue) • Circumstantial reasons: the ‘what is at hand’ principle • Conclusions/changes • Strategy change needed when SAQ is mailed • Marketing of the web alternative • Other technical solution when receiving web questionnaires • Construction environment. European Conference on Quality in Official Statistics Rome, Italy July 11, 2008

  16. Thank You! Anders Holmberg and Boris Lorenc firstname.surname@scb.se

More Related