1 / 18

Maximizing Survey Response Rates: Does Paper Still Matter in Push-to-Web Design?

Explore the significance of using paper questionnaires in a push-to-web survey design, analyzing response rates and demographic effects. Learn about refining strata and addressing age variations for a more accurate sample profile.

faylinn
Download Presentation

Maximizing Survey Response Rates: Does Paper Still Matter in Push-to-Web Design?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Does paper matter?The value of the second mode within a push-to-web survey design Joel Williams Presented at the ESRA Conference, Zagreb, 16th July 2019

  2. The Community Life Survey in summary Annual UK Gov study of >10,000 adults living in England Neighbourhood life, volunteering, charity, civil engagement Address sample frame; all adults (≤4 pHH) can take part Web & paper questionnaire options (PtW)

  3. The Community Life Survey letter & paper q’re front

  4. The Community Life Survey contact sequence Letter 1: four online logins; paper version on request +2wks: Letter 2: as letter 1 (but different features stressed) +4wks: Letter 3a: as letters 1/2 +4wks: Letter 3b: as letters 1/2 but 2 paper q’res included

  5. Letter 3 distribution by area type stratum (2016/17 onwards) Design weighted

  6. Response rates in 2016/17, by stratum 21% 23% 24% 21% 23% Design weighted

  7. Estimated response rates in 2016/17, by age 13% 17% 21% 22% 29% 33% 25% Design weighted

  8. Plenty of tools… Lots of tools for manipulating the response rate: Offering/not providing paper questionnaires in letter 3 Varying the number of letters (1, 2, 3 or 4) Varying the incentive (e.g. £0, £5, £10) Varying age eligibility (e.g. X% with 16-34, 100-X% 16+)

  9. But need more refined strata These tools only improve sample profile if strata refined Address-based samples stratified using local aggregation But lots of population variation within the aggregation Especially age! Is there anything we can do to improve that?

  10. Potential for address-level age-based stratification using auxiliary commercial data?

  11. For example this design would produce a marginally better profile… % of 3rd letters with 2 paper questionnaires

  12. Q1: How important is it to cover non-users of internet? Controlling for demographic characteristics & mode… Heavier internet users > (reported) civic participation… … and volunteering and giving to charity Strong effects (OR>2) observed for most client metrics

  13. Selected EMMs (evaluated with factors equally weighted and covariates held at global mean)

  14. Q2: Are there measurement effects web v paper? Controlling for demographic characteristics & web use… Paper respondents more likely to (report) giving to charity … and being generally neighbourly But less likely to report formal volunteering/civic activity Otherwise mostly modest effects

  15. Selected EMMs (evaluated with factors equally weighted and covariates held at global mean)

  16. Sum up Uniform survey design not optimal for sample profile But strata not quite refined enough to give us real control Rare/never internet users are different & must be covered Some ‘mode’ effects but do they hide selection effects? Plenty of evolution to get through…

  17. THANK YOU!For more, see:https://tinyurl.com/y2j2785u

  18. Does paper matter?The value of the second mode within a push-to-web survey design Joel Williams Presented at the ESRA Conference, Zagreb, 16th July 2019

More Related