1 / 28

Timing Isn’t Everything, But Money Talks

University of Essex, July 2013. Timing Isn’t Everything, But Money Talks. How to encourage a face-to-face household panel to go online?. Gerry Nicolaas Carl Cullinane. Contents. Background Design of Experiment Results Summary of Results and Conclusions. Background. 1. Background.

jalene
Download Presentation

Timing Isn’t Everything, But Money Talks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. University of Essex, July 2013 Timing Isn’t Everything, But Money Talks How to encourage a face-to-face household panel to go online? Gerry Nicolaas Carl Cullinane

  2. Contents Background Design of Experiment Results Summary of Results and Conclusions

  3. Background • 1.

  4. Background Switch to mixed mode data collection at wave 7 of Understanding Society Potential for cost savings Potential for reducing attrition

  5. Context Longitudinal household panel Face-to-face interviewing of all household members at waves 1 to 6 Greatest potential for reducing data collection costs when an interviewer does not have to visit the household Previous experiment mixing telephone & face-to-face: • Costs can be reduced • BUT response rates suffer

  6. Innovation Panel – wave 5 Vehicle for methodological development & testing About 2,500 individuals in 1,500 households Main objective of IP5 = • Determine whether it is possible to reduce costs by mixing web questionnaires and face-to-face interviews without sacrificing data quality • Sequential mixed mode design starting with web

  7. IP5 Design Experimental group Web only phase F2F phase, web open Control group No web F2F only phase 6

  8. Research question Is it possible to boost the proportion of whole households completing web questionnaires by • Timing the arrival of the invitation to go online? • Offering a web bonus?

  9. Design of experiment within mixed-mode sample 2.

  10. Design of experiment within mixed-mode sample • (1) Timing of invitation to go online • Random allocation of households to: • Letter (+ email) to arrive on Friday • Letter (+ email) to arrive on Monday • Reminder letters (+ emails) sent 2 and 4 days later • (2) Web bonus • Random allocation of households to: • No web bonus • £5 per household member conditional on all completing online questionnaire

  11. Design of experiment within mixed-mode sample Invite arrival on Friday Invite arrival on Monday No web bonus 270 households 265 households Cond. £5 per hhld member 266 households 276 households

  12. Results 3.

  13. Experimental effects on Web Response Base: IP5 WEB Sample (n=1077)

  14. Experimental effects on Web Response Base: IP5 WEB Sample (n=1077)

  15. Effect of Bonus by Sample Type Base: IP5 WEB Sample (n=1077)

  16. Effect of Bonus by Upfront Incentives Base: IP5 WEB Sample (n=1077)

  17. Unconditional Incentives Original Sample £5 £10 Refreshment Sample £10 £20 £30 16

  18. Effect of Bonus by Children in Household Base: IP5 WEB Sample (n=1077)

  19. Effect of Bonus by Advance Mailing Base: IP5 WEB Sample (n=1077)

  20. Effect of timing by Advance Mailing Base: IP5 WEB Sample (n=1077)

  21. Experimental effects on Final Response 75.6 72.8 Base: IP5 WEB Sample (n=1077)

  22. Experimental effects on Final Response 75.6 74 74.4 72.8 Base: IP5 WEB Sample (n=1077)

  23. Estimation of cost savings Web bonus of £5 compared to no web bonus: Small reduction in travel and mileage costs 15% reduction in interviewer fees Offsetting the cost of web bonuses reduces the saving in interviewer fees to less than 5%

  24. Summary of results and conclusions 4.

  25. Summary of results Friday mailing had a small but diluted effect Web bonus increased full household web response Effect of bonus varied by a number of factors, e.g. • Diminishing returns with larger upfront incentives • Greater effect for households with children • Greater effect when household contacted by email Web bonus did not increase final response rate But potential for cost savings

  26. Limitations Small sample sizes Confounding of sample type and value of unconditional incentives sent with advance letter Estimation of costs

  27. Points for discussion Timing to be looked at further? Potential for greater cost savings through targeting Large upfront incentives vs conditional web bonus- costs

  28. Thank you If you want further information or would like to contact the author, Carl Cullinane T. 020 7549 7158 E. carl.cullinane@natcen.ac.uk Visit us online, natcen.ac.uk

More Related