1 / 34

Increasing response rates

Increasing response rates. Ineke Stoop SCP. Is it possible?. and is it effective?. Nonresponse research. How to increase response rates? How to measure and correct for nonresponse bias?. SCP-survey. Face-to-face questionnaire + long drop off for every 6+ years household member

loman
Download Presentation

Increasing response rates

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Increasing response rates Ineke Stoop SCP Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  2. Is it possible? Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  3. and is it effective? Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  4. Nonresponse research • How to increase response rates? • How to measure and correct for nonresponse bias? Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  5. SCP-survey • Face-to-face questionnaire + long drop off for every 6+ years household member • Use of social and cultural services, amenities, facilities (no values, opinions) • Very low non-contact rate • High rate of refusal conversion • Final response rate 1999 66% Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  6. Response rates AVO Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  7. Cumulative response rates Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  8. Increase response ratesDescribe response process Contactability Reluctance Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  9. Paradata • Call records (nr, timing, outcome) • Characteristics interaction • Interviewer observations • Sample frame data • Low-level geographic data bases • Interviewer characteristics Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  10. Call (nr, time, mode) First contact ContactabilityTNC/NAH/CFC Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  11. At home, interviewer calls, contact rate Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  12. Timing contacts • Interviewers prefer working hours • Contact rates higher in the evening • Cooperation independent of timing • Fewer calls if evening calls only • Length fieldwork period/costs? Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  13. Easy Evening calls Phone number available Detached dwelling Child at home Difficult Big city dweller Poor maintenance neighbourhood Young Single Employed, student High cultural participation Contactability Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  14. Continuum of resistance • Extrapolate from hard to reach respondents to noncontacts? • Socio-demographics • Core variables • Two types of hard to contact (short term/long term) Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  15. Reluctance Respondent has been contacted Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  16. Refusal Establish first contact No Yes Inter- view Hard converted refusal Request to cooperate Request to cooperate Request to cooperate Yes Yes Yes Inter- view No Follow -up Con- tact Inter- view No Follow -up Yes Con- tact No No Yes No No Refusal Soft converted refusal Refusal Yes Cooperative respondent Refusal Refusal Reluctance Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  17. Inter- view Refusal at first contact Re- approached Final refusal Final refusal What happens in the AVO after a first refusal? Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  18. Many among Big city dwellers Few among Men Singles Higher educated Reluctant respondents Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  19. Cooperative and reluctant respondents Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  20. Refusers, reluctant and cooperative respondents Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  21. Region and refusal conversion Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  22. Results refusal conversion • Males, singles and higher educated underrepresented among converted refusals • Do interviewer strategies work for everybody? • Are only promising cases re-contacted? • May worsen final sample composition • Effect refusal conversion • Works only for honest burghers, Mr. and Mrs. Average? Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  23. Conclusions regular AVO • Contactability related to socio-demographics, being out of the house and fieldwork strategy • Reluctance related to family composition, gender and education and persuasiveness Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  24. Do nonrespondents differ?Are final refusals similar to converted refusals • Sample frame information • Neighbourhood information • Fieldwork information • Late respondents (similar to non-contacts?) • Reluctant respondents (similar to refusers?) • Socio-demographic information Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  25. Frame Observation Fieldwork Easy Hard to contact Reluctant Follow-up Response and nonresponse Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  26. Follow-up survey • Small subsample persistent refusers • 1 person, 20 minutes, multi-mode • Experienced, motivated, well paid interviewers • Wide range of incentives • 235 successful interviews • 70% cooperation rate Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  27. Why did refusers cooperate? • High quality interviewers (and telling them they are the best) • Extensive briefing • Trust (money for incentives) and support (newsletter) • Importance (newspaper article) • Wide range of incentives • Better payment • Commitment sponsor and fieldwork organisation Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  28. Idiosyncratic choice of incentives Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  29. Final analysis Hard to contact respondents Reluctant respondents Refusers who cooperated in follow-up survey Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  30. Follow-up survey • Sample of persistent refusals • Socio-demographic differences (made up for composition regular respondents) • Small differences in survey variables (mostly related to socio-demographics) • Lower participation in classical culture • Less PC ownership • Fewer sports activities • Slightly less active in many aspects • Not at all similar to converted refusals Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  31. Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  32. Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  33. Reducing bias instead of increasing response rates • Increasing response rates MAY not lead to better survey estimates • Difficult respondents may not be similar to final nonrespondents • Obtain information on the process (cause of/reason for nonparticipation) • Collect independent information on nonrespondents • Frame information + registers • Central question procedure • Follow-up survey among refusers Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

  34. Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

More Related