1 / 17

Minimizing interviewer effects and reducing refusal rates at Statistics Iceland

This study examines the implementation of interviewer training at Statistics Iceland to reduce survey errors, specifically coverage error, non-response error, sampling error, and measurement error. The training program aims to standardize interviewing techniques, reduce interviewer variance, and minimize refusal rates. The effectiveness of the training is assessed by comparing data collected in 2008 and 2009, including refusal rates and interviewer effects. The results indicate positive effects on refusal rates and suggest ongoing efforts to monitor and improve interviewer performance.

corinew
Download Presentation

Minimizing interviewer effects and reducing refusal rates at Statistics Iceland

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Minimizing interviewer effects and reducing refusal rates at Statistics Iceland Anton Örn Karlsson

  2. Survey errors • Four categories: • Coverage error • Non response error • Sampling error • Measurement error

  3. Interviewertrainingtoreduceerrors • Refusal aversion training • Increase response rates • Reduce non response error • Standardized interview techniques • Reduce interviewer variance • Reduce measurement error

  4. Interviewers at Statistics Iceland • Permanent • 13 CATI • 8 field interviewers • Temporary • 40 CATI • For ICT and SILC

  5. Training for interviewers • Newtrainingprogramstartedin 2009 • Twogoals: • Reduction of interviewer relatederror • Standardizedinterviewing • Reading questions as worded • Probing inadequate or incomplete answers • Recording answers directly • Being neutral with regards to the respondent • Minimizingrefusalrates • Tailoringtheparticipationrequest • BasedonGrovesandMcGonagle (2001)

  6. Assessment of the trainingprogram • Comparing data collected in 2008 with data collected in 2009 • For both ICT and SILC • Refusal rates • Should be lower in 2009 compared to 2008 • Intraclass correlation coefficient (ICC), a measure of interviewer effects • Should be lower in 2009 compared to 2008

  7. Data collection • The sequence of surveys: • 2008, January to the end of May: • ICT • Travel survey • SILC • 2009, February to the beginning of May • SILC • ICT • All data for ICT and SILC was collected via CATI

  8. Calculations(1) • Refusalrates: • ICC • Fewquestionsselected • SILC: Amounts regarding the habitat of the household, debts and other economic issues of the household. • ICT: Yes/no questions about the technical equipment in the household. • Threecriteria: • 1) At least 100 valid answers • 2) At least 10 interviews per interviewer • 3) Same wording in both years

  9. Calculations (2) • A formula from Groves (1989): • ICC: • Based on a one-way analysis of variance • Va = Between mean square • Vb = Within mean square • M = Number of interviews per interviewer • A function was built in R in order to calculate ICC in that way

  10. Results(1)

  11. Results(2)

  12. Results(3)

  13. Results(4): Refusal rates for ICT and SILC from 2003 – 2010

  14. Discussion(1) • Interviewer variability • Some positive effects of the new training, but there is still more work to do • Compared to other studies: High variability • Mean coefficients lower in 2009 • Majority of coefficients in ICT higher in 2009 • Effects of interviewer experience • The danger of high variability • Variance inflation

  15. What can be done? • Monitor interviewers • Give feedback about interviewer performance. • Assess how interviewers read questions and how they probe for answers. • Rewrite questions • Some will have to be rewritten so all respondents will understand them in the same way • Giving interviewers standard phrases to use in interviews

  16. Discussion(2) • Refusal rates • Seems that the refusal training has had positive effects on the refusal rates of both surveys. • SILC: A drop of four percentage points from 2008 to 2010. • ICT: The lowest recorded refusal rates are from 2009 and 2010.

  17. Thank you!

More Related