1 / 32

Managing Multi Mode Collection Instruments in the 2011 UK Census

Explore the challenges, outcomes, and interventions in utilizing electronic questionnaires for the 2011 UK Census to improve response quality, reduce burden, and enhance timeliness. Discover the impact of various interventions on respondent behavior and questionnaire completion time.

gellison
Download Presentation

Managing Multi Mode Collection Instruments in the 2011 UK Census

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Managing Multi Mode Collection Instruments in the 2011 UK Census Frank Nolan, Heather Wagstaff, Ruth Wallis Office for National Statistics UK

  2. Contents What I plan to cover • Electronic Questionnaires • Challenges • Outcomes IAOS2008

  3. Context • Traditional Census • High Profile • Involves ALL citizens • Enumerators – face-to-face • Increased Diversity • Lower response rates • Modern Government / NSI - eforms • Options for Multi Mode IAOS2008

  4. 1. Electronic Questionnaires IAOS2008

  5. Cost / Benefits overview • Intuitively we might think that there were both costs and benefits • Two dimensions • Respondent Burden • Quality • Graphically IAOS2008

  6. Burden Electronic X Paper Interventions

  7. Respondent Burden • Reductions • Burden of logging in – security, .. • Skipping not applicable questions • Increases • Repeated entry for edit failures IAOS2008

  8. Quality Electronic X Paper Interventions

  9. Quality • Increases • Key entry – no recognition issues • Handwriting / Within box marks / .. • Prompt for non response / …… • Reductions • Constraints on response • Repeated entry • False response (to get around edits) IAOS2008

  10. Improved Timeliness (of response) • Internet completions can arrive prior to Census day (quality?) • More dynamic survey taking? IAOS2008

  11. Reduction in Cost • Key entry completed by respondent • So no scanning / recognition • IF QUALITY IMPROVES • Less back office correction • Still cost of internet software set up etc. IAOS2008

  12. 2. Challenges IAOS2008

  13. Interventions • How can we improve the electronic questionnaire – balance mode effects • Introduce interventions / edits • Similar to Interviewer controlled, but less variability (machine not human driver?) IAOS2008

  14. Spectrum of Interventions Spectrum from • Low impact / sensible / simple / expected • ……. • ……. • High Impact / burdensome / complex IAOS2008

  15. Spectrum of changes • Repeated data automatically entered • Skip sections not applicable (filter) • Constrain when only ONE response required • Range check • Highlight item non response • Within and between record checks • Voluntary or Mandatory check (hard / soft) IAOS2008

  16. Model • Theoretical models developed • To be followed with field work • Current draft of questionnaire • Simplified with • Working adults (35 questions) • Children (20 questions) • Measurements • Time to complete = Burden • Non response = Quality (Inverse) IAOS2008

  17. Model - Burden T = KS + B + L + I + C • KS - Key Strokes • B – Tick Boxes • L – Lines for write in questions • I – Instruction lines • C – Cognition levels – understanding and recall IAOS2008

  18. Model – Non Response NR = ∑ NRq • A simple score function • NRq is the 2001 question non response rates • NRqi is applied for each intervention i IAOS2008

  19. 3. Outcomes IAOS2008

  20. Response (Time) v Interventions IAOS2008

  21. Time to Complete • Time reduced with (small effect) • Repeated questions • Skipping • One response (radio button) • Time Increased (larger effect) • Range check • Non response - soft edit • Non response – hard edit IAOS2008

  22. Time to Complete • More effect for working adults than children • Resulting from more questions • Is there an effect for perceived saving in addition to the actual saving? IAOS2008

  23. Non Response v Interventions IAOS2008

  24. Non Response • Only small effect for first five interventions • One repeated question • Few skips • Radio button (does not effect non response) • Gains • Improve item non response • see Williams 2006 Australian Census IAOS2008

  25. Where is the Balance? • YES to • Repeated responses • Filters to skip • Radio Buttons (one tick) • Positive for response and burden IAOS2008

  26. Where is the Balance? • BUT • Range checks • Item non response soft check • Positive for improved response and • Negative for response burden IAOS2008

  27. Where is the Balance? • NO to • Item non response hard check • Negative for response and burden IAOS2008

  28. 4. Conclusion IAOS2008

  29. Drivers • Meet user needs • Multi Mode: Duality of delivering / collecting data in a range of formats • Tension between consistency and relevance • Mode Effect: Same questionnaire versus advantages from electronic questionnaires IAOS2008

  30. Internet Offers • Significant cost advantages of high Internet take up for Census • High field and processing costs • BUT internet not universal • AND Internet respondents are not representative of population • Younger, larger household, mobile, studying and in labour force, higher income, higher qualifications, … [Williams 2007] IAOS2008

  31. Electronic Questionnaires • Improve quality of response and save respondent costs with some checks • Net reduction in quality of response with too many check • Quality and Respondent Burden balance depends on number of applicable questions • Good quality improvements with soft edits for range and non response checks IAOS2008

  32. Thank You IAOS2008

More Related