180 likes | 378 Views
Mixed mode in LFS: questionnaire design and mode-effects. Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of May 2014. StatFin/EssNet web-project:. Designing LFS-CAWI questionnaire + iterative testing with cognitive interviews (N = 22)
E N D
Mixed mode in LFS: questionnaire design and mode-effects Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15th of May 2014
StatFin/EssNet web-project: • Designing LFS-CAWI questionnaire + iterative testing with cognitive interviews (N = 22) • Pilot survey (N = 8 000), experimental design (”split-plot”) with DK-option, working time and job search questions. Comparison to CAWI data from the same period. Paradata, respondent feedback.
Content of the presentation: • Questionnaire design: results & recommendations • Data collectionprocess • Mode-effects: WSTATOR (employment status) • Mode-effects: HWACTUAL (actualworkinghours) • Qualitative feedback
1. Questionnaire design: layout • BlaiseIS default layout was not perceived as attractive • Number of question / page • Navigation bars • Instructions • Error messages • Don’t know –option (DK)
Status bar Number of questions, updating the remaining questions Error message, DK Instructions, info-button Scrolling down for navigation bars
When the cursor points the info-symbol, the instructions appear in a box below.
Kyllä = Yes En = No (Don’t know not visible) Navigation bars: Takaisin = Back Jatka = Continue Jatka myöhemmin = Continue later/pause
DK-option, version 1 An error message: Please respond to this question or choose the “Don’t know” option. Kyllä = yes Ei = no En osaa sanoa = Don’t know
DK-option, version 2 Kyllä = yes Ei = no En osaa sanoa = Don’t know
2. Data collection process • Response rate 30 % potential for savings in data collection costs when the sample is large enough • CAWI-data more ”distorted” ? CAPI non-response more evenly distriputed with regard to educational background, gender and age than CAWI-data. • Speed: 40 % responses were received withing 3 days after that switch to CAPI?
3. Mode-effects: WSTATOR (employment status) • No significant mode-effects, but…. • media attention remains if unemployment rate 8,2 % instead of 7,4 % • sample for pilot study was small • CAWI respondents answer more often that they have been looking for job • = mode effect or selection bias?
4. Mode-effects: HWUSUAL (actual working hours) ”One question” –format for HWACTUAL: “How many hours did you work in your main job during the week 7.10.-13.10.2013?” ___ hours ___ minutes
”Grid” –format for HWACTUAL: “How many hours did you work in your main job during the week 7.10.-13.10.2013?” Date Hours Minutes Monday 7.10.2013 ___ ___ Tuesday 8.10.2013 ___ ___ Wednesday 9.10.2013 ___ ___ Thursday 10.10.2013 ___ ___ Friday 11.10.2013 ___ ___ Saturday 12.10.2103 ___ ___ Sunday 13.10.29013 ___ ___ Total ___ ___
Average number of hours worked (CATI-data and two different CAWI-datas)
5. Qualitative feedback • Too intimate information on employer • Questions do not fit for people with multiple jobs without main job or with scholarship • All kind of entrepreneurial activity is not covered • Working time items do not fit for people in atypical employment • Pensioners do not understand point of reference week
Conclusions and lessons learnt • Response rate of 30 % and speed of data collection encourage to continue towards mixed-mode LFS. • Visual design is important: balanced layout, intuitive navigation bars, informative error messages, a few questions per page, hidden instructions, hidden DK. • CAWI data more ”distorted” • In working time questions different format for CATI and CAWI may be needed to reduce mode-effects (”functional equivalency”). Differences in WSTATOR need to be further analysed.
Thank you! Contact: liisa.larja@stat.fi