140 likes | 143 Views
Dependent Interviewing. A Remedy or a Curse for Measurement Error in Surveys?. Paulina Pankowska Vrije Universiteit Amsterdam Bart Bakker Statistics Netherlands. Daniel Oberski Ut recht University Dimitris Pavlopoulos Vrije Universiteit Amsterdam.
E N D
Dependent Interviewing A Remedy or a Curse for Measurement Error in Surveys? Paulina Pankowska Vrije Universiteit Amsterdam Bart Bakker Statistics Netherlands Daniel Oberski Utrecht University Dimitris Pavlopoulos VrijeUniversiteit Amsterdam
measurement error in longitudinal surveys • Measuring a state: onemistake • Measuringtransitions: twomistakes Correct Correct Wrong Correct Correct Year 4 Year 1 Year 3 Year 5 Year 2 • (Potential) solution- DI
Dependent interviewing (DI) • Respondents are reminded of previous answers
Di & Random Measurement error Correct Correct Wrong Correct Correct Correct Correct Correct Correct Correct Year 4 Year 4 Year 1 Year 1 Year 3 Year 3 Year 5 Year 5 Year 2 Year 2 • Expected to reduce (random) error • Assists recall less spurious change • Reduces cognitive burden
Di & systematic Measurement error • Might lead to systematic error • e.g. satisficing error carry-over • Overall effect on reliability and estimates uncertain Wrong Wrong Correct Wrong Wrong Wrong Wrong Correct Wrong Wrong Year 4 Year 4 Year 1 Year 1 Year 3 Year 3 Year 2 Year 2 Year 5 Year 5
Natural experiment dutchlfs: temporary contract “Do you currently have a permanent contract?” “Last time you had a temporary contract. Is this still the case?” • Until 2009: Dependent • PDI - “remind, still” • Only if no job change occurred • As from 2010: Independent • Also in 2009 if job change occurred • Permanent contract: value carried over
Natural experiment: the design • A total of 3 scenarios:
Method: Hidden Markov Model • Hidden Markov Models (HMMs): • Separate ‘true’ change from error • Without error-free data
HMM & Dependent interviewing • DI affects also systematic error • Relax the local independence assumption • Multiple indicators per time point…
Data • Linked survey (LFS) – register (ER) data • Sample consists of 86,075 individuals aged 25 to 55 who • Started LFS in 2009 - DI • Started LFS in 2010 - INDI • Dataset contains quarterly information on each individual for 5 time points
Results: random error • Reference: INDI would have been DI • DI reduces random error
Results: systematic error • Reference: INDI would have been DI • DI does not increase systematic error
Conclusions: promising for di! • DI reduces random error (spurious change) • DI is not associated with a higher incidence of systematic (autocorrelated) error • There is a high probability of repeating errors, regardless of the interviewing regime