Errors in Factual Questions and their Consequences

1. Errors in Factual Questions and their Consequences Annette Scherpenzeel QMSS Seminar 12-08-2004 Lugano

2. Swiss Household Panel

3. International Standard Classification of Occupations (ISCO) • Legislators, senior officials, and managers • Professionals • Technicians and associate professionals • Clerks • Service workers and shop and market sales workers • Skilled agricultural and fishery workers • Craft and related trades workers • Plant and machine operators and assemblers • Elementary occupations • Armed forces

4. ISCO Example1 Nuclear physicist unit group 2111 (physicists and astronomers) minor group 211 (physicists, chemists and related professions) sub-major group 21 (physical, mathematical and engineering science professionals) group 2 (professionals) [1] Example taken from Bergman and Joye, "Comparing Social Stratification Schemas".

5. n = 4993 in 1999, n = 3783 in 2002 Elementary occupations Plant and machine 100 operator assemblers 12 11 11 11 Craft and related 80 trades workers 11 12 12 13 Skilled agricultural Percentage and fishery workers 14 14 60 14 13 Service workers, market sales workers 40 26 26 25 25 Clerks Technicians and ass- ociate professionals 20 17 18 19 19 Professionals Legislators, senior 8 8 8 7 0 officials, managers 1999 2000 2001 2002 Year of panel data collection International Standard Classification of Occupations Distributions of first four waves

6. n = 4993 in 1999, n = 4673 in 2000 Elementary occupations 100 3 % Plant and machine 1 % 12 operator assemblers 11 1 % 80 Craft and related 64 % 12 trades workers 13 6 % Skilled agricultural 60 14 and fishery workers 13 Percentage 15 % Service workers, market sales workers 40 25 26 4 % Clerks Technicians and ass- 4 % ociate professionals 20 17 18 Professionals Legislators, senior 8 7 officials, managers 0 1999 2000 Year of panel data collection International Standard Classification of Occupations Change between first and second wave

7. * Total number of working persons with completed individual interviews in both 1999 and 2000. For some variables in the table the n was smaller because of filters to the questions (e.g. "Percentage of part-time work" was only asked to people working part-time).

8. Stability of various other demographic variablesPercentage of people with the same score in 1999 and 2000 * Total number of persons with completed individual interviews in both 1999 and 2000. For some variables in the table the n was smaller because of filters to the questions ** Total number of working persons with completed individual interviews in both 1999 and 2000. For some variables in the table the n was smaller because of filters to the questions.

9. d3 d2 d1 0.93 0.99 0.94 T4 T3 T2 T1 ISCO ISCO ISCO ISCO catg 5 catg 5 catg 5 catg 5 0.83 0.82 0.82 0.82 isc45 isc35 isc25 isc15 e4 e3 e2 e1 Four-wave Simplex model of ISCOCategory 5 (Service workers, market sales workers)

10. d1 d3 d2 T4 T3 0.99 0.93 0.99 T2 T1 manage manage manage manage 0.78 0.81 0.79 0.77 p02w34 p01w34 p00w34 p99w34 e4 e2 e3 e1 Four-wave Simplex model using one indicator of hierarchy of work-position.

11. Coefficients estimated with the four-wave Simplex model for all ISCO categories. * Frequency distribution of ISCO-1999 for persons having a valid ISCO score in all four waves. ** Total number of persons who have a valid ISCO score in all four waves.

12. condition 1 in wave 1 condition 1 in wave 2 condition 2 in wave 1 condition 2 in wave 2 event between waves condition 3 in wave 1 condition 3 in wave 2 satisfaction in wave 1 relative situation satisfaction in wave 2 Stable Component General two-wave causal model of satisfaction

13. Contribution of the different factors to the explanation of satisfaction 80 48 23 60 11 28 40 41 FACTOR Percentage of explained variance 38 Stable component 24 20 11 Relative situation Events 12 Living conditions 0 finances health work relations DOMAIN

14. Conclusions • Repeated cross-sections give a false impression of stability in "objective" variables. In reality, these variables can contain considerable random variation • This unreliability can only be known by repeating the same question in each wave. Because of the nature of the "objective" variables, we can distinguish unreliability from real change • The unreliability in the "objective" variables affects their explicative power in longitudinal models. It impairs the opportunities to analyze labor market mobility by panel studies

15. Discussion: What is the cause • Swiss Household Survey data collection • Not likely: many other variables are stable • ISCO coding database • SOEP-EG: 20% no change in occupational title but change in occupational class. • But: other working variables also unstable? • Interviewer: interpretation, training, search strategy, etc • Explains only about 2% of the variation over waves • Time-interval between measurements: missing events? • Selection bias: attrition nor occupational mobility are randomly sampled • But: this would select for stability overestimation? • Respondent: interpretation, frame of reference, memory, satisfice, etc • Question formulation

16. Next: • ISCO: Compare with retrospective data from the same respondents • ISCO: Compare ISCO occupational title with occupational class • Other variables: methodological experiment?