Latent Class Analysis of Rotation Group Bias: The Case of Unemployment

1. Latent Class Analysis of Rotation Group Bias: The Case of Unemployment Paul Biemer, UNC and RTI Bac Tran, US Census Bureau Jane Zavisca, University of Arizona SAMSI Conference, 11/10/2005

2. Overview • Motivation: To understand measurement error in the official unemployment rate • Method: Latent Class Analysis: measurement error as classification error • Distinction from previous research: Focus on measurement error mechanisms, as opposed to correcting marginal estimates. • Ultimate goal: To improve survey design.

3. The Official Unemployment Rate { In Labor Force Source: The Current Population Survey, 2004

4. The Official Unemployment Rate • Categories • Employed: worked at least one hour in previous week, or temporarily absent from job. • Unemployed: not employed and “actively” looking for work (unprompted categories), or temporarily laid off. • Not in Labor Force (NILF): All others.

5. Evidence for Measurement Error in Labor Force Status (LFS) in the CPS • Re-interview inconsistency • Rotation group bias

6. Re-interview Inconsistency • 1% random sample of original sample of ≈ 50,000 households is re-interviewed monthly (without replacement). • Re-interview occurs in same week as the original interview. • Inconsistent responses suggest measurement error.

7. Re-interview Inconsistency (2001-2003) 8.9% of cases are inconsistently classified.

8. Unemployment Inconsistency (2001-2003)

9. Rotation Group Design

10. Rotation Group Bias (2002 Full CPS)

11. What Could Cause Rotation Group Bias? • Non-response bias: rotation groups may represent different populations. • Differences in interview setting • telephone vs. face-to-face • proxy vs. self • Time in sample effect • Improved understanding of questionnaire • Embarrassment at admitting prolonged unemployment • Interview changes behavior

12. Latent Class Analysis to Test Hypotheses • Sources of Rotation Group Bias • Non-response bias (different populations): Does latent employment status vary by rotation group? • Measurement error: Does rotation group influence error rates? • Differences in setting: • Does interview mode (telephone vs. face-to-face) initial interview influence error rates? • Does interview mode account for apparent rotation group effects on error rates? • Social pressure: • Gender influences latent employment status • Does gender also influence error rates? • Does the effect of rotation group vary by gender?

13. Correlation between Month-in-Sample and Interview Mode

14. Re-interview Data Set • N = 24,297 (un-weighted data) • X = True Labor Force Status (Latent Variable) • A = Observed Labor Force Status at Inititial Interview • B = Observed Labor Force Status as Time 2 (Reinterview)

15. Basic Latent Class Model A X B (with usual constraints for identifiability) X, A|X, B|X Shorthand:

16. Grouping Variable A S X B S, X|S, A|X, B|X

17. External Variable influencing Classification Error A S X M B SM, X|S, A|XM , B|XM

18. Grouping versus External Variables A S X M B SM, X|S, A|XMS {AXM AXS} , B|XMS {BXM BXS}

19. Covariates • S = Gender • Men: 47% • Women: 52% • M = Month in Sample • 1 or 5: 28% • 2-4, 6-8: 72% • T = Interview Mode (Initial Interview) • Telephone: 72% • In Person: 18%

20. Statistical Power & Identifiability Issues • Large total N, but relatively small N for unemployed. • More variables means more identifiable models, but • also diminishing cell counts and boundary solutions.

21. Principles of Model Construction • Always include X|S A|X B|X • Assume 3 latent classes & S as grouping variable • Fit classification table of A*B*M*T*S. • Vary following effects • M as grouping variable • M &/or T affecting classification error for A & B • T affecting A but not B • S affecting A & B when identifiable based on other restrictions (including interaction of M & S)

22. Principles of Model Construction • Try equality constraints • Equal influence of M & or S on error rate for A & B. • Error rate for T at time A = error rate at time B (when T does not affect B).

23. Principles of Model Selection • Limit search to theoretically plausible models. • Limit search to identifiable models. • Overall model fit • P-value of likelihood ratio test vs. saturated model > .01 • Dissimilarity index < .05 • Model selection among those meeting above criteria: • Bayesian information criterion (BIC) • Likelihood ratio test for nested models • Check substantive interpretation within set of possible best models.

24. Best-Fitting Models

25. Estimated Unemployment Rate • Model 1 (similar to other top models) • UE = 4.9% • Observed M.I.S. 1 & 5 • UE = 6.0% • Observed M.I.S. 2-4, 6-8 • UE = 4.7%

26. Conditional Probabilities for Employment Status

27. Conditional Probabilities for A|TX & B|TX

28. Conditional Probabilities for A|MX & B|MX

29. Summary Findings • Change in structural model (treating month-in-sample as grouping variable) does not change the preferred measurement model. • Models fit nearly as well without M as grouping variable; casts doubt on non-response bias hypothesis. • M-I-S bias is not just a function of interview mode. • Covariate effects (esp. S) on response error should be examined further in model with more df; need another grouping variable.

30. Unresolved Issues • Ambiguous results for model selection • Most interested in fit of unemployment classification, but this is overwhelmed in measures of overall fit • Software limitations: clustering, local & boundary solutions, standard errors not consistently output

31. Future Research Agenda • Try finer coding of month-in-sample • Develop models for other variables: age, race, proxy vs. self • Pool more years of data • Develop hypotheses & interpretation based on review of: • experimental work • analyses of non-response • related models including Markov latent class models of employment status transitions

32. Rotation Group Bias (2001-2003, reinterview data)