UNECE Workshop on Confidentiality Manchester, 17.-19. December 2007

1. Comparing Fully and Partially Synthetic Data Sets for Statistical Disclosure Control in the German IAB Establishment Panel Jörg Drechsler, Stefan Bender (Institute for Employment Research, Germany)& Susanne Rässler (University of Bamberg) UNECE Workshop on Confidentiality Manchester, 17.-19. December 2007

2. Overview • Multiple Imputation for Statistical Disclosure Control • The IAB Establishment Panel • Application of The Two Approaches • Comparison of The Results • Conclusion

3. Fully synthetic data sets (Rubin 1993) X • advantages: - data are fully synthetic • - re-identification of single units almost impossible - all variables are still fully available • disadvantages: - strong dependence on the imputation model • - setting up a model might be difficult/impossible Ynot observed Ysynthetisch Ysynthetisch Ysynthetisch Ysynthetisch Ysynthetic Yobserved

4. Partially synthetic data sets (Little 1993) • only potentially identifying or sensitive variables are replaced

5. Partially synthetic data sets (Little 1993) • only potentially identifying or sensitive variables are replaced

6. Partially synthetic data sets (Little 1993) • only potentially identifying or sensitive variables are replaced • advantages: - model dependence decreases • - models are easier to set up • disadvantages: - true values remain in the data set • - disclosure might still be possible

7. Overview • Multiple Imputation for Statistical Disclosure Control • The IAB Establishment Panel • Application of The Two Approaches • Comparison of the Results • Conclusions

8. The IAB Establishment Panel • Annually conducted Establishment Survey • Since 1993 in Western Germany, since 1996 in Eastern Germany • Population: All establishments with at least one employee covered by social security • Source: Official Employment Statistics • Response rate of repeatedly interviewed establishments more than 80% • Sample of more than 16.000 establishments in the last wave • Contents: employment structure, changes in employment, business policies, investment, training, remuneration, working hours, collective wage agreements, works councils

9. Overview • Multiple Imputation for Statistical Disclosure Control • The IAB Establishment Panel • Application of the Two Approaches • Comparison of the Results • Conclusions

10. Generating fully synthetic data sets for the IAB Establishment Panel • Create a synthetic data set for selected variables from the wave 1997 from the Establishment Panel • Draw 10 new sample from the Official Employment Statistics using the same sampling design as for the Establishment Panel (Stratification by industry, size, and region) • The number of observations in each sample equals the number of observations in the panel ns=np=7332 • Every sample is imputed ten times using sequential regression • Number of variables from the establishment panel: 48 • Imputations are generated using IVEware by Raghunathan, Solenberger and Hoewyk (2001)

11. Imputation procedure for partially synthetic data • Only two variables are synthesized: - number of employees • - industry (16 categories) • Same variables for the imputation models • Imputation by sequential regression • Imputation model: - multinomial logit for the industry • - linear model for the cubic root of the nb of employees • - 4 independent linear models defined by quartiles for the establishment size • Imputations based on own coding in R.

12. Overview • Multiple Imputation for Statistical Disclosure Control • The IAB Establishment Panel • Application of The Two Approaches • Comparison of the Results • Conclusion

13. Analytical validity • Compare regression results from the original data with results from the synthetic data • First regression: • Zwick (2005) analyses the productivity effects of different continuing vocational training forms in Germany • Probit regression to explain, why firms offer vocational training • 13 Explanatory variables including: Share of qualified employees, establishment size, industry, collective wage agreement, high qualification needs expected… • Second regression: • Log(number of employees) on 15 industry dummies • Two data utility measures: • - Comparison of the beta coefficients from the original data set and the synthetic data sets • - confidence interval overlap

14. Suggested by Karr et al. (2006) Measure the overlap of CIs from the original data and CIs from the synthetic data The higher the overlap, the higher the data utility Compute the average relative CI overlap for any Confidence interval overlap CI for the synthetic data CI for the original data

15. Significant at the 0,1 % level Significant at the 1 % level Significant at the 5 % level Results from the first regression (Zwick 2005)

16. Average confidence interval (CI) overlap for the estimates from the first regression 0,808 0,926 Average overlap

17. Results from the second regression (log(nb. of employees) on industry) = Significant at the 0,1 % level = Significant at the 1 % level = Significant at the 5 % level = insignificant

18. Average confidence interval (CI) overlap for the estimates from the second regression 0,699 0,839 Average overlap

19. Disclosure risk • Difficult to compare between partially and fully synthetic data sets • Disclosure risk is low for fully synthetic data sets, although not zero • DR is higher for partially synthetic data sets, because: • True values remain in the data set • Only survey respondents are included • For partially synthetic data sets a careful disclosure risk evaluation is necessary

20. Overview • Multiple Imputation for Statistical Disclosure Control • The IAB Establishment Panel • Application of The Two Approaches • Comparison of the Results • Conclusions

21. Conclusions • Generating synthetic data sets can be a useful method for SDC • Advantages for partially synthetic data sets: • Higher data validity • Imputation models easier to set up • Lower risk of biased imputations • Disadvantages for partially synthetic data sets: • Higher risk of disclosure • Careful disclosure risk evaluation necessary • Agencies will have to decide depending on the complexity of the survey and the expected risk of disclosure

22. Thank you for your attention