Comparative Analysis of Calibration Estimators in Job Vacancy Surveys

1. How similar are different calibration estimators in the presence of a zero-inflated auxiliary variable? Evidence from the German job vacancy survey Hans Kiesl Institute for Employment Research (IAB), Germany hans.kiesl@iab.de NTTS 2009 – New Techniques and Technologies for Statistics Brussels • February 18-20, 2009

2. Background • Regulation (EC) No. 453/2008 of the European Parliament and of the Council of 23 April 2008 on quarterly statistics on Community job vacancies • Member states have to provide • quarterly data on job vacancies (broken down to NACE section level) • quality reports In Germany, the data will be provided by the IAB.

3. Background (2) • Information on job vacancies in Germany • Business units might report job vacancies to the Federal Employment Agency • Federal Employment Agency publishes monthly statistics on number of registered job vacancies (by NACE-sector) • Since 1989, IAB conducts a yearly (4th quarter) sample survey among business units to estimate number of job vacancies (registered or not) and to get additional information (e.g. about recruiting strategies) • Mail questionnaire (8 pages in length); voluntary • CATI interviews in quarters 1 - 3

4. Basic estimation strategy • stratified simple random sampling (by size classes and industry sector) • calculate design weights as inverse (realized) sampling rate within each stratum • calibrate design weights to known totals from external data • number of business units by size • number of business units by industry sector • number of employees by size • number of employees by industry sector • number of registered vacancies by industry sector

5. Calibration estimators (1) RAKCON • raking estimator with weight restrictions • within each stratum only two different weights allowed • units with vacancies, units without vacancies • reason: control variance of weights and variance of estimates • start with design weights and repeat following two steps until convergence of weights: • proportional fitting of weights for units with vacancies to number of registered vacancies by sector • iterative proportional fitting of all weights to number of units by size and by sector

6. Calibration estimators (2) Generalized regression estimator (GREG) • minimizes so that • GREG1 calibrated to • number of units by size • number of units by sector • number of registered vacancies by sector • GREG2 additionally calibrated to • number of employees by size • number of employees by sector

7. Calibration estimators (3) Generalized regression estimator (GREG) with weight restrictions • GREGCON1: N1 = set of units with vacancies • GREGCON2: N1 = set of units with registered vacancies

8. Result of different calibration estimators • 4th quarter 2007, Germany (west) • realized sample size: 7,485 (response rate: 20%)

9. Highly skewed distribution of job vacancies % of 0’s 91% 97% 86% 96% 77% 91% 68% 86% 48% 75% 36% 71% size

10. Simulation study • Create synthetic population by sampling with replacement from original sample • Draw 300 samples from synthetic population with same sampling design and realized sample sizes as original sample • Calculate all estimators described above • Repeat for different nonresponse models • RHG1: equal response probability within strata • RHG2: equal response probabilities within two group (units with and without vacancies) in every stratum • RHG3: equal response probabilities within two group (units with and without registered vacancies) in every stratum

11. Sampling distributions under RHG 1

12. Sampling distributions under RHG 2

13. Sampling distributions under RHG 3

14. Two step GREG estimation • If we accept RHG2, unconstrained GREG is biased. • No information in the frame or among non-responding units to directly estimate the response probabilities. • Suggestion: two step GREG estimation. • First step: GREG estimation, calibrating to registered vacancies • Using the calibrated weights, we can get estimates for response probabilities. • Second step: adjust design weights for different response probabilities, add another GREG estimation step

15. How do we estimate response probabilities? population 1st stage: equal inclusion probabilities sample (model RHG 2) respondents

16. Sampling distributions under RHG 1

17. Sampling distributions under RHG 2

18. Sampling distributions under RHG 3

19. Conclusions • Weight restrictions lead to larger variance of estimators. • Calibration estimators work under an implicit nonresponse model. • Two step GREG estimator applicable if • theory suggests certain response homogeneity groups, • there is no complete information about RHG membership in the frame or among the non-responding units, • the only information is an auxiliary variable applicable for calibration which identifies part of the RHG group. • Special case: existence of a zero-inflated calibration variable with the property that units with a value greater than zero are in the same RHG, but units with a value of zero might be in different RHGs.

20. Thank you very much for your attention! NTTS 2009 – New Techniques and Technologies for Statistics Brussels • February 18-20, 2009