State Space Modelling For UK LFS Unemployment

1. State Space Modelling For UK LFS Unemployment Gary Brown, Ping Zong Time Series Analysis Branch Office for National Statistics Jan Angenendt   Knowledge, Analysis and Intelligence HM Revenue and Customs Moshe Feder Southampton Statistical Sciences Research Institute (S3RI) University of Southampton

2. Overview • Introduction • LFS Rolling Quarterly Data • State Space Model • The General State Space Model • The Specific Model Proposed for UK LFS • Results • Further Work

3. Introduction • A State Space Model (SSM) represents a structural time series approach to capturing the characteristics of a time series. • Similarly to X-12-ARIMA, a time series can be decomposed into trend, seasonal and irregular using SSM. • A key advantage of using SSM: • allows explicitly modelling of unobservable components

4. Aims of the SSM Project • Currently the UK LFS publishes a single estimate for each rolling quarter, based on a rotating panel design with five waves of interviews • The aim of the SSM project is to model the complex LFS structure by fitting wave-specific rolling quarter data, to better account for: • sampling error autocorrelation (between wave-specific estimates) • rotation group bias (systematic differences between wave-specific estimates)

5. LFS Sample Design • The LFS sample size, around 120,000 people in 40,000 households, is split into 190 Interviewer Areas (IAs). • Each IA is split into 13 weekly ‘stints’ – in this way a representative sample is achieved every 13 weeks. • To weight the sample, the 13 weeks are allocated to ‘months’ in a 4-4-5 pattern.

6. LFS Sample Design (cont) • Interviews are (approximately) split by mode: • First interview – face-to-face • Second interview (13 weeks later) – telephone • Third, fourth and fifth interviews (each 13 weeks after the previous) – telephone. • After the fifth interview (wave 5) households drop out of the survey and are replaced with a new set of households (wave 1).

7. Data Structure Table 1: Rolling quarterly estimates

8. Rolling quarterly estimate • Each three months yields a representative sample, and each month is in three of these • For example, survey responses from June are included in three rolling quarterly estimates: (April,May,Jun), (May,Jun,Jul), (Jun,Jul,Aug) • Given this structure, the overall unemployment quarterly estimate at time t is a combination of three months: • where ‘Y’ = unemployment rate, ‘ILO’ = ILO unemployed, and ‘EA’ = economically active.

9. Table 2: Sample rotation in wave-specific data

10. Sample rotation The sample rotation means: • The same wave does not include the same households (samples) in each rolling quarter. • The same households (cohort) appear in different waves after one quarter. • There are different data collection methods in waves. These characteristics need to be accounted for. • Using the SSM approach for UK LFS unemployment enables this to happen

11. State space model • The General State Space Model (GSSM) is: (1) (2) • where: • yt is the measurement equation • at is the state vector (the transition equation) • Z, T, H and Q are matrices • et and ht are error terms Compare the General Linear Regression Model (GLRM) (3)

12. Comparing SSM and GLRM • In (1) and (3), y is a function of time, but • GLRM: the coefficient is a which is fixed • SSM: the coefficient is atwhich will vary over time • Hence, GLRM is a static regression model and SSM is a dynamic regression model • In the SSM, each coefficient atvaries according to a random walk at = at-1 + htwhich gives a state vector in the form at = Tat-1 + htas in Equation (2). • So, equation (1) expresses the dynamic regression process, and equation (2) expresses the dynamic change condition

13. SSM with Signal and Noise The SSM model can be expressed as two parts: • signal qt • noise et (4) where: • yt is the design unbiased survey estimate • qt is signal - the unknown population quantity • et is noise - the survey errors

14. Signal qt - Basic Structural Model (BSM) (4a) (4b) (4c) (4d) if using dummy seasonality • where: • Lt is the level • Rt is the slope • St is the seasonal • hL,t, hR,t, hS,t are white noise terms

15. Noise et - the Extended SSM model (5) • where: • f is the coefficient of AR process • et-j is the sampling error • he,t is white noise The standard assumption is independence of errors

16. BSM + Extended SSM • Both signal and noise have their own measurement equation (ZBSM,t and Ze,t) and state vector (aBSM,tandae,t) in the transition equation, ie - measurement equation for signal - state vector for signal - measurement equation for noise - state vector for noise if AR(4) • These two parts, signal and noise, are brought together to form a completed SSM model.

17. The Specific SSM Proposed for UK LFS State Vector (Lt, Rt, St, et) • As survey responses from month t are included in estimates based on three representative samples centred at (t-1,t,t+1), the state vector will not only consider parameters at time t but will take all three time periods (t-1,t,t+1) into account • All the original Lt, Rt, St and et will include for level for slope for seasonal for sample error

18. State vector (Lt, Rt, St, et) - cont • The slope (R) will be the same at three different levels, so is kept at the t+1 value. • Also because there are five waves, and each wave includes three time periods (t-1, t, t+1) for sample error, there are in total 15 sample error state variables in our model. • Total = 30 state variables for the state vector.

19. Survey error structure (et) • Survey errors do not overlap in the wave structure data but do appear between two quarter across waves (cohorts), ie • someone interviewed in wave i at time t will be interviewed in wave (i+1) at time t+3 – the sample errors will correlated so are defined as follows. (6)

20. Building the SSM for UK LFS • Matrices are used as the basic method for building the SSM • The main SSM matrices/vectors for UK LFS are: • observation matrices (Z) • transition matrices (T) • covariance matrices (Q) • state vectors (at) • disturbance vectors

21. Observation matrices (Z) • Observation matrices: • ZBSM (signal) is a 5x15 matrix for SSM with dummy seasonality • Ze (noise) is a 5x15 matrix

22. State vectors (at, at-1) and transition matrix (T): BSM • State vectors (aBSM,t), transition matrix (TBSM) + disturbance

23. State vectors (at, at-1) and transition matrix (T): e • State vectors (ae,t), transition matrix (Te) + disturbance

24. Join Signal and Noise - block all matrices together • Observation matrices: • State vectors: • Disturbance vectors: • Transition matrices: where TBSM is the 15x15 matrix and Te is the 15x15 matrix with

25. Covariance matrices (Q) where with and with

26. Covariance matrices (Q) – cont. and with

27. The Model Estimate Setting • As long as all SSM matrices are set appropriately and all parameters in the model are known, the state vector can be predicted, filtered and smoothed using the Kalman Filter • In fact, all these parameters are unknown, thus we need initialisation of all parameters: • (at-1) in the state vector • in the disturbance matrix • AR parameters (r and r*) in the transition matrix

28. Initialisation for (at-1) in the state vector • For non-stationary components: • initialised the non-stationary components mean by zero • initialised the associated non-stationary component variances with a very large value (ie 10000) • For stationary components: • initialised the stationary component (sampling error et) mean with unconditional mean • initialised the stationary component variance with its own pseudo-error variance

29. Initialisation for (ht) in the disturbance vector • All in the disturbance vector can be estimated using Maximum Likelihood in the model. • There are two approaches to estimating these parameters: • The hyper-parameters approach assumes that all parameters are unknown and are estimated simultaneously in the SSM model. • The pseudo-error approach is different ...

30. Initialisation in the pseudo-error approach Different approaches for different parameters • hL,t, hR,t,hS,t are treated as unknown parameters, and he,t is treated as a known parameter (estimated in a separate process) • Initialised variance value for the unknown parameter vector (in Q matrices) • are set based on a separate estimation process (‘Proc ucm’ in SAS, ‘StructTS’ in DLM/R) • obtained based on calculation of the autocorrelation through the pseudo-error process • AR parameters, r and r*, are estimated using Yule-Walker equations and substituted into SSM

31. Simulation results 1. Trend prediction:

32. Seasonality prediction

33. Sample error prediction

34. Further work The project is not complete – work remaining: • test whether including (t-1, t, t+1) into the model is necessary (through comparison analysis) • test whether the proposed method for sample error estimation is correct • test a consistent approach with one used in SSM to estimate the AR(1) coefficients of the pseudo-error • consider MA models • consider including rotating group bias and the claimant count

35. Any questions? ping.zong@ons.gsi.gov.uk

36. Appendix • Trigonometric seasonality model was using sines and cosines. where E[ ]=0, E[ ]=0, Var[ ]= Var[ ]= and for j = 1,...,6.

37. The observation matrix (ZBSM)(5 x17 matrix) • Total sate vector ( ) - 32 variables:

38. The transition matrix (TBSM) (5 x17 matrix)