Census Bureau Seasonal Adjustment Software and Research

1. Census Bureau Seasonal Adjustment Software and Research David.F.Findley@census.gov U S C E N S U S B U R E A U

2. Outline of Talk: Software X-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS” Windows version ( Jurgen Doornik & GiveWin) Supporting software Genhol (holiday regressors) SAS Software: X-12-Graph (14+ types of diagnostic graphs) Interface (simplifies analyses sets of series) X-12-Write (easy prod./modif. of .spc files) X-12-Review (1 page diagnostic summaries)

3. Outline of Talk: Research TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series) • Filters and Filter Diagnostics • Automatic modeling: TRAMO vs. X-12’s “TRAMO” • Revisions State-Space Models using Sampling Error Data Non-Gaussian “Structural” State-Space Models for More Stable Resistance to “Outliers”

4. Statistical Research Division Time Series Group Research and “X-12-ARIMA+” Programming Brian.C.Monsell Kellie.C.Wills William.R.Bell (honorary) David.F.Findley (honorary) Donald.E.Martin (Part-time, Howard University) Trang.Ta.Nguyen (1-year in-house visitor) John.Alexander.Aston (2-year Post-Doc from Imperial College, London) S.J.M. Koopman (Fellow, Free Univ. of Amsterdam)

5. Economic and Statistical Programming DivisionTime Series Methods BranchResearch and SAS, Excel Programming Catherine.C.Hood Kathleen.M.McDonald.Johnson Golam.Farooque Roxanne.Feldpausch

6. Outline of Talk: Software X-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS” Windows version ( Jurgen Doornik & GiveWin) Supporting software Genhol (holiday regressors) SAS Software: X-12-Graph (14+ types of diagnostic graphs) Interface (simplifies analyses sets of series) X-12-Write (easy prod./modif. of .spc files) X-12-Review (1 page diagnostic summaries)

7. X-12-ARIMA Improvements over StatsCan’s X-11-ARIMA • regARIMA models (including outliers, user-defined regressors, etc.) vs. ARIMA models • Much more extensive automatic options for modeling, including trading day, holiday est., additive vs. multiplicative adjustment • More diagnostics (e.g. spectra, revisions) • Specialized output files, e.g. log files for users favorite diagnostics, from many X-12-Graph (SAS, but for non-SAS-users)

8. RegARIMA Models (Forecasts, Backcasts, and Preadjustments) Modeling and Model Comparison Diagnostics and Graphs X-11 Seasonal Adjustment Seasonal Adjustment Diagnostics and Graphs

9. REGARIMA Model transformation ARIMAProcess Regressors for trading day and holiday or calendar effects, additive outliers, temporary changes, level shifts, ramps user-defined effects Leap-year adjustment, or “subjective” strike adjustment, etc.

10. Types of Regression Variables Available in X-12-ARIMA • Outlier and Trend-Change Effects Additive (or Point) Outliers Temporary Change Outliers Level shifts, Ramps • Seasonal Effects Calendar month indicators* Trigonometric Seasonal (Sines-Cosines)* • Calendar Effects Trading Day (Flows or Stocks)* Leap-year February*, Length of Month* Shifting Holidays (e.g. Easter) • Constant Term • User-Defined Effects *Two-regime option available Note: Regression coefficients can be fixed

11. X-12-ARIMA Releases Ver. 0.2.10 July (Statistics Canada options) Ver. 0.3 Summer (TRAMO-type automatic ARIMA model selection) -based on information gleaned from TRAMO code provided by Victor Gomez Ver. 1.0 End of year (Better organized output and manual, more testing etc.)

12. “X-12-ARIMA/SEATS” • Offers both x11{}and seats{ } commands to provide “X-11” or SEATS type seasonal adjustments with X-12-ARIMA diagnostics as well as SEATS diagnostics • Is being updated from SEATS2000 to SEATS2001&2002 (with support from Agustin Maravall and Gianluca Caporello) • Schizophenic (duplicate) output, currently • Distribution for research and testing to statistical agencies and central banks in 2003

13. Diagnoses from X-12-ARIMA/SEATS 1. Spectrum diagnostic reveals source of “Invalid Decomposition” problem

14. X-12-A/SEATS COMMAND FILE series {file="serie.txt" format="tramo"} transform{function=log} outlier{critical=3.7} arima{model=(0 1 1)(0 1 1)} check{} #x11{} seats{}

15. Message from seats{ } run: • NOTE: Spectral plot for the seasonally adjusted series cannot be done when SEATS cannot perform a signal extraction.

16. Parameter Estimate Errors ----------------------------------------------------- Nonseasonal MA Lag 1 0.3846 0.12087 Seasonal MA Lag 12 -0.3665 0.12612

17. 10*LOG(SPECTRUM) of the regARIMA model residuals Spectrum estimated from 1990.Jan to 1995.Oct. +++++++I+++++++++++++++++++++++++++++ -22.11I * I * I * I * * -23.34I * * I * * * I * * T I * * *T -24.57I * * *T I * * *T I * * * *T I * * * *T* -25.81I * * * *T* I * * * *T*

18. series {file="serie.txt" format="tramo"} transform{function=log} outlier{critical=3.7} arima{model=(0 1 1)(0 1 1)} check{} x11{} #seats{}

19. X-12-ARIMA/SEATS Seasonal Adjustment Program Version Number 0.3s Build 24 WARNING: At least one visually significant trading day peak has been found in one or more of the estimated spectra.

20. G.1 10*LOG(SPECTRUM) of the differenced, transformed seasonally adjusted data. Spectrum estimated from 1990.Jan to 1995.Oct. ++++++++++I+++++++++++++++++++++++++++++++++++ I T I T I T -20.10I T I T I T I T -22.01I T I T I T

21. series {file="serie.txt" format="tramo"} transform{function=log} outlier{critical=3.7} arima{model=(0 1 1)(0 1 1)} regression{variables=td} check{} seats{}

22. X-12-ARIMA/SEATS Seasonal Adjustment Program Version Number 0.3s Build 24 Reading input spec file from metalss.spc Storing any program output into metalss.out Storing any program error messages into metalss.err WARNING: At least one visually significant seasonal peak has been found in one or more of the estimated spectra.

23. Standard Parameter Estimate Errors ----------------------------------------------------- Nonseasonal MA Lag 1 0.1995 0.12871 Seasonal MA Lag 12 0.3843 0.15795

24. X-12-ARIMA Diagnoses for SEATS 2. T/S Practice of adding outliers to improve kurtosis, etc. can substantially increase the size of revisions of the initial seasonal adjustments: Example (from Catherine Hood) US Exports of Passenger Cars: History diagnostic shows cost to revisions of adding outlier regressors to reduce kurtosis

27. Outline of Talk: Software X-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS” Windows version ( Jurgen Doornik & GiveWin) Supporting software Genhol (holiday regressors) SAS Utilities: X-12-Graph (14+ types of diagnostic graphs) Interface (simplifies analyses of many series) X-12-Write (easy prod./modif. of .spc files) X-12-Review (1 page diagnostic summaries)

28. Genhol • From holiday date file: Generates regressormatrices and associated command files to enable X-12-ARIMA estimation of complex moving holiday effects (e.g. for Easter, Ramadan, etc.). • Regressors for up to three intervals: • before-the-holiday interval • surrounding-the-holiday interval • past-the-holiday interval (“recovery” interval)

29. Proportionality Regressors: An Example • Assume : • An effect interval is 10 days long, and this year 2 of its days fall in January and 8 in February. The interval regressor’s values for this year will be: • 0.2 in January • 0.8 in February • 0.0 for the rest of the year

30. Interface Program (SAS): for seasonal adjustment of sets of series Example: Seasonally Adjusted Total U.S. Imports = sum of 140 component series, c. 80% of which are seasonally adjusted. What is the effect on the month-to-month changes and quality diagnostics of the S. A. Total Imports if the seasonal adjustment options are changed for 5 of the component series?

31. Outline of Talk: Research TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series) • Filter Diagnostics • Automatic modeling: TRAMO vs. X-12’s “TRAMO” • Revisions

32. Filters and Filter Diagnostics • Filter (spectral) diagnostics needed • To understand limitations/issues with short series (finite filter diagnostics, also for concurrent adjustments, trends) • To decide between closely competitive models Paper by David Findley and Donald Martin.

34. Outline of Talk: Research TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series) • Automatic modeling: TRAMO vs. X-12’s “TRAMO” • Accuracy • Results from simulated series • Revisions • Results from Census Bureau series

35. ESMPD’s Automatic Modeling Study • First presented at the International Forecasters Symposium, June 2001 • Continuation of this work to appear at the ASA meetings, August 2002, in a paper by Kathleen McDonald-Johnson, et al.

36. Series 306 time series from the US Census Bureau’s Import/Export series and Retail Sales

37. Results • 88 series (29%) with same regARIMA model • 27 series (9%) with same differencing and same regressors but different ARMA choices • 123 series (40%) with same differencing, but different regressors • 32 series (10%) with different nonseasonal differencing order (but sometimes offset by a constant) • 36 series (12%) with different seasonal differencing order

38. Conclusions • TRAMO’s weakness is the procedure for deciding about trading day modeling • TRAMO developers are aware of our results • X-12-ARIMA has a problem with choosing nonparsimonious models • Monsell has already implemented some changes, including a unit root test.

39. Why Are Different Models Chosen? • Model estimation method is different • TRAMO : Hannan-Rissanen and m.l.e conditional on AR part of model • X-12-ARIMA : Exact MLE • Model residuals are different, which can lead to different choices of outliers • Outlier procedure itself is different • TRAMO removes insignificant outliers after each iteration • TRAMO uses approximate BIC

40. Accuracy: X-12-ARIMA vs T/S (ESMPD) • Results from 54 simulated series were first presented at the ASA meetings, August 2000 • Continuation of the first SEATS studies, beginning in 1997

41. The Simulated Series • Fifty-four series • Six different trends – three from SEATS and three from X-12 • Six different seasonal factors – three from SEATS and three from X-12 • Irregular sampled from three sets of irregular factors combined from SEATS and X-12

42. Results of Accuracy Study • SEATS performed better on the majority of series with large irregulars if the series are 9+ years long, but most adjustments werenot acceptable. • Both programs did better than expected on the short series, but X-12-ARIMA adjustments were usually better than SEATS adjustments on series 4-7 years long

43. Revisions: X-12-ARIMA vs T\S • New ESMPD study using X-12-SEATS on Census series. “Final” results will be presented at the ASA meetings, August 2002. • Can we identify characteristics in the series that will indicate if its “linearized” series will be a better candidate for a model-based adjustment than for an X-11 filter adjustment or vice versa?

44. Methods • Use X-12-SEATS to get revision diagnostics from both an X-11/X-12-type adjustment and a SEATS adjustment • Used TRAMO to get the ARIMA model, and then used either an x11 or a seats “spec”

45. Very Preliminary Results • 260 US Import/Export series • Only a very small subset (18 series) where we can see definite differences in the revision diagnostics for the seasonal adjustment

46. An Observation: Series with • Large revisions in X-12 and smaller revisions in SEATS had generally large values for 12 (most greater than 0.95) and values for X-12’s I/S ratio < 5. • Large revisions in SEATS and smaller revisions in X-12 had generally 0.4 < 12 < 0.6 and values for I/S > 6. In both cases, smaller revisions are associated with more constant seasonal factor estimates

47. Next Steps • Look at more series • Look at more diagnostics/characteristics of the series to try to find patterns, not just revisions

48. Outline of Talk: Research Projects almost ready to yield results: State-Space Models using Sampling Error Data Non-Gaussian “Structural” State-Space Models for More Stable Resistance to “Outliers”

49. State-Space Models with Sampling Error Statistics: Bell and Nguyen 100+ Disaggregate Construction series with “high” sampling error variances Consider model-based adjustment with regARIMA+observation error models that incorporate sampling error variance and autocovariance estimates to achieve acceptable or better seasonal adj’s. (Need state-space for model est. & seas adj.)

50. Non-Gaussian “Structural” State-Space Models for More Stable Resistance to “Outliers”Koopman and Aston X-12-ARIMA and T\S use outlier regressors identified by t-statistics and critical values. Identifications can change as new data arrive, causing seasonal adjustment revisions. Use heavy tailed non-Gaussian models instead of critical values. (Hard to estimate such models, simplest for “Harvey’s structural models”)