FRONTIERS OF REAL-TIME DATA ANALYSIS

1. FRONTIERS OF REAL-TIME DATA ANALYSIS Dean Croushore Associate Professor, University of Richmond Interim Director, Real-Time Data Research Center, Federal Reserve Bank of Philadelphia October 2008

2. Introduction • First paper to do real-time data analysis: • Gartaganis-Goldberger, Econometrica (1955) • Statistical properties of the statistical discrepancy between GNP and gross national income changed after data were revised in 1954

3. Research Categories • Data Revisions • Forecasting • Monetary Policy • Macroeconomic Research • Current Analysis

4. Introduction • Data sets • Real-Time Data Set for Macroeconomists • Philadelphia Fed + University of Richmond • Need for good institutional support • Club good: non-rival but excludable

5. Introduction • Data sets • Unrestricted access: • U.S.: Philadelphia Fed, St. Louis Fed, BEA • OECD • Bank of England (recently updated) • Restricted access: • EABCN • Fate unclear: • Canada • One-time research projects: • Many, most not continuously updated

6. Introduction • Analysis of data revisions is not criticism of government statistical agencies! • May help agencies improve data production process • Revisions reflect limited resources devoted to data collection • Revised data usually superior to unrevised data (U.S. CPI vs. PCE price index)

7. Introduction • Structure of data sets • The data matrix • Columns report vintages (dates on which data series are observed) • Rows report dates for which economic activity is measured • Moving across rows shows revisions • Main diagonal shows initial releases • Huge jumps in numbers indicate benchmark revisions with base year changes

8. REAL OUTPUT Vintage: 11/65 2/66 5/66 . . . 11/07 2/08 Date 47Q1 306.4 306.4 306.4 . . . 1570.5 1570.5 47Q2 309.0 309.0 309.0 . . . 1568.7 1568.7 47Q3 309.6 309.6 309.6 . . . 1568.0 1568.0 . . . . . . . . . . . . . . . . . . . . . 65Q3 609.1 613.0 613.0 . . . 3214.1 3214.1 65Q4 NA 621.7 624.4 . . . 3291.8 3291.8 66Q1 NA NA 633.8 . . . 3372.3 3372.3 . . . . . . . . . . . . . . . . . . . . . 07Q1 NA NA NA . . . 11412.6 11412.6 07Q2 NA NA NA . . . 11520.1 11520.1 07Q3 NA NA NA . . . 11630.7 11658.9 07Q4 NA NA NA . . . NA 11677.4

9. Data Revisions

10. Data Revisions • What Do Data Revisions Look Like? • Are They News or Noise? • Is the Government Using Information Efficiently? • Are Revisions Forecastable? • How Should We Model Data Revisions? • Key issue: are data revisions large enough economically to worry about?

11. Data Revisions • What Do Data Revisions Look Like? • Short Term (example) • Long Term (example) • What Do Different Types of Data Revisions Look Like? • Short run revisions based on additional source data • Benchmark revisions based on structural changes or updating base year

14. Data Revisions • Are Data Revisions News or Noise? • Data Revisions Add News: Data are optimal forecasts, so revisions are orthogonal to early data; revisions are not forecastable • Data Revisions Reduce Noise: Data are measured with error, so revisions are orthogonal to final data; revisions are forecastable

15. Data Revisions • Are Data Revisions News or Noise? • Mankiw-Runkle-Shapiro (1984): Money data revisions reduce noise • Mankiw-Shapiro (1986): GDP data revisions contain news • Mork (1987): GMM results show “final” NIPA data contain news; other vintages are inefficient and neither noise nor noise • UK: Patterson-Heravi (1991): revisions to most components of GDP reduce noise

16. Data Revisions • Is the Government Using Information Efficiently? • Theoretical Issue: Should the government report its sample information or project an unbiased estimate using extraneous information?

17. Data Revisions • Is the Government Using Information Efficiently? • Key Issue: What is the trade-off the government faces between timeliness and accuracy? • Zarnowitz (1982): evaluates quality of different series • McNees (1989): found within-quarter estimate of GDP to be as accurate as estimate released 15 days after quarter end

18. Data Revisions • Findings of bias and inefficiency based on ex-post tests • UK: Garratt-Vahey (2003) • US: Aruoba (2008)

19. Data Revisions • Findings of bias and inefficiency of seasonally revised data • Kavajecz-Collins (1995) • Swanson-Ghysels-Callan (1999) • Revisions to seasonals may be larger than revisions to NSA data: Fixler-Grimm-Lee (2003) • Key question: Are seasonal revisions predictable? Who cares if that is an artifact of construction?

20. Data Revisions • Key Issue: If early government data are projections, then state of business cycle may be related to later data revisions. • Dynan-Elmendorf (2001): GDP is misleading at turning points • Swanson-van Dijk (2004): volatility of revisions to industrial production and producer prices increases in recessions

21. Data Revisions • Are Revisions Forecastable? • Conrad-Corrado (1979): use Kalman filter to improve government’s monthly data on retail sales • Aruoba (2008): revisions to many U.S. variables are forecastable

22. Data Revisions • Are Revisions Forecastable? • Key Issue: can revisions be forecast in real-time (or just ex-post)? • Guerrero (1993): combines historical data with preliminary data on Mexican industrial production to get improved estimates of final data • Faust-Rogers-Wright (2005): Examines G-7 countries’ output forecasts; find Japan & U.K. output revisions forecastable in real time

23. Data Revisions • How Should We Model Data Revisions? • Howrey (1978) • Conrad-Corrado (1979) • UK: Holden-Peel (1982) • Harvey-McKenzie-Blake-Desai (1983) • UK: Patterson (1995) • UK: Kapetanios-Yates (2004)

24. Data Revisions • How Should We Model Data Revisions? • Is there any scope for new research here? • Show predictability between different vintages to help data agencies improve methods • Ex: US data on PCE inflation

25. Forecasting

26. Forecasting • Forecasts are only as good as the data behind them • Literature focuses on model development: trying to build a better forecasting model, especially comparing forecasts from a new model to other models or to forecasts made in real time • Details: Croushore (2006) Handbook of Economic Forecasting

27. Forecasting • Does the fact that data are revised matter significantly (in an economic sense) for forecasts?

28. Forecasting • EXAMPLE: THE INDEX OF LEADING INDICATORS • Leading indicators: seem to predict recessions quite well. • But did they do so in real time? The evidence suggests skepticism. • Diebold and Rudebusch (1991) investigated the issue, using real-time data • Their conclusion: The leading indicators do not lead and they do not indicate! • The use of revised data gives a misleading picture of the forecasting ability of the leading indicators.

30. Forecasting • EXAMPLE: THE INDEX OF LEADING INDICATORS • Chart shows not much problem • But recession started in November 1973 • Subsequently, leading indicators were revised & ex-post they do much better

32. Forecasting • Why Are Forecasts Affected by Data Revisions? • Change in data input into model • Change in estimated coefficients • Change in model itself (number of lags) • See experiments in Stark-Croushore (2002)

33. Forecasting • What Do We Use as Actuals? • Answer: Depends on purpose • Best measures are probably latest-available data for “truth” (though perhaps not in fixed-weighting era) • But forecasters would not anticipate redefinitions and generally forecast to be consistent with government data methods (example: pre-chain-weighting period; 2013 capitalization of R&D)

34. Forecasting • What Do We Use as Actuals? • Real-Time Data Set: many choices • first release (or second, or third) • four quarters later (or eight or twelve) • Date of annual revision (July for U.S. data) • last benchmark (the last vintage before a benchmark revision) • latest available

35. Forecasting • How Should Forecasts Be Made When Data Are Revised? • Key issue: temptation to cheat! • Try method; it doesn’t work; but that’s because of one outlier; dummy out that observation; the method works! • If data are not available, use a real-time proxy, don’t peak at future data • Cheating is inherent because you know the history already

36. Forecasting • Forecasting with Real-Time versus Latest-Available Data • Denton-Kuiper (1965): first to compare forecasts with real-time vs revised data • Cole (1969): data errors reduce forecast efficiency & may lead to biased forecasts • Trivellato-Rettore (1986): data errors in a simultaneous equations model affect everything: estimated coefficients and forecasts; but for small model of Italian economy, addition to forecast errors were not large

37. Forecasting • Forecasting with Real-Time versus Latest-Available Data • Faust-Rogers-Wright (2003): research showing forecastability of exchange rates depended on a particular vintage of data; other vintages show no forecastability • Molodtsova (2007): combining real-time data with Taylor rule allows predictability of exchange rate • Moldtsova-Nikolsko-Rzhevskyy-Papell (2007): dollar/mark exchange rate predictable only with real-time data

38. Forecasting • Summary: for forecasting, sometimes data vintage matters, other times it doesn’t

39. Forecasting • Levels versus Growth Rates • Howrey (1996): level forecasts of GNP more sensitive than growth forecasts; so policy should feed back on growth rates, not levels • Kozicki (2002): choice of forecasting with real-time or latest-available data is important for variables with large level revisions

40. Forecasting • Model Selection and Specification • Swanson-White (1997): explores model selection • Robertson-Tallman (1998): real-time data affect model specification for industrial production but not for GDP • Harrison-Kapetanios-Yates (2005): it may be optimal to estimate a model without using most recent preliminary data • Summary: model choice is sometimes affected by data revisions

41. Forecasting • Evidence on Predictive Content of Variables • Croushore (2005): consumer confidence indicators have no predictive power in real time, even when they appear to when using latest-available data

42. Forecasting • Optimal Forecasting When Data Are Subject to Revision • Howrey (1978): adjusts for differing degrees of revision using Kalman filter; in forecasting, use recent data but filter it • Harvey-McKenzie-Blake-Desai (1983): use state-space methods with missing observations to account for irregular data revisions: large gain in forecast efficiency compared with ignoring data revisions

43. Forecasting • Optimal Forecasting When Data Are Subject to Revision • Howrey (1984): use of state-space models to improve forecasts of inventory investment yields little improvement • Patterson (2003): illustrates how to combine measurement process with data generation process to improve forecasts for income & consumption

44. Forecasting • Optimal Forecasting When Data Are Subject to Revision • What information set to use? • Koenig-Dolmas-Piger (2003), Kishor-Koenig (2005): focus on diagonals to improve forecasting; treat data the same that have been revised to the same degree

45. Forecasting • Optimal Forecasting When Data Are Subject to Revision • Summary: There are sometimes gains to accounting for data revisions; but predictability of revisions (today for US data) is small relative to forecast error (mainly seasonal adjustment)

46. Forecasting • A Troublesome Issue • Specifying a process for data revisions • Some papers specify an AR process • But research on revisions suggests that benchmark revisions are not so easily characterized

47. Forecasting • Key Issue: What are the costs and benefits of dealing with real-time data issues versus other forecasting issues?

48. Monetary Policy

49. Monetary Policy: Data Revisions • How Much Does It Matter for Monetary Policy that Data Are Revised? • How Misleading Is Monetary Policy Analysis Based on Final Data Instead of Real-Time Data? • How Should Monetary Policymakers Handle Data Uncertainty?

50. Monetary Policy: Data Revisions • How Much Does It Matter for Monetary Policy that Data Are Revised? • Example: Fed’s favorite inflation measure is the Personal Consumption Expenditures Price Index Excluding Food & Energy Prices (PCEPIXFE) • But it has been revised substantially