The Spectral Representation of Stationary Time Series

1. The Spectral Representation of Stationary Time Series

2. Stationary time series satisfy the properties: • Constant mean (E(xt) = m) • Constant variance (Var(xt) = s2) • Correlation between two observations (xt, xt + h) dependent only on the distance h. • These properties ensure the periodic nature of a stationary time series

3. Recall and X1, X1, … , Xk and Y1, Y2, … , Yk are independent independent random variables with is a stationary Time series where l1, l2, … lk are k values in (0,p)

4. We can give it a non-zero mean, m, by adding mto the equation With this time series and

5. We now try to extend this example to a wider class of time series which turns out to be the complete set of weakly stationary time series. In this case the collection of frequencies may even vary over a continuous range of frequencies [0,p].

6. The Riemann integral The Riemann-Stiltjes integral If F is continuous with derivative f then: If F is is a step function with jumps pi at xi then:

7. First, we are going to develop the concept of integration with respect to a stochastic process. Let {U(l): l[0,p]} denote a stochastic process with mean 0 and independent increments; that is E{[U(l2) - U(l1)][U(l4) - U(l3)]} = 0 for 0 ≤ l1 < l2 ≤ l3 < l4 ≤ p. and E[U(l) ] = 0 for 0 ≤ l ≤ p.

8. In addition let G(l) =E[U2(l) ] for 0 ≤ l ≤ p and assume G(0) = 0. It is easy to show that G(l) is monotonically non decreasing. i.e. G(l1) ≤ G(l2) for l1 < l2 .

9. Now let us define: analogous to the Riemann-Stieltjes integral

10. Let 0 = l0 < l1 < l2 < ... < ln = p be any partition of the interval. Let . Let lidenote any value in the interval [li-1,li] Consider: Suppose that and there exists a random variable V such that *

11. Then V is denoted by:

12. Properties:

13. The Spectral Representation of Stationary Time Series

14. Let {X(l): l [0,p]} and {Y(l): l [0,p]} denote a uncorrelated stochastic process with mean 0 and independent increments. Also let F(l) =E[X2(l) ] =E[Y2(l) ] for 0 ≤ l≤ p and F(0) = 0. Now define the time series {xt : tT}as follows:

15. Then

16. Also

19. Thus the time series {xt : tT} defined as follows: is a stationary time series with: F(l) is called the spectral distribution function: If f(l) = Fˊ(l) is called then is called the spectral density function:

20. Note The spectral distribution function, F(l), and spectral density function, f(l) describe how the variance of xt is distributed over the frequencies in the interval [0,p]

21. The autocovariance function, s(h), can be computed from the spectral density function, f(l), as follows: Also the spectral density function, f(l), can be computed from the autocovariance function, s(h), as follows:

22. Example: Let {ut : tT} be identically distributed and uncorrelated with mean zero (a white noise series). Thus and

23. Graph:

24. Example: Suppose X1, X1, … , Xk and Y1, Y2, … , Yk are independent independent random variables with Let l1, l2, … lk denote k values in (0,p) Then

25. If we define {X(l): l[0,p]} and {Y(l): l[0,p]} Note:X(l) and Y(l) are “random” step functions and F(l) is a step function.

27. Another important comment In the case when F(l) is continuous then

28. Sometimes the spectral density function, f(l), is extended to the interval [-p,p] and is assumed symmetric about 0 (i.e. fs(l) = fs(-l) = f(l)/2 ) in this case It can be shown that

29. From now on we will use the symmetric spectral density function and let it be denoted by, f(l). Hence

30. Linear Filters

31. Let {xt : tT} be any time series and suppose that the time series {yt : t T} is constructed as follows: : The time series {yt : t T} is said to be constructed from {xt : t T} by means of a Linear Filter. Linear Filter as output yt input xt

32. Let sx(h) denote the autocovariance function of {xt : tT} and sy(h) the autocovariance function of {yt : t T}. Assume also that E[xt] = E[yt] = 0. Then: :

35. Hence where of the linear filter

36. Note: hence

37. Let a0 =1, a1, a2, … aq denote q + 1 numbers. Spectral density function Moving Average Time series of order q, MA(q) Let {ut|t T} denote a white noise time series with variance s2. Let {xt|t T} denote a MA(q) time series with m = 0. Note: {xt|t T} is obtained from {ut|t T} by a linear filter.

38. Now Hence

39. Example: q = 1

40. Example: q = 2

41. Spectral density function for MA(1) Series

42. Let b1, b2, … bp denote p + 1 numbers. Spectral density function Autoregressive Time series of order p, AR(p) Let {ut|t T} denote a white noise time series with variance s2. Let {xt|t T} denote a AR(p) time series with d = 0. Note: {ut|t T} is obtained from {xt|t T} by a linear filter.

43. Now Hence

44. Example: p = 1

45. Example: p = 2

46. Example : Sunspot Numbers (1770-1869)

48. Autocorrelation function and partial autocorrelation function

49. Spectral density Estimate

50. Assuming an AR(2) model