1 / 38

REGRESSION (CONTINUED)

LECTURE 4. REGRESSION (CONTINUED) Analysis of Variance; Standard Errors & Confidence Intervals; Prediction Intervals; Examination of Residuals. Supplementary Readings : Wilks , chapters 6,9;

selma
Download Presentation

REGRESSION (CONTINUED)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LECTURE 4 REGRESSION (CONTINUED) Analysis of Variance; Standard Errors & Confidence Intervals; Prediction Intervals; Examination of Residuals Supplementary Readings: Wilks, chapters 6,9; Bevington, P.R., Robinson, D.K., Data Reduction and Error Analysis for the Physical Sciences, McGraw-Hill, 1992.

  2. Define: Recall from last time… We call these residuals What should we require of them?

  3. GAUSSIAN Recall from last time… What should we require of them?

  4. Recall from last time… Analysis of Variance (“ANOVA”)? 2(n=5) Gaussian data

  5. is guaranteed by linear regression procedure Analysis of Variance (“ANOVA”) Why “n-2”?

  6. Analysis of Variance (“ANOVA”) Define:

  7. Analysis of Variance (“ANOVA”) Define:

  8. Analysis of Variance (“ANOVA”)

  9. Analysis of Variance (“ANOVA”) 1 and n-2 degrees of freedom

  10. Analysis of Variance (“ANOVA”) Source df SS MS F-test Total n-1 SST Regression 1 SSR MSR=SSR MSR/MSE Residual n-2 SSE MSE=se2 1 and n-2 degrees of freedom

  11. Analysis of Variance (“ANOVA”) for Simple Linear Regression Source df SS MS F-test Total n-1 SST Regression 1 SSR MSR=SSR MSR/MSE Residual n-2 SSE MSE=se2 We’ll discuss ANOVA further in the next lecture (“multivariate regression”)

  12. ‘Goodness of Fit’

  13. If we have: ‘Goodness of Fit’ Linear Correlation

  14. ‘Goodness of Fit’ For simple linear regression

  15. ‘Goodness of Fit’ Outside the “support” of the regression, in general,

  16. ‘Goodness of Fit’ Outside the “support” of the regression, in general,

  17. ‘Goodness of Fit’ Reliability Bias

  18. ‘Goodness of Fit’ Reliability Bias

  19. Analysis of Variance (“ANOVA”) Under Gaussian assumptions, the estimates from linear regression of the parameter a and b represent unbiased estimates of means of a Gaussian distribution Where the standard errors in the regression parameters are:

  20. Confidence Intervals The estimated regression slope ‘b’ is likely to be within some range of the true ‘b’

  21. Confidence Intervals This naturally defines a t test for the presence of a trend:

  22. Prediction Intervals MSE in a predicted value or, (‘Prediction Error’) is larger than the nominal MSE, increasing as the predictand value departs from the mean Note that sy approaches se as the ‘training’ sample becomes large

  23. Linear Correlation ‘r’ suffers from sampling error both in the regression slope and the estimates of variance…

  24. Linear Correlation ‘r’ suffers from sampling error both in the regression slope and the estimates of variance…

  25. Linear Correlation Coefficient

  26. Examining Residuals Heteroscedasticity A trend in residual variance violates the assumption of Gaussian residuals…

  27. Examining Residuals Heteroscedasticity Often a simple transformation of the original data will yield more closely Gaussian residuals…

  28. Examining Residuals Leverage Points can still be a problem!

  29. Examining Residuals Autocorrelation Durbin-Watson Statistic

  30. Suppose we have the simple (‘first order autoregressive’) model For example: Examining Residuals Autocorrelation Then we can still use all of the results based on Gaussian statistics, but with the modified sample size:

  31. Suppose we have the simple (‘first order autoregressive’) model Examining Residuals Autocorrelation Then we can still use all of the results based on Gaussian statistics, but with the modified sample size: Different for tests of variance

  32. Suppose we have the simple (‘first order autoregressive’) model Examining Residuals Autocorrelation Then we can still use all of the results based on Gaussian statistics, but with the modified sample size: Different again for correlations

  33. Suppose we have the simple (‘first order autoregressive’) model Examining Residuals We can remove the serial correlation through

  34. Autocorrelation AR(1) PROCESS For simplicity, we will assume zero mean

  35. Autocorrelation AR(1) PROCESS What is the standard deviation of an AR(1) process?

  36. Using, Autocorrelation AR(1) PROCESS Recursively, we thus have for an AR(1) process, For a long series, the sampling distribution for r is approximately Gaussian (like the slope parameter in linear regression), with standard deviation, How do we determine if r is significantly non-zero?

  37. Autocorrelation This is essentially the t test statistic Example: consider the case r=0.2, n=200 Z=0.2/(0.96/200)1/2 =0.2/0.069=2.9 This is approximately the 3 sigma level of a Gaussian distribution. p=0.004 for a two-sided test, p=0.002 for a one-sided test

  38. Matlab Examples

More Related