1 / 21

Multiple Linear Regression

Multiple Linear Regression. Learning Objectives Extend Simple Linear Regression concepts to regression with multiple explanatory variables Apply the Matlab regression tools and interpret their output Choose the variables to use in a multiple regression

cynthiaevans
Download Presentation

Multiple Linear Regression

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multiple Linear Regression Learning Objectives • Extend Simple Linear Regression concepts to regression with multiple explanatory variables • Apply the Matlab regression tools and interpret their output • Choose the variables to use in a multiple regression • Quantify the uncertainty of MLR predictions

  2. Readings • Kottegoda and Rosso, Chapter 6 (6.2) • Helsel and Hirsch, Chapters 9 and 11 • Hastie, Tibshirani and Friedman, Chapter 3 • Matlab Statistics Toolbox Users Guide, Chapter 6.

  3. Multiple Linear Regression Model

  4. Data for Multiple Linear Regression Input Output Carrier Matrix Residuals

  5. Solving Multiple Linear Regression Minimizing results in (KR 6.2.7) Vector of estimated mean values at each observation Vector of Residuals (KR 6.2.9)

  6. Error Variance (KR 6.2.13) Sum of squares of observation deviations from the mean Sum of squares of regression estimates deviations from the mean

  7. Significance Tests on the Regression Overall Significance (KR 6.2.16) Nested/Partial F Test (Significance of ‘new’ parameters) (KR 6.2.19) (HH p297) Complicated model with p1 parameters versus simpler model with p0 parameters

  8. Significance and confidence limits on regression parameters (KR 6.2.17) (KR 6.2.18) [b,bint,r,rint,stats]=regress(Y,X); b,bint b =0.0057 0.2187 -0.0074 bint = -0.1128 0.1242 0.1319 0.3054 -0.0248 0.0100

  9. Confidence limits on mean response (KR 6.2.32) (KR 6.2.33)

  10. Confidence limits on individual future value (KR 6.2.34)

  11. Regression Diagnostics Do not rely only on R2, F, SSE and T statistics. (Read Helsel and Hirsch page 244 and 300) Use graphical tools to diagnose MLR deficiencies Partial Residual Plot

  12. Diagnostic Plots

  13. Helsel and Hirsch page 245

  14. The Hat Matrix [,1] [,2] [,3] [,4] [,5] [,6] [,7] [1,] 0.46 0.36 0.25 0.14 0.04 -0.07 -0.18 [2,] 0.36 0.29 0.21 0.14 0.07 0.00 -0.07 [3,] 0.25 0.21 0.18 0.14 0.11 0.07 0.04 [4,] 0.14 0.14 0.14 0.14 0.14 0.14 0.14 [5,] 0.04 0.07 0.11 0.14 0.18 0.21 0.25 [6,] -0.07 0.00 0.07 0.14 0.21 0.29 0.36 [7,] -0.18 -0.07 0.04 0.14 0.25 0.36 0.46 H is independent of the observed outputs (y). Linear regression predictions are a weighted average of the original y-values

  15. Weights from the Hat matrix. Each line in the plot represents the weights used to determine the fitted y-value at the indicated point y x 7 7 1 6 5 4 2 6 3 2 1 3 5 Weight 4 x-value

  16. Diagonals of the Hat Matrix Quantify the Leverage that a point has on the regression

  17. SLR MLR: Hat matrix H = Leverage outlier with high leverage but low influence outlier with high leverage and high influence Helsel and Hirsch page 246

  18. Outliers are harder to detect in MLR Standardized residual (Compare to Normal or t distribution) (KR 6.2.26) Prediction residual (leave one out estimate) (HH p247) Prediction Error Sum of Squares (HH p247) Studentized residual (compare to t distribution) (HH p247)

  19. Cook’s Distance: Leverage v. Actual Influence • The Hat matrix (hii) indicates the leverage of point i. • The leverage is not the same as the actual influence. • Actual influence is only realized if the predicted value is very different than the observed point. • Cook’s Distance (Outlier if > 1) (KR 6.2.27)

  20. Choosing Variables in MLR(Helsel and Hirsch p 309) • Stepwise regression (forward or backward based on F or t statistic). Best model not guaranteed • Plausible theory why variable should influence response • Evaluate all possibilities using overall measure of quality (HH p313)

  21. Overall Measures of Quality • Mallow’s Cp • Prediction Error Sum of Squares • Adjusted R2 (HH p313) (HH p247) (HH p313)

More Related