1 / 15

LECTURE 12 Multiple regression analysis

LECTURE 12 Multiple regression analysis. Epsy 640 Texas A&M University. Multiple regression analysis. The test of the overall hypothesis that y is unrelated to all predictors, equivalent to H 0 :  2 y  123… = 0 H 1 :  2 y  123… = 0 is tested by

doris
Download Presentation

LECTURE 12 Multiple regression analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LECTURE 12Multiple regression analysis Epsy 640 Texas A&M University

  2. Multiple regression analysis • The test of the overall hypothesis that y is unrelated to all predictors, equivalent to • H0: 2y123… = 0 • H1: 2y123… = 0 • is tested by • F = [ R2y123… / p] / [ ( 1 - R2y123…) / (n – p – 1) ] • F = [ SSreg / p ] / [ SSe / (n – p – 1)]

  3. Multiple regression analysis SOURCE df Sum of Squares Mean Square F x1, x2… p SSreg SSreg / p SSreg/ 1 SSe /(n-p-1) e (residual) n-p-1 SSe SSe / (n-p-1) total n-1 SSy SSy / (n-1) • Table 8.2: Multiple regression table for Sums of Squares

  4. Multiple regression analysis predicting Depression LOCUS OF CONTROL, SELF-ESTEEM, SELF-RELIANCE

  5. SSreg ssx1 SSy SSe ssx2 Fig. 8.4: Venn diagram for multiple regression with two predictors and one outcome measure

  6. Type I ssx1 SSx1 SSy SSe SSx2 Type III ssx2 Fig. 8.5: Type I contributions

  7. Type III ssx1 SSx1 SSy SSe SSx2 Type III ssx2 Fig. 8.6: Type IIII unique contributions

  8. Multiple Regression ANOVA table SOURCE df Sum of Squares Mean Square F (Type I) • Model 2 SSreg SSreg / 2 SSreg / 2 • SSe / (n-3) • x1 1 SSx1 SSx1 / 1 SSx1/ 1 • SSe /(n-3) • x2 1 SSx2  x1 SSx2  x1SSx2  x1/ 1 • SSe /(n-3) • e n-3 SSe SSe / (n-3) • total n-1 SSy SSy / (n-3) • Table 8.3: Multiple regression table for Sums of Squares of each predictor

  9. PATH DIAGRAM FOR REGRESSION  = .5 X1 .387 r = .4 Y e X2  = .6 R2 = .742 + .82 - 2(.74)(.8)(.4)  (1-.42) = .85

  10. Depression e .471 .4 LOC. CON. -.317 DEPRESSION SELF-EST R2 = .60 -.186 SELF-REL

  11. Shrinkage R2 • Different definitions: ask which is being used: • What is population value for a sample R2? • R2s = 1 – (1- R2)(n-1)/(n-k-1) • What is the cross-validation from sample to sample? • R2sc = 1 – (1- R2)(n+k)/(n-k)

  12. Estimation Methods • Types of Estimation: • Ordinary Least Squares (OLS) • Minimize sum of squared errors around the prediction line • Generalized Least Squares • A regression technique that is used when the error terms from an ordinary least squares regression display non-random patterns such as autocorrelation or heteroskedasticity. • Maximum Likelihood

  13. Maximum Likelihood Estimation • Maximum likelihood estimation • There is nothing visual about the maximum likelihood method - but it is a powerful method and, at least for large samples, very preciseMaximum likelihood estimation begins with writing a mathematical expression known as the Likelihood Function of the sample data. Loosely speaking, the likelihood of a set of data is the probability of obtaining that particular set of data, given the chosen probability distribution model. This expression contains the unknown model parameters. The values of these parameters that maximize the sample likelihood are known as theMaximum Likelihood Estimatesor MLE's.  Maximum likelihood estimation is a totally analytic maximization procedure. • MLE's and Likelihood Functions generally have very desirable large sample properties:  • they become unbiased minimum variance estimators as the sample size increases • they have approximate normal distributions and approximate sample variances that can be calculated and used to generate confidence bounds • likelihood functions can be used to test hypotheses about models and parameters  • With small samples, MLE's may not be very precise and may even generate a line that lies above or below the data pointsThere are only two drawbacks to MLE's, but they are important ones:  • With small numbers of failures (less than 5, and sometimes less than 10 is small), MLE's can be heavily biased and the large sample optimality properties do not apply • Calculating MLE's often requires specialized software for solving complex non-linear equations. This is less of a problem as time goes by, as more statistical packages are upgrading to contain MLE analysis capability every year.

  14. Outliers • Leverage (for a single predictor): • Li = 1/n + (Xi –Mx)2 / x2 (min=1/n, max=1) • Values larger than 1/n by large amount should be of concern • Cook’s Di = (Y – Yi) 2 / [(k+1)MSres] • the difference between predicted Y with and without Xi   

  15. Outliers • In SPSS Regression, under the SAVE option, both leverage and Cook’s D will be computed and saved as new variables with values for each case

More Related