Multiple Regression

1. Multiple Regression

2. Outline • Purpose and logic : page 3 • Parameters estimation :page 9 • R-square :page 13 • Hypothesistesting: page 17 • Partial and semi-partial regression : page 21 • Confidence intervals : page 26

3. Multiple Regression • There are more than one predictor • Example with 2 predictors

4. Multiple Regression • Because we have two predictors, it is possible to illustrate the situation using a 3-dimensional scatter plot

5. Multiple Regression • The different relations can be illustrated by a bivariate scatter plots. x1 x2 y x1 x2 y

6. Multiple Regression • As well as the bivariate correlations (see SSCP) .

7. Multiple Regression • From the regression line to the regression hyperplane

8. Multiple Regression • Graphics It is not possible to illustrates vectors in 5 dimensions. However, the computations will be the same.

9. Multiple Regression How the regression coefficients can be obtained ? Universal formula, it does not change whatever the number of predictors (it is a special case of multivariate regression).

10. Multiple Regression • The b0 coefficient can also be directly obtained if we include the unity vector 1 as a variable.

11. Multiple Regression How the regression coefficients can be obtained ?

12. Standardized Regression Coefficients It allow to measure the « importance » of the predictors, since they all have a variability of 1 and a mean of 0. Or Therefore, an increase of one unit by z1, will increase 0.74 standard deviation by yZ. ^

13. R-Square Like in simple regression, in multiple regression we will use the R-square coefficient (R2); also named coefficient of determination . This R2 have the same interpretation as the one in simple regression: percentage of explained variance given by all the predictors. • Sum of Squares and Cross Product Matrix (SSCP)

14. R-Square • Recall: By dividing the SSCP matrix by the corresponding degrees of freedom, we get the variance-covariance matrix. • We can also get the bivariate correlations

15. R-Square • In addition, the SSCP matrix can be partitioned in function of the independent (predictors) and dependent variables (criterion). Spc Spp Scp Scc

16. R-Square • R2 is obtained by: Spc Spp Scp Scc • R2adjis an unbiased estimation of the population variability given a sample Number of participants Number of predictors (independent variables)

17. Hypothesis testing • The hypothesis is that the R-square between the predictors and the criterion is null. In other words, we are trying to know if the X and y variables are linearly independent. If we reject that hypothesis, then the two populations are not independent, there is a linear relation between the two.

18. Hypothesis testing Using confidence intervals. Using observedstatistics.

19. Hypothesis testing • The hypothesis is that the R-Square between the predictors and the criterion is null. In other words, we are trying to know if the X and y variables are linearly independent. If we reject that hypothesis, then the two populations are not independent, there is a linear relation between the two. Explained variability Unexplained variability Because the Fobs >Fcrit (22.0273>19.00), we reject H0 and we therefore accept H1. The 2 populations are linearly dependant.

20. ANOVA Table • The hypothesis is that the R-Square between the predictors and the criterion is null. = Because F(2,2)=22.0273, p.<0.05, we reject H0 and we accept H1. The 2 populations are linearly dependant.

21. Partial and semi-partial correlations • The idea is to put forward the effect of one predictor on the dependant variable by controlling the effects of the other predictors. • Squared semi partial correlation coefficient It is the total variability (R2) minus the total variability excluding the studied predictor from the data set. It is the portion of variability that is unique to this particular predictor • Squared partial correlation coefficient It is the variability proportion associated with one predictor but not with the others. In other words, it is the unestimated variability by the other predictors that is estimated by the predictor studied.

22. Partial and semi-partial correlations Y e b a c x1 x2

23. Partial and semi-partial correlations • Example x1 x2 39 % of the y variability is explained solely by the first predictor. 9% of the y variability is explained solely by the second predictor. 90% of the unexplained y variability by the second predictor is explained by the first predictor. 67% of the unexplained y variability by the first predictor is explained by the second predictor.

24. Partial and semi-partial correlations • Signification testing The various parameters (pri, bi, Bi) are directly related from the portion of explained variability by the semi partial coefficient. Therefore, if this last coefficient is statistically significant, then all the other parameters will be. x1 x2

25. Standard errors associated with the regression parameters Standard error associated with the regression coefficients Standard errors associated with the standardized regression coefficients

26. Confidance intervals associated with the regression parameters Confidence intervals associated with the regression coefficients Confidence intervals associated with the standardized regression coefficients