Simple Linear Regression

1. Simple Linear Regression One reason for assessing correlation is to identify a variable that could be used to predict another variable If that is your intention – the sample used for the assessment of the correlation must be ‘representative’ of the population within which you wish to make the predictions.

2. Assume you would like to predict the scores that each of you would get on Exam 2 in the course. If we assume that you are a sample from the ‘population’ of students who have taken this course, we could look at past performance to predict future performance. If all you knew was that you were students from the same population taking the same course – what would you predict for your Exam 2 scores?

3. 100 90 Exam 2 (85.7) Mean for Exam 2 based on previous students 80 70 Students Given no other information, the best guess would be that each student would get the mean Exam 2 grade, BUT if there was a variable related to Exam 2 scores that was available for each student, it could be used to improve the predictions

4. For example, it turns out that grades on Exam 1 and grades on Exam 2 are significantly correlated (r = .64), so they share some variance and knowing a student’s Exam 1 grade should help predict her Exam 2 grade Note ‘outlier’

5. Note that original data set produced this scatter plotWhat is the problem? Note ‘outlier’ Reduces r from .64 to .52

6. Simple Linear Regression is just an application using Pearson’s r (a coefficient of the strength and direction of linear association) so assumptions are the same Involves finding the linear relationship between X and Y, that minimizes the differences between actual Y scores and predicted Yp scores (predicted from X)

7. line of best fit (minimizes errors) - where formula for points on line is Y = a + bX a is ‘intercept’: value of Y when X = 0 b is the slope of the line: change in Y with each change in X so all predicted scores (Yp) fall on the line of best fit

8. Mean Y Y Y slope X Intercept- a Y when X = 0 0 Mean X X

9. In regression language a is the ‘regression constant’ b is the ‘regression coefficient’- based on r and the variability of Y relative to the variability of X If r = 1, have perfect straight line relationship If r is less than 1 equation becomes Yp = a + bX (+ residual)

10. Mean Y Y Y slope Line of Best Fit would minimize deviations of scores from regression line X Intercept- a Y when X = 0 0 Mean X X

11. The regression line of ‘best fit’ minimizes those errors of prediction least squares regression line Sum (Yactual – Yp)2 by = r(SDy/SDx) If X and Y are converted to z scores, both SDs = 1, so by = r by also can be found by Covxy/Varx • even though r is correlation of X & Y, b will vary depending on which one is used to predict other – changes which SD goes on top/bottom of ratio ay = Mean Y – by (Mean X) Value of Yp when X = 0

12. Partitioning the Variability in Y SSTotal = Sum (Y - Mean Y)2 variability of Y scores from the mean Separated into SSregression = Sum (Yp – Mean Y) 2 Improvement in predictions when using X (variability in Y explained by X), rather than assuming everyone gets the Mean SSresidual = Sum (Y - Yp) 2 Degree to which predictions do not match the actual scores (prediction errors that have been minimized)

13. SSregression / SST = r2 % of total variance in Y accounted for by X or --variance in Y explained by X SSresidual / SST = 1 – r2 % unexplained variance in Y (errors) Variance of errors = SSresidual/df = MSresidual Standard Error of Estimate = SQRT (MSresidual) typical amount by which predicted score deviates from actual score Across each value of X, what is the typical deviation of actual from predicted scores of Y

14. Standard Error of Estimate For predicting scores for any individual, can estimate SEE for that prediction from SEEest = SEE * SQRT 1 + 1 + (X - Mx)2 N (N-1) * (Sx2) the error is higher as the score on X deviates from the Mean X, and with a smaller sample size used for making the estimate

15. Example in Handout Packet – Predicting IQ from GPA IQ GPA 120.00 3.80 115.00 3.00 110.00 3.40 105.00 3.50 100.00 2.80 95.00 2.40 90.00 2.50 Mean 105.00 3.06

16. Mean IQ = 105 This would be your best ‘guess’ for every person if you had no useful predictor Improvement in Prediction using GPA Residual – distance from the line Residual much greater here Mean GPA = 3.06

17. Adjusted R2 is adjusted for the sample size and the number of predictors in the model. Since the sample value will be an inflated estimate of the value for R2 in the population, use adjusted R2 when applying results to the population. Although listed as R and R2 in SPSS regression output, these are, in Simple Linear Regression analyses, just the Pearson r and r2 SPSS will report an ANOVA Table with Regression output, but for the Simple Linear Regression, all you need report is the t-value (which has df = n-p-1; p = # of predictors) that tests the significance of the single predictor (gpa) in the Model (is r reliably different from 0). Note that the Standardized Coefficient, beta, which is the regression coefficient when all variables are standardized (z-scores) is the same as r Predicted IQ = 53.05 + 16.99 (GPA) + error approximate 95% CIs are + 2(6.57) for predicting mean IQ of those with a given GPA

18. 95% confident that the ‘true’ line of best fit lies within these CIs Note that accuracy of predictions decreases as you move away from the means

19. 95% confident that the ‘true’ line of best fit lies within the CIs If you consider all the lines that might fall in the intervals, can see that variability increases as you move away from the ‘center’ (Mx, My) Note that accuracy of predictions decreases as you move away from the means

20. Predicting grades for PSYC 6102 Exam 2 from Exam 1 scores r = +.64 Exam2 = 38.54 + .56 * Exam1 R2 = .41 Usual convention is to report regression constant and coefficient to 3 decimal places

21. Predicted Exam 2 score = 38.54 + .56 (Exam 1) + error approximate 95% CI, at best, + 2(5.15)

22. Usual convention is to report regression constant and coefficient to 3 decimal places Note that the 95% CI’s are much wider for making predictions for individual’s Exam 2 scores rather than predicting the typical Exam 2 score (on the line)

23. Partial Correlation logic Finding the correlation of X and Y after ‘partialling’ out the relationship of each with Z predict X from Z – for each person, find residuals (amount prediction missed by) predict Y from Z – for each person, find residuals (amount prediction missed by) Correlate the two sets of residuals relationship of X and Y after removing relationship each has with Z