Chapters 10 and 11: Using Regression to Predict

1. Chapters 10 and 11: Using Regression to Predict Math 1680

2. Overview • Predicting Values • The Regression Line • The RMS Error • The Regression Effect • A Second Regression Line • Summary

3. Predicting Values • We have previously seen that a pair of data sets, X and Y, can be characterized by their five-statistic summary • µX, the average value in X • SDX, the standard deviation of X • µY, the average value in Y • SDY, the standard deviation of Y • r, the correlation coefficient • Often, we want to predict a y-value given a particular x-value • Want to use only the five-statistic summary to make prediction

4. Predicting Values • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • If you had to guess what the weight of any man would be, what is your best bet?

5. Predicting Values • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • Suppose you know the man is 1 SD above average • Would your best guess for his weight be 1 SD above average?

6. The SD line is the dashed line running through the scatter plot • If we guessed 1 SD above average weight, where would we be on the plot? • What would a better guess be?

7. The Regression Line • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • It turns out that the correlation coefficient determines the best guess • For every SD we move in X, we should move r SD’s in Y

8. The Regression Line • The regression line from X to Y • Runs through the point of averages • Has a slope of r time the slope of the SD line • The regression line predicts the average value for y within the narrowed-down range specified by a given x

9. The Regression Line • The formula for the regression line from X to Y is • Or, alternately, • When is the regression line the same as the SD line? When r = 1 or -1

10. The regression line is the solid line running through the scatter plot • If we looked at heights 1 SD above the average, the regression line runs through the point 0.47 SD’s above average in weight

11. The Regression Line • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • What is the average weight of all the men who are 73 inches tall? • For a man 73 inches tall, what weight should we predict? 176.1 lbs

12. The Regression Line • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • What is the average weight of all the men who are 64 inches tall? • For a man 64 inches tall, what weight should we predict? 133.8 lbs

13. The Regression Line • To use the regression line from X to Y… • Standardize the given x-value to get zx • Use the regression equation to go from X to Y • zY= rzX • Unstandardize zY to get y

14. The Regression Line • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • Predict the weight of a man who is 6’4” 190.2 lbs

15. The Regression Line • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • Predict the weight of a man who is 5’6” 143.2 lbs

16. The Regression Line • Important notes about the regression line from X to Y • It predicts the average value for y given an x value • If the scatter plot is football shaped, this prediction will be above about half of the sample and below the other half • This is because the variables are approximately normal • The slope of the regression line will always be

17. The RMS Error • Recall that an average alone did not uniquely describe a data set • A spread measure was needed • Since the regression method only gives us an average value as its prediction, we can’t really tell by this alone how good a guess it is

18. The prediction given by the regression line for a height of 73 inches is at (73 in, 176 lbs) • How much does the heaviest 73” tall man weigh? • How much does the lightest 73” tall man weigh?

19. The RMS Error • If we are given a specific man to predict, we are likely to be a little off with the regression prediction • You can think of the prediction error as being the vertical distance from the point to the regression line • That is, error = actual – predicted • If we want to get a good sense of what the typical error for a given x-value is, we can find the RMS of all the errors for all the points • This value is called the RMS error for the regression line

20. The RMS Error • The RMS error is to the regression line what the SD is to the average • The RMS error measures the spread around a prediction from the regression line • Recall we are generally assuming the data sets are approximately normal • About 68% of the points on a scatter plot will fall within the strip that runs from one RMS error below to one RMS error above the regression line

21. 1 RMS error, 68% Regression Line 2 RMS errors, 95% The RMS Error

22. The RMS Error • The RMS error for regression from X to Y (denoted R) can be calculated from the five-statistic summary by • What units would R have? • What happens when r gets close to 0? • What happens when r gets close to 1 or -1?

23. The RMS Error • The RMS error allows us to give a range around our prediction • If the scatter plot is football-shaped, the RMS error is roughly constant across the entire range of the data set • The vertical spread around one part is about the same as the vertical spread around other parts

24. The RMS Error • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • Predict and give the RMS error for the weight of a man who is 6’2” 180.8 ± 26.5 lbs

25. The RMS Error • Suppose we have the following five-number summary stats for the height (X) and weight (Y) of men in the US • µX= 70 inches, SDX= 3 inches • µY= 162 lbs, SDY= 30 lbs • r = 0.47 • Predict and give the RMS error for the weight of a man who is 5’4” 133.8 ± 26.5 lbs

26. The Regression Effect • A preschool program attempts to boost students’ IQ scores • The children are tested when they enter the program (pretest) • The children are retested when they leave the program (post-test)

27. The Regression Effect • On both occasions, the average IQ score was 100, with an SD of 15 • Also, students with below-average IQs on the pretest had scores that went up on the average by 5 points • Students with above average scores on the pretest had their scores drop by an average of 5 points

28. The Regression Effect • Does the program equalize intelligence? No. If the program really equalized intelligence, then the SD for the post-test results should be smaller than that of the pre-test results. This is an example of the regression effect.

29. The Regression Effect • The regression effect is a byproduct of the fact that predictions from a regression line are average values • Some of the people who did very well on the pre-test may simply have had a good test day • Their scores shouldn’t necessarily be as high on the post-test as they were on the pretest • Similarly, some of the people who did poorly on the pre-test may simply have had a bad test day • Their scores shouldn’t necessarily be as low on the post-test as they were on the pretest

30. The Regression Effect • Sometimes researchers mistake the regression effect for some important underlying cause in the study (regression fallacy) • Tall fathers tend to have tall sons who are slightly shorter than the father • There is no biological cause for this reduction • It is strictly statistical

31. The Regression Effect • As part of their training, air force pilots make practice landings with instructors, and are rated on performance • The instructors discuss the ratings with the pilots after each landing • Statistical analysis shows that pilots who make poor landings the first time tend to do better the second time • Conversely, pilots who make good landings the first time tend to do worse the second time

32. The Regression Effect • The conclusion is that criticism helps the pilots while praise makes them do worse • As a result, instructors were ordered to criticize all landings, good or bad • Was this warranted by the facts? No. This is an example of regression fallacy.

33. The Regression Effect • An instructor gives a midterm • She asks the students who score 20 points below average to see her regularly during her office hours for special tutoring • They all score at class average or above on the final • Can this improvement be attributed to the regression effect? Why/why not? No. If it was only the regression effect, most of the students still would have scored below average. The fact that everyone in the tutoring group scored above average indicated that the tutoring had the proper effect.

34. A Second Regression Line • The focus so far has been on the regression line from X to Y • Note, however, that there is also a regression line from Y to X • What would the difference between the two lines be? The regression line from X to Y is given by zY= rzX, while the regression line from Y to X is given by zX= rzY

35. A Second Regression Line • A study of 1,000 families gives the following • The husbands’ average height was 68 inches with an SD of 2.7 inches • The wives’ average height was 63 inches with an SD of 2.5 inches • The correlation between them was 0.25 • Predict and give the RMS error for the husband’s height when his wife’s height is 68 inches 69.35 inches, give or take 2.61 inches

36. A Second Regression Line • A study of 1,000 families gives the following • The husbands’ average height was 68 inches with an SD of 2.7 inches • The wives’ average height was 63 inches with an SD of 2.5 inches • The correlation between them was 0.25 • Predict and give the RMS error for the wife’s height when her husband’s height is 69.35 inches 63.31 inches, give or take 2.42 inches

37. Regression Line from Y to X SD Line A Second Regression Line Regression Line from X to Y

38. Regression Line from Y to X SD Line A Second Regression Line Regression Line from X to Y

39. Regression Line from Y to X SD Line A Second Regression Line Regression Line from X to Y

40. Summary • When trying to make predictions from a football-shaped plot, a good predictor is the average value for one variable within a restricted range in the other • The regression line runs through all of these averages • For every SD moved in the independent variable, the regression line predicts a move of r SD’s in the dependent variable • The prediction from the regression line is likely to be off by the RMS error • The RMS error can be calculated as

41. Summary • The regression effect is purely statistical • It does not reflect a significant underlying trend in the data • There are two regression lines for a scatter plot • Which one to use depends on which variable you are predicting