Multiple Regression II 4/11/12

1. Multiple Regression II • 4/11/12 • Categorical explanatory variables • Adjusted R2 Not in book Professor Kari Lock Morgan Duke University

2. Project 1 Regrade Requests • Regrade requests must be submitted in writing, with the original project, by Friday, 4/13/12, at 5pm • If regraded, I will grade the entire project, and the grade may go up or down

3. To Do • Project 2 Proposal (due TODAY, 5pm) • Homework 9 (due Monday, 4/16) • Project 2 Presentation (Thursday, 4/19) • Project 2 Paper (Wednesday, 4/25)

4. Chapter 9 Odd Solutions • Chapter 9 odd solutions available

5. Lab • Your group members are in the same lab (mostly) so this is a great time to meet with your group in person • There is no “On Your Own” and I’ve tried to make the lab short, so you should have some time with your group to get started with your project • If you want to work through the lab on your own before coming to class, you can have the whole lab time with your group to work on the project • Come to lab tomorrow!

6. US States • We will build a model to predict the % of the state that voted Republican in the 2008 US presidential election, using the 50 states as cases • Sample? Population? • This can help us to understand how certain features of a state are associated with political beliefs

7. US States • Response Variable: • Our first explanatory variable is region of the country: • Midwest, Northeast, South, or West

8. Categorical Variables • For this to make any sense, each x value has to be a number. • How do we include categorical variables in a regression setting?

9. Categorical Variables • Take one categorical variable, and replace it with several “dummy” variables • A dummy variable is 1 if the case falls into the category represented by the dummy variable, and 0 otherwise • Create one dummy variable for each category of the categorical variable

10. Dummy Variables

11. Dummy Variables • When using dummy variables, one has to be left out of the model • The dummy variable left out is called the reference level • When using region of the country (Northeast, South, Midwest, West) to predict % McCain vote, how many dummy variables will be included? • One b) Two c) Three d) Four

12. Dummy Variables • Predicting % vote for McCain with one categorical variable: region of the country • If “midwest” is the reference level:

13. Voting by Region Based on the output above, which region had the highest percent vote for McCain? Midwest Northeast South West

14. Voting by Region What is the predicted % Republican vote for a state in the northeast? –10.2% 48.6% 38.4% 58.8%

15. Voting by Region What is the predicted % Republican vote for a state in the midwest? 50% 48.6% 0% 58.8%

16. Categorical Variables • The p-value for each dummy variable tests for a significant difference between that category and the reference level • For an overall p-value for the significance of the categorical variable with multiple categories, use • z-test • T-test • Chi-square test • ANOVA

17. Categorical Variables

18. Categorical Variables in R • R automatically creates dummy variables for you if you include a categorical explanatory variable • The first level alphabetically is usually the reference level • If you want to change the reference level, see me

19. Categorical Variables • Either all dummy variables associated with a categorical variable have to be included in the model, or none of them • RegionW is not significant, but leaving it out would clump West with the reference level, Midwest, which does not make sense

20. Variables • Let’s include some more explanatory variables! • What helps to predict % voting Republican?

21. Categorical Variables • Be careful not to include a categorical variable for which every case is it’s own category • Example: using “State” as an explanatory variable would be silly, even though R2 = 1! • If you want to know how each state voted, it would make more sense to just look directly at McCainVote, rather than fitting a model and giving each state it’s own coefficient

22. Explanatory Variables • Also, be careful not to include explanatory variables that are essentially just another form of the response variable • For example, ObamaMcCain is “M” if the state went for McCain, and “O” if the state went for Obama • This is certainly associated with the % of people in the state that voted for McCain, but tells us nothing interesting

23. Explanatory Variables • Models should be created either to learn about relationships between explanatory variables and the response, or for prediction • Make sure the explanatory variables you include in the model are not contradicting the point of the model

24. Visualization How would we visualize the association between region and % vote for McCain? Scatterplot Side-by-side boxplots Bar chart Pie chart Mosaic plot

25. Side-by-Side Boxplots

26. Test for Association How would we test for an association between region and % vote for McCain? t-test for difference in means test for a correlation ANOVA chi-square test test for a difference in proportions

27. ANOVA

28. Visualization All of the other potential explanatory variables are quantitative. How would we visualize the association between each of them and % vote for McCain? Scatterplot Side-by-side boxplots Bar chart Pie chart Mosaic plot

29. What do you see?

30. Test for Association How would we test the association between each of these variables and % vote for McCain? t-test for difference in means test for a correlation ANOVA chi-square test test for a difference in proportions

31. Test for Correlation

32. Regression Model

33. Physical Activity Given all the other variables in the model, states with a higher percentage of physically active citizens are more likely to vote (a) Republican (b) Democratic

34. West Region • With only region as an explanatory variable, interpret the meaning of the negative coefficient of RegionW. • With all the other explanatory variables included, interpret the meaning of the positive coefficient of RegionW. In this data set, states in the West voted less Republican than states in the Midwest. States in the West voted more Republican than would be expected based on the other variables in the model, as compared to states in the Midwest.

35. Goal of the Model? • If the goal of the model is to see what and how each variable is associated with a state’s voting patterns, given all the other variables in the model, then we are done • If the goal is to predict the % of the vote that will be Republican, say in the 2012 election, we want to prune out insignificant variables to improve the model

36. Over-fitting • It is possible to over-fit a model: to include too many explanatory variables • The fewer the coefficients being estimated, the better they will be estimated • Usually, a good model has pruned out explanatory variables that are not helping

37. R2 • Adding more explanatory variables will only make R2 increase or stay the same • Adding another explanatory variable can not make the model explain less, because the other variables are all still in the model • Is the best model always the one with the highest proportion of variability explained, and so the highest R2? • Yes (b) No

38. Adjusted R2 • Adjusted R2is like R2, but takes into account the number of explanatory variables • As the number of explanatory variables increases, adjusted R2 gets smaller than R2 • One way to choose a model is to choose the model with the highest adjusted R2

39. Adjusted R2 You now know how to interpret all of these numbers!