Bivariate Regression Analysis

1. Bivariate Regression Analysis • Theoretical Models • Basic Linear Models: Deterministic Version • Basic Linear Models: Stochastic Version • Statistical Assumptions • Estimating Linear Models • Residuals (and the Pursuit of Truth…) • An Example

2. Theoretical Linear Models • The basis of “causality” in models • Time ordering • Co-variation • Non-spuriousness • Examples • Fire Deaths f (# of fire trucks at the scene) • Job Retention f (current job satisfaction) • Income f (education)

3. a b Deterministic Linear Models • Theoretical Model: • b0andb1are constant terms • b0 is the intercept • b1 is the slope • Xi is a predictor of Yi Yi b0 Xi

4. Stochastic Linear Models • E[Yi] = b0+b1Xi • Variation in Y is caused by more than X: error (ei) • So:

5. ei ei=0 X Assumptions Necessary for Estimating Linear Models 1. Errors have identical distributions Zero mean, same variance, across the range of X 2. Errors are independent of X and other ei 3. Errors are normally distributed

6. Y X Normal, Independent & Identical ei Distributions (“Normal iid”) Problem: We don’t know: a) if error assumptions are true; b) values for b0 and b1 Solution: Estimate ‘em!

7. Estimating Linear Models This is the formula for RESIDUALS -- which you will come to know and cherish.

8. Residuals: Statistical Forensics • Residuals measure prediction error: • ei > 0 if Yi > Yi • ei < 0 if Yi < Yi Y ^ ^ X

9. Stata and Regression: Predicting Incarceration with Average income • Stata dataset: Guns.dta • From “Data for empirical exercises” • What are your expectations? Why? • Stata command: • Regression: “regress incarc_rate avginc” • Output:

10. In our data, some observed values are larger than would be predicted by average income alone Residual Analysis

11. Normality of Residuals

12. More on Normality: Q-Normal

13. Distribution of Residuals by X

14. BREAK TIME