The Simple Regression Model

1. The Simple Regression Model y = b0 + b1x + u

2. Introduction • Most applied econometric analysis begins with the premise that Y and X are two variables, representing some population, and we are interested in explaining Y in terms of X. Or, how Y varies with changes in X. • There is never an exact relationship between two variables • What is the functional relationship? • How can we be sure that we are capturing a “ceteris paribus” relationship between Y and X. • We can write down: y = b0 + b1x + u

3. A Simple Assumption • The average value of u, the error term, in the population is 0. That is, • E(u) = 0 • This is not a restrictive assumption, since we can always use b0to normalize E(u) to 0 (HM1)

4. Zero Conditional Mean • We need to make a crucial assumption about how u and x are related • We want it to be the case that knowing something about x does not give us any information about u, so that they are completely unrelated. That is, that • E(u|x) = E(u) = 0, which implies • E(y|x) = b0 + b1x

5. E(y|x) as a linear function of x, where for any x the distribution of y is centered about E(y|x) y f(y) . E(y|x) = b0 + b1x . x1 x2

6. Ordinary Least Squares • Basic idea of regression is to estimate the population parameters from a sample • Let {(xi,yi): i=1, …,n} denote a random sample of size n from the population • For each observation in this sample, it will be the case that • yi = β0 + β1xi + ui

7. Population regression line, sample data points and the associated error terms y E(y|x) = b0 + b1x . y4 { u4 . u3 y3 } . y2 u2 { u1 . } y1 x2 x1 x4 x3 x

8. Deriving OLS Estimates • To derive the OLS estimates we need to realize that our main assumption of E(u|x) = E(u) = 0 also implies that • Cov(x,u) = E(xu) = 0 • Why? Remember from basic probability that Cov(X,Y) = E(XY) – E(X)E(Y)

9. Deriving OLS Estimates, cont. • We can write our 2 restrictions just in terms of x, y, b0 and b1 , since u = y – b0 – b1x • E(y – b0 – b1x) = 0 • E[x(y – b0 – b1x)] = 0 • These are called moment restrictions

10. Deriving OLS using M.O.M. • The method of moments approach to estimation implies imposing the population moment restrictions on the sample moments • What does this mean? Recall that for E(X), the mean of a population distribution, a sample estimator of E(X) is simply the arithmetic mean of the sample

11. More Derivation of OLS • We want to choose values of the parameters that will ensure that the sample versions of our moment restrictions are true • The sample versions are as follows:

12. More Derivation of OLS • Given the definition of a sample mean, and properties of summation, we can rewrite the first condition as follows

13. More Derivation of OLS

14. The OLS estimated slope is

15. Summary of OLS slope estimate • The slope estimate is the sample covariance between x and y divided by the sample variance of x • If x and y are positively correlated, the slope will be positive • If x and y are negatively correlated, the slope will be negative • Only need x to vary in our sample

16. More OLS • Intuitively, OLS is fitting a line through the sample points such that the sum of squared residuals is as small as possible, hence the term least squares • The residual, û, is an estimate of the error term, u, and is the difference between the fitted line (sample regression function) and the sample point

17. Sample regression line, sample data points and the associated estimated error terms y . y4 { û4 . û3 y3 } . y2 û2 { û1 } . y1 x2 x1 x4 x3 x

18. Alternate approach to derivation • Given the intuitive idea of fitting a line, we can set up a formal minimization problem • That is, we want to choose our parameters such that we minimize the following:

19. Alternate approach, cont. • If one uses calculus to solve the minimization problem for the two parameters you obtain the following first order conditions, which are the same as we obtained before, multiplied by n

20. Algebraic Properties of OLS • The sum of the OLS residuals is zero • Thus, the sample average of the OLS residuals is zero as well • The sample covariance between the regressors and the OLS residuals is zero • The OLS regression line always goes through the mean of the sample

21. Algebraic Properties (precise)

22. More terminology

23. Using Stata for OLS regressions • Regressions in Stata are very simple, to run the regression of y on x, just type • reg y x • A few words about Stata • I’ll be using Stata throughout the course • Examples will be in Stata but you are free to use whatever program you wish • Highly recommend purchasing Stata if you plan to do empirical work. See me if interested.

24. Two important issues in applied economics are • Understanding how changing the units of measurement of the dependent and/or independent variables affects OLS estimates • Knowing how to incorporate popular functional forms used in economics into regression analysis

25. Suppose we have the equation • salâry = 963.191 + 18.501roe where salary is measured in 1000s of dollars and roe equals return on equity • What happens when we measure salary in dollars rather than 1000s of dollars? • salâry$ = 963,191 + 18,501roe Summary: if the dependent variable is multiplied or divided by a constant c, then each coefficient is multiplied or divided by the same constant c.

26. What happens when we measure roedec equal to roe/100 (the decimal equivalent of roe)? • salâry = 963.191 + 1,850.1roedec If return on equity changes by 1% = 0.01, then salary changes by 1,850.1(.01) = 18.501 or $18,501 Summary: if an independent variable is multiplied by a constant c, then only that coefficient is divided by the same constant c and vice versa.

27. Incorporating Non-linearities • Consider the equation • wâge = 5.60 + 0.54 education For each additional year of education, wage is predicted to increase by $0.54. This is the increase for either the first year of education or the 20th year-this may not be reasonable • Suppose instead that the percentage increase is the same for each additional year of education. • log(wâge) = β0+ β1 education

28. w educ • If Δu = 0, then • % Δwage ≈ (100•β1) Δ education • log(wâge) = 5.60 + 0.083education • Often called a “semi-elasticity” model

29. Incorporating Non-linearities (cont.) • Constant Elasticity Model • ε =%Δy / %Δx • Consider the following equation • log(salâry) = 4.82 + 0.25log(sales) Interpretation: when sales increase 1%, salary increases by .25%

30. Incorporating Non-linearities (cont.) • When should you use levels and when logs? • Depends • What does theory tell you? Which seems more reasonable? Is there a standard in your field? For example, in the public finance literature, tax price (1-MTR) almost always enters in log form. • There are econometric tests that can help answer this—we will get to them. • Stata Example 1-1