Linear Regression

Linear Regression

METHODOLOGY OF ECONOMETRICS 1. Statement of theory or hypothesis. 2. Specification of the mathematical model of the theory 3. Specification of the statistical, or econometric, model 4. Obtaining the data 5. Estimation of the parameters of the econometric model 6. Hypothesis testing 7. Forecasting or prediction 8. Using the model for control or policy purposes.

1. Meaning of Regression

Meaning of Regression • Examine relationship between dependent and independent variables • Ex: how is quantity of a good related to price? • Predict the population mean of the dependent variable on the basis of known independent variables • Ex: what is the consumption level , given a certain level of income

Meaning of Regression • Also test hypotheses: • Ex: About the precise relation between consumption and income • How much does consumption go up when income goes up.

2. Regression Example

Example

Regression Example • Assume a country with a total population of 60 families. • Examine the relationship between consumption and income. • Some families will have the same income • Could split into groups of weekly income ($100, $120, $140, etc)

Regression Example • Within each group, have a range of family consumption patterns. • Among families with $100 income we may have six families, whose spending is 65, 70, 74, 80, 85, 88. • Define income X and spending Y. • Then within each of these categories, we have a distribution of Y, conditional upon a certain X.

Regression Example • For each distributions, compute a conditional mean: • E(Y|(X=Xi). • How do we get E(Y|(X=Xi) ? • Multiply the conditional probability (1/6) by Y value and sum them • This is 77 for our example. • We can plot these conditional distributions for each income level

Regression Example • The population regression is the line connecting the conditional means of the dependent variable for fixed values of the explanatory variable(s). • Formally: E(Y|Xi) • This population regression function tells how the mean response of Y varies with X.

Population Regression Line

Regression Example • What form does this function take? • Many possibilities, but assume its a linear function: E(Y|Xi) = 1 + 2Xi • 1 and 2 are the regression coefficients (intercept and slope). • Slope tells us how much Y changes for a given change in X. • We estimate 1 and 2 on the basis of actual observations of Y and X.

3. Linearity

Linearity • Linearity can be in the variables or in the parameters. • Linearity in the variables • Conditional expectation of Y is a linear function of X - • The regression is a straight line • Slope is constant • Can't have a function with squares, square root, or interactive terms- these have a varying slope.

Linearity • We are concerned with linearity in the parameters • The parameters are raised to the first power only. • It may or may not be linear in the variables.

Linearity • Linearity in the parameters • The conditional expectation of Y is a linear function of the parameters • It may or may not be linear in Xs. • E(Y|Xi) = 1 + 2Xi is linear • E(Y|Xi) = 1 + 2Xi is not. • Linear if the betas appear with a power of one and are not multiplied or divided by other parameters.

4. Stochastic Error

Stochastic Error Example • Model has a deterministic part and a stochastic part. • Systematic part determined by price, education, etc. • An econometric model indicates a relationship between consumption and income • Relationship is not exact, it is subject to individual variation and this variation is captured in u.

What Error Term Captures • Omitted variables • Other variables that affect consumption not included in model • If correctly specified our model should include these • May not know economic relationship and so omit variable. • May not have data • Chance events that occur irregularly--bad weather, strikes.

What Error Term Captures • Measurement error in the dependent variable • Friedman model of consumption • Permanent consumption a function of permanent income • Data on these not observable and have to use proxies such as current consumption and income. • Then the error term represents this measurement error and captures it.

What Error Term Captures • Randomness of human behavior • People don't act exactly the same way even in the same circumstances • So error term captures this randomness.

11. Hypothesis Testing

Hypothesis Testing • Set up the null hypothesis that our parameter values are not significantly different from zero • H0:2 = 0 • What does this mean?: • Income has no effect on spending. • So set up this null hypothesis and see if it can be rejected.

12. Coefficient of Determination--R2

Coefficient of Determination The coefficient of determination, R2, measures the goodness of fit of the regression line overall

Correlation Coefficient • The correlation coefficient is the square root of R2 • Correlation coefficient measures the strength of the relationship between two variables.

13. Forecasting

If the chosen model does not refute the hypothesis or theory under consideration,we may use it to predict the future value(s) of the dependent, orforecast,variable Y on the basis of known or expected future value(s) of theexplanatory, or predictor, variableX.

the estimated consumption function is: suppose we want to predict the mean consumption expenditurefor 1997. The GDP value for 1997 was 7269.8 billion dollars. Putting this GDP figure on the right-hand side of we obtain:

Forecast Error given the value of the GDP, the mean,or average, forecast consumption expenditure is about 4951 billion dollars. The actual value of the consumption expenditure reported in 1997 was4913.5 billion dollars. The estimated model thus overpredictedthe actual consumption expenditure by about 37.82 billion dollars. Wecould say the forecast error is about 37.82 billion dollars.

what is important for now is to notethat such forecast errors are inevitable given the statistical nature of our analysis.

Linear Regression