8.4 Weighted Least Squares Estimation

1. 8.4 Weighted Least Squares Estimation Before the existence of heteroskedasticity-robust statistics, one needed to know the form of heteroskedasticity -Het was then corrected using WEIGHTED LEAST SQUARES (WLS) -This method is still useful today, as if heteroskedasticity can be correctly modeled, WLS becomes more efficient than OLS -ie: WLS becomes BLUE

2. 8.4 Known Heteroskedasticity -Assume first that the form of heteroskedasticity is known and expressed as: -Where h(X) is some function of the independent variables -since variance must be positive, h(X)>0 for all valid combinations of X -given a random sample, we can write:

3. 8.4 Known Het Example -Assume that sanity is a function of econometrics knowledge and other factors: -However, by studying econometrics two things happen: either one becomes more sane as one understands the world, or one becomes more crazy as one is pulled into a never-ending vortex of causal relationships. Therefore:

4. 8.4 Known Heteroskedasticity -Since h is a function of x, we know that: -Therefore -So inclusion of the h term in our model can solve heteroskedasticity

5. 8.4 Fixing Het – And Stay Down! -We therefore have the modified equation: -Or alternately: -Note that although our estimates for BJ will change (and their standard errors become valid), their interpretation is the same as the straightforward OLS model (don’t try to bring h into your interpretation)

6. 8.4 Het Fixing – “I am the law” -(8.26) is linear and satisfied MLR.1 -if the original sample was random, nothing chances so MLR.2 is satisfied -If no perfect collinearity existed before, MLR.3 is still satisfied now -E(ui*|Xi*)=0, so MLR.4 is satisfied -Var(ui*|Xi*)=σ2, so MLR.5 is satisfied -if ui has a normal distribution, so does ui*, so MLR. 6 is satisfied -Thus if the original model satisfies everything but het, the new model satisfies MLR. 1 to 6

7. 8.4 Het Fix – Control the Het Pop -These BJ* estimates are different from typical OLS estimates and are examples of GENERALIZED LEAST SQUARES (GLS) ESTIMATORS -this GLS estimation provides standard errors, t statistics and F statistics that are valid -Since these estimates satisfy all 6 CLM assumptions, and because they are BLUE, GLS is always more efficient than OLS -Note that OLS is a special case of GLS where hi=1

8. 8.4 Het Fix – Who broke it anyhow? -Note that the R2 obtained from this regression is useful for F statistics but is NOT useful for its typical interpretation -this is due to the fact that it explains how much X* explains y*, not how much X explains y -when GLS estimators are used to correct for heteroskedasticity, they are called WEIGHTED LEAST SQUARES (WLS) ESTIMATORS -most econometric programs have commands to minimize the weighted sum of squared residuals:

9. 8.4 Incorrect Correcting? What happens if h(x) is misspecified and WLS is run (ie: if one expects x1 to cause het but x3 actually causes het) • WLS is still unbiased and consistent (similar to OLS) • Standard Errors (thus t and F tests) are no longer valid -to avoid this, one can always apply a fully robust inference for WLS (as we say for OLS in 8.2) -this can be tedious

10. 8.4 Incorrect Correcting? WLS is often criticized as being better than OLS ONLY IF the form of het is correctly chosen -one may argue that making some correction for het is better than none at all -there is always the option of using robust WLS estimation -in cases of doubt, both robust WLS and robust OLS results can be reported

11. 8.4 Averages and Het Heteroskedasticity will always exist when AVERAGES are used -when using averages, each observation is the sum of all individual observations divided by group size: -Therefore in our true regression, our error term is the sum of all individual observations’ error terms divided by group size:

12. 8.4 Averages and Het If the individual model is homoskedastic, and no correlation exists between groups, then the average equation is heteroskedastic with a weight of hi=1/mi -In this way larger groups receive more weight in the regression and is due to the fact that For example, assume that we run a regression on how math knowledge impacts grades in econ classes. Bigger classes (Econ 299) would be weighted to give more information than smaller classes (Econ 349.5 – Love and Econ.)

13. 8.4 Feasible GLS -In the previous section we assumed that we knew the form of the heteroskedasticity, hi(x) -often this is not the case an we need to use data to estimate hihat -this yields an estimator called FEASIBLE GLS (FGLS) or ESTIMATED GLS (EGLS) -Although h(x) can be measured many ways, we assume that

14. 8.4 Feasible GLS -Note that while the BP test for Het assumed Het was linear, here we allow for non-linear Het -although testing for linear Het is effective, correcting for Het has issues with linear models as h(X) could be negative, making Var(u|X) negative -since delta is unknown, it must be estimated -using (8.30), -Where v, conditional on X, has a mean of unity

15. 8.4 Feasible GLS -Where e has zero mean and is independent of X -note that the intercept changes, which is unavoidable but not drastically important -as usual, we only have residuals, not errors, so we run the regression and obtain fitted values -If we assume v is independent of X, -To obtain:

16. 8.4 FGLS To use FGLS to correct for Heteroskedasticity, • Regress y on all x’s and obtain residuals uhat • Create log(uhat2) • Regress log(uhat2) on all x’s and obtain fitted values ghat • Estimate hhat=exp(ghat) • Run WLS using weights 1/hhat

17. 8.4 FGLS If we used the actual h(X), our estimator would be unbiased and BEST -since h(X) is estimated using the same data as FGLS, it is biased and therefore not BEST -however, FGLS is consistent and asymptotically more efficient than OLS -therefore FGLS is a good alternative to OLS in large samples -note that FGLS estimates are interpreted the same as OLS -note also that heteroskedasticity-robust standard errors can always be calculated in cases of doubt

18. 8.4 FGLS Alternative One alternative is to estimate ghat as: Using fitted y values from the OLS equation -This changes step 3 above, but the remaining steps are the same -Note that the Park (1996) test is based on FGLS but is inferior to our previous tests due to FGLS only being consistent

19. 8.4 F Tests and WLS When conducting F tests using WLS, • First estimate the restricted and unrestricted model using OLS • After determining weights, use these weights on both the restricted and unrestricted model • Conduct F tests Luckily most econometric programs have commands for joint tests

20. 8.4 WLS vs. OLS – Cage Match In general, WLS and OLS estimates should always differ due to sampling error -However, some differences are problematic: • If significant variables change signs • If significant variables drastically change magnitudes -This usually indicates a violation of a Gauss-Markov assumption, generally the zero conditional mean assumption (MLR.4) -this violation would cause bias -the Hausman (1978) test exists to test for this, but “eyeballing” is generally sufficient

21. 8.5 Linear Probability Model We’ve already seen that the Linear Probability Model (LPM), where y is a Dummy Variable, is subject to Heteroskedasticity -the simplest way to deal with this Het is to use OLS estimation with heteroskedastic-robust standard errors -since OLS estimators are generally inefficient in LPM, we can use FGLS:

22. 8.5 LPM and FGLS We know that: Where p(X) is the response probability; probability that y=1 -OLS gives us fitted values and estimates variance using Given that we now have hhat, we can apply FGLS, except for one catch…

23. 8.5 LPM and FGLS If our fitted values, yhat, are outside our (0,1) range, hhat becomes negative or zero -if this happens WLS cannot be done as each observation i is multiplied by The easiest way to fix this is to use OLS and heteroskedasticity-robust statistics -One alternative is to modify yhat to fit in the range, for example, let yhat=0.01 if yhat is too low and yhat=0.99 if yhat is too high -unfortunately this is very arbitrary and thus not the same among estimations

24. 8.5 LPM and FGLS To estimate LPM using FGLS, • Estimate the model using OLS to obtain yhat • If some values of yhat are outside the unit interval (0,1), adjust those yhat values • Estimate variance using: 4) Perform WLS estimation using the weight hhat

25. 8. Heteroskedasticic Review 1) Heteroskedasticity does not affect consistency or biasedness, but does affect standard errors and all tests 2) 2 ways to test for Het are: a) Breuch-Pagan Test b) White Test 3) If the form of Het is know, WLS is superior to OLS 4) If the form of Het is unknown, FGLS can be run and is asymptotically superior to OLS 5) Failing 3 or 4, heteroskedastic-robust standard errors can be used in OLS