1 / 67

Inference in the General Normal Linear Model

Inference in the General Normal Linear Model. Hypothesis Testing Under General Linear Model. Current model: Y=X β + e How is the dependent variable, Y, influenced by changes in explanatory variables?

halil
Download Presentation

Inference in the General Normal Linear Model

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Inference in the General Normal Linear Model

  2. Hypothesis Testing Under General Linear Model • Current model: Y=Xβ + e • How is the dependent variable, Y, influenced by changes in explanatory variables? • How do we test hypotheses about the magnitudes of such influences or combinations of individual coefficients (e.g., elasticities)?

  3. Hypothesis Testing Under General Linear Model • Previously we derived the sampling property results assuming normality: • Y = Xb + e where et~N(0,s2) • → Y~N(Xb,s2IT) • bs=(X'X)-1X'Y, E(bs)=b • Cov(bs)= β=s2(X'X)-1 • bl~N(b, s2(X'X)-1) • σU2 unbiased estimate of σ2 • An estimate of Cov(βs) = βs=σU2(X'X)-1 el = y - Xβl

  4. Hypothesis Testing Under General Linear Model • Single Parameter (βk,L) Hypothesis Test • βk,l~N(βk,Var(βk)) Σβs=σu2(X'X)-1 kth diagonal element of βs unknown true coeff. • When σ2 is known: • When σ2 not known:

  5. Hypothesis Testing Under General Linear Model • Can obtain (1-) CI for βk: • There is a (1-α) probability that the true unknown value of β is within this range • Does this interval contain our hypothesized value? • If it does, than we can not reject H0

  6. Hypothesis Testing Under General Linear Model • Testing More Than One LinearCombination of Estimated Coefficients • Assume we have a-priori information about the value of β • We can represent this information via a set of J-Linear hypotheses (or restrictions): • In matrix notation

  7. Hypothesis Testing Under General Linear Model known coefficients

  8. Hypothesis Testing Under General Linear Model • Assume we have a model with 5 parameters to be estimated • Joint hypotheses: β1=8 and β2=β3 • J=2, K=5 β2-β3=0

  9. Hypothesis Testing Under General Linear Model • How do we obtain parameter estimates if J hypotheses are true? • Constrained (Restricted) Least Squares • bR is β that minimizes: S=(Y-Xβ)'(Y-Xβ) s.t. Rβ=r = e'e s.t. Rβ=r e.g. we act as if H0 are true • S*=(Y-Xβ)'(Y-Xβ)+λ'(r-Rβ) • λ is (J x1) Lagrangian multipliers associated with J-joint hypotheses • We want to choose β such that we minimize SSE but also satisfy the J constraints (hypotheses), βR

  10. Hypothesis Testing Under General Linear Model • Min. S*=(Y-Xβ)'(Y-Xβ) + λ'(r-Rβ) • What and how many FOC’s? • K+J FOC’s K-FOC’s J-FOC’s

  11. Hypothesis Testing Under General Linear Model S*=(Y-Xβ)'(Y-Xβ)+λ'(r-Rβ) • What are the FOC’s? CRM βS • Substitute these FOC into 2nd set ∂S*/∂λ = (r-RβR) = 0J →

  12. Hypothesis Testing Under General Linear Model • The 1st FOC • Substitute the expression for λ/2 into the 1st FOC:

  13. Hypothesis Testing Under General Linear Model • βR is the restricted LS estimator of β as well as the restricted ML estimator • Properties of Restricted Least Squares Estimator • →E(bR) b if Rb  r • V(bR) ≤ V(bS) →[V(bS) - V(bR)]is positive semi-definite • diag(V(bR)) ≤ diag(V(bS)) True but Unknown Value

  14. Hypothesis Testing Under General Linear Model • From above, if Y is multivariate normal and H0 is true • βl,R~N(β,σ2M*(X'X)-1M*') ~N(β,σ2M*(X'X)-1) • From previous results, if r-Rβ≠0 (e.g., not all H0 true), estimate of β is biased if we continue to assume r-Rβ=0 ≠0

  15. Hypothesis Testing Under General Linear Model • The variance is the same regardless of he correctness of the restrictions and the biasedness of βR • → βR has a variance that is smaller when compared to βs which only uses the sample information.

  16. Hypothesis Testing Under General Linear Model • Beer Consumption Example Joint Hypotheses: • Real Prices Matter? • All prices and INC  by 10% • β1 + β2 + β3 + β4=0 • Equal Price Impacts? • Liquor and Other Goods • β2=β3 • Unitary Income Elasticity? • β4=1 • Data used in the analysis

  17. Hypothesis Testing Under General Linear Model • Given the above, what does the R-matrix and r vector look like for these joint tests? • Lets develop a test statistic to test these joint hypotheses • We are going to use the Likelihood Ratio (LR) to test the joint hypotheses

  18. Hypothesis Testing Under General Linear Model • LR=lU*/lR* • lU*=Max [l(|y1,…,yT); =(β, σ) ] = “unrestricted” maximum likelihood function • lR*=Max[l(|y1,…,yT); • =(β, σ); Rβ=r] = “restricted” maximum likelihood function • Again, because we are possibly restricting the parameter space via our null hypotheses, LR≥1

  19. Hypothesis Testing Under General Linear Model • If lU* is large relative to lR*→data shows evidence that the restrictions (hypotheses) are not true (e.g., reject null hypothesis) • How much should LR exceed 1 before we reject H0? • We reject H0 when LR ≥ LRC where LRC is a constant chosen on the basis of the relative cost of the Type I vs. Type II errors • When implementing the LR Test you need to know the PDF of the dependent variable which determines the density of the test statistic

  20. Hypothesis Testing Under General Linear Model • For LR test, assume Y has a normal distribution • →e~N(0,σIT) • This implies the following LR test statistic (LR*) • What are the distributional characteristics of LR*? • Will address this in a bit

  21. Hypothesis Testing Under General Linear Model • We can derive alternative specifications of LR test statistic • LR*=(SSER-SSEU)/(Js2U) (ver. 1) • LR*=[(Rbe-r)′[R(X′X)-1R′]-1(Rbe-r)]/(Js2U)(ver. 2) • LR*=[(bR-be)′(X′X)(bR-be)]/(Js2U) (ver. 3) βe =βS=βl • What are the Distributional Characteristics of LR* (JHGLL p. 255) • LR* ~ FJ,T-K • J = # of Hypotheses • K= # of Parameters • (including intercept)

  22. Hypothesis Testing Under General Linear Model • Proposed Test Procedure • Choose a = P(reject H0| H0 true) = P(Type-I error) • Calculate the test statistic LR* based on sample information • Find the critical value LRcrit in an F-table such that: a = P(F(J, T – K)³ LRcrit), where α = P(reject H0| H0 true) f(LR*) α = P(FJ,T-K ≥ LRcrit) LRcrit α

  23. Hypothesis Testing Under General Linear Model • Proposed Test Procedure • Choose a = P(reject H0| H0 true) = P(Type-I error) • Calculate the test statistic LR* based on sample information • Find the critical value LRcrit in an F-table such that: a = P(F(J, T – K)³ LRcrit), where α = P(reject H0| H0 true) • Reject H0 if LR* ³ LRcrit • Don’t reject H0 if LR* < LRcrit

  24. Hypothesis Testing Under General Linear Model • Beer Consumption Example • Does the regression do a better job in explaining variation in beer consumption than if assumed the mean response across all obs.? • Remember SSE=(T-K)σ2U • Under H0: All slope coefficients=0 • Under H0, TSS=SSE given that that there is no RSS and TSS=RSS+SSE

  25. Hypothesis Testing Under General Linear Model SSE = 0.059972 *25 = 0.08992 R2=1- 0.08992/0.51497 TSS=SSER Mean of LN(Beer)

  26. Hypothesis Testing Under General Linear Model • Results of our test of overall significance of regression model • Lets look at the following GAUSS Code • GAUSS command: • CDFFC(29.544,4,25)=3.799e-009 • CDFFC Computes the complement of the cdf of the F distribution (1-Fdf1,df2) • Unlikely value of F if hypothesis is true, that is no impact of exogenous variables on beer consumption • Reject the null hypothesis • An alternative look

  27. Hypothesis Testing Under General Linear Model • Beer Consumption Example • Three joint hypotheses example • Sum of Price and Income Elasticities Sum to 0 (e.g., β1 + β2 + β3 + β4=0) • Other Liquor and Other Goods Price Elasticities are Equal (e.g., β2=β3) • Income Elasticity = 1 (e.g., β4=1) • cdffc(0.84,3,25)=0.4848

  28. Hypothesis Testing Under General Linear Model • Location of our calculated test statistic PDF F3,25 F 0.84 area = 0.4848

  29. Hypothesis Testing Under General Linear Model • A side note: How do you estimate the variance of an elasticity and therefore test H0 about this elasticity? • Suppose you have the following model: FDXt = β0 + β1Inct + β2 Inc2t + et • FDX= food expenditure • Inc=household income • Want to estimate the impacts of a change in income on expenditures. Use an elasticity measure evaluated at mean of the data. That is:

  30. Hypothesis Testing Under General Linear Model FDXt = β0 + β1Inct + β2 Inc2t + et • Income Elasticity (Γ) is: • How do you calculate the variance of Γ? • We know that: Var(α′Z)= α′Var(Z)α • Z is a column vector of RV’s • α a column vector of constants • Treat β0, β1 and β2 are RV’s. The α vector is: Linear combination of Z

  31. Hypothesis Testing Under General Linear Model • This implies var(Γ) is: (1 x 3) σ2(X'X)-1 (3 x 3) (3 x 1) Due to 0 α value (1 x 1)

  32. Hypothesis Testing Under General Linear Model C22 • This implies: var(Γ) is: C12 2C1C2

  33. Hypothesis Testing Under General Linear Model • Example of Estimation of Single Output Production Function • Example 6.2, Greene, pp:102-104 • SIC-33: Primary Metals • Cross-Sectional • VA=f(Labor, Capital) • 27 observations

  34. Hypothesis Testing Under General Linear Model

  35. Hypothesis Testing Under General Linear Model • Example of Estimation of Production Function • Cobb-Douglas specification • Translog Functional Form • Cobb-Douglas H0: β3= β4= β5=0 • GAUSS code for calculating critical values of various test statistics

  36. Hypothesis Testing Under General Linear Model • Coefficient on lnK<0 (β2 = -1.8931) • Does this mean that its output elasticity (ψY,K ≡ ∂lnY/∂lnK) is negative? • ψY,K=β2 + β4lnK + β5lnL

  37. Hypothesis Testing Under General Linear Model • Using mean of the natural logarithm of inputs → ψY,K=0.5425 • Note, another point of evaluation could have been at the natural logarithm of the means: • Is ψY,K in fact positive? • Var(ψY,K)=W'ΣβSW • ΣβS is the full 6x6 parameter covariance matrix (1 x 1)

  38. Hypothesis Testing Under General Linear Model • Var(ψY,K)=W'ΣβSW=0.01259 • H0: ψY,K=0, H1: ψY,K>0 • t.025,22 = 2.074 (critical value) • = Ψstd. dev. • t=0.5425/0.1122= 4.84 • Reject H0 • Conclude that the output elasticity for capital is positive

  39. Asymptotic Hypothesis Tests • Asymptotic Tests (t→∞) H0: Rβ = r H1: Rβ≠ r • In general, as T→∞, J*FJ,T-K≈χ2J tT-K≈N(0,1) • → If equation error term not normally distributed and have a “large” number of observations→ joint hypothesis test should be based on J*LR*~c2J (Greene:104-108, JHGLL:268-270) • Define Ver. 2 of LR*

  40. Asymptotic Hypothesis Tests • Using the Central Limit Theorem: • If R is known:

  41. Asymptotic Hypothesis Tests • With J=1 • That is, when you have a single linear restriction and the error term is not normally distributed inferences with respect to this single restriction (hypothesis) should be based on a normal distribution rather than a t-distribution

  42. Asymptotic Hypothesis Tests • Proposed Asymptotic Test Procedure: • Choose α = P(Reject H0|H0 true) = P(Type I Error) • Estimate LR** =J x LR* based on the sample information • Find the critical value of such that a=P(c2J >c2J,C) • Reject H0 if LR**>c2J,C, • Do not reject H0 if LR**< c2J,C

  43. Testing Nonlinear Restrictions • Previously limited to linear H0 • Lets extend the model to: • H0: c(β)=r • c(β) may be J nonlinear functions • When J = 1: • t-stat and F-stat is same as before • Need to obtain estimate of variance of nonlinear function, c(β)

  44. Testing Nonlinear Restrictions • Greene (p.109) shows the asymptotic variance of a nonlinear function is: • If c(β) is linear (e.g., Rβ=r) then: σ2U(X'X)-1 Asymptotic variance

  45. Testing Nonlinear Restrictions • Wald statistic for testing J H0: • c(β) = r • W~χ2J • For an example see Greene p.109-110 (1 x 1)

  46. Testing Non-Nested Models • Up to this point our hypotheses have expressed hypotheses in terms of restrictions of the underlying model • For the linear specification we have y = Xβ + ε H0: Rβ = q H1: Rβ≠ q • This could be a limiting type of hypothesis testing environment • Which of two possible sets of regressors are more appropriate • Whether a linear or log-linear model is more appropriate • Lets consider the following: H0: y = Xβ + ε0 H1: y = Zγ + ε1 Can not use previous procedures to evaluate nonnested hypotheses

  47. Testing Non-Nested Models H0: y=Xβ+ε0 H1: y=Zγ+ε1 • Greene, p. 153-159 • Usual procedure is to seek evidence in the sample to reject H0 (with a positive probability of Type I error) • Considering the above non-nested hypothesis, each competing hypothesis is given equal weight • There is no natural null hypothesis • Common for the analyst to first test using one as H0 and then switch roles and undertake a test with the other is H0 • It could happen that both or neither model is rejected

  48. Testing Non-Nested Models H0: y=Xβ+ε0 H1: y=Zγ+ε1 • We will examine 3 methods for undertaking non-nested tests: • Cox’s method based on maximum likelihood procedures • The Encompassing Principle: Can a maintained model explain the features of (encompass) its competitors • The Comprehensive Model Approach: Contains both models as a special case • Encompassing Principle: • Model 0 encompasses Model 1 if the features of Model 1 can be explained by Model 0 but the reverse is not true

  49. Testing Non-Nested Models H0: y=Xβ+ε0 H1: y=Zγ+ε1 • Let • X* be the set of variables in X that are not in Z • Z* be the set of variables in Z that are not in X • W be the set of variables in both • →we can combine H0 and H1 into a single supermodel y = X*β* + Z*γ* + Wδ + ε • H1 is rejected if γ*=0 using conventional F-test (joint linear test) • H0 is rejected if β*=0 using F-test • Two problems • 1st: δ remains a part of β and γ→test does not really distinguish between Model under H0 and under H1 but between model H1 and hybrid model

  50. Testing Non-Nested Models H0: y=Xβ+ε0 H1: y=Zγ+ε1 • Two problems • 2nd: the compound model may have a large number of regressors → collinearity could be a problem • An Alternative Encompassing Model specification: • If H0 is correct, then y may be fully explained by X except for a random error term, ε • Suppose one then attempts to estimate γ by regressing y on Z → • If H0 is correct then the estimated coefficient will be the same if one were to regress Xβ on Z since ε0 is random noise under H0 • Since β must be estimated, use βS instead, XβS = f(Z) → γ0

More Related