Quantile Regression

1. Quantile Regression

2. Quantile Regression • The Problem • The Estimator • Computation • Properties of the Regression • Properties of the Estimator • Hypothesis Testing • Bibliography • Software

3. Quantile Regression The Problem

4. Quantile Regression • Problem • The distribution of Y, the “dependent” variable, conditional on the covariate X, may have thick tails. • The conditional distribution of Y may be asymmetric. • The conditional distribution of Y may not be unimodal. Neither regression nor ANOVA will give us robust results. Outliers are problematic, the mean is pulled toward the skewed tail, multiple modes will not be revealed.

5. Quantile Regression • Problem • ANOVA and regression provide information only about the conditional mean. • More knowledge about the distribution of the statistic may be important. • The covariates may shift not only the location or scale of the distribution, they may affect the shape as well.

6. Quantile Regression

7. Quantile Regression

8. Quantile Regression

9. Quantile Regression The Estimator

10. Quantile Regression • Ordinarily we specify a quadratic loss function. That is, L(u) = u2 • Under quadratic loss we use the conditional mean, via regression or ANOVA, as our predictor of Y for a given X=x.

11. Quantile Regression • Definition: Given p ∈ [0, 1]. A pth quantile of a random variable Z is any number ζp such that Pr(Z< ζp ) ≤ p ≤ Pr(Z ≤ζp ). The solution always exists, but need not be unique.Ex: Suppose Z={3, 4, 7, 9, 9, 11, 17, 21} and p=0.5 then Pr(Z<9) = 3/8 ≤ 1/2 ≤ Pr(Z ≤ 9) = 5/8

12. Quantile Regression • Quantiles can be used to characterize a distribution: • Median • Interquartile Range • Interdecile Range • Symmetry = (ζ.75- ζ.5)/(ζ.5- ζ.25) • Tail Weight = (ζ.90- ζ.10)/(ζ.75- ζ.25)

13. Quantile Regression • Suppose Z is a continuous r.v. with cdf F(.), then Pr(Z<z) = Pr(Z≤z)=F(z) for every z in the support and a pth quantile is any number ζp such that F(ζp) = p • If F is continuous and strictly increasing then the inverse exists and ζp =F-1(p)

14. Quantile Regression • Definition: The asymmetric absolute loss function is • Where u is the prediction error we have made and I(u) is an indicator function of the sort

15. Quantile RegressionAbsolute Loss vs. Quadratic Loss

16. Quantile Regression • Proposition: Under the asymmetric absolute loss function Lp a best predictor of Y given X=x is a pth conditional quantile. For example, if p=.5 then the best predictor is the median.

17. Quantile Regression • Definition: A parametric quantile regression model is correctly specified if, for example,That is, is a particular linear combination of the independent variable(s) such thatwhere F( ) is some univariate distribution.

18. Quantile Regression .25

19. Quantile Regression • Definition: A quantile regression model is identifiable ifhas a unique solution.

20. Quantile Regression • A family of conditional quantiles of Y given X=x. • Let Y=α + βx + u with α = β = 1

21. Quantile Regression

22. Quantile Regression

23. Quantile Regression

24. Quantile Regression • Quantiles at .9, .75, .5, .25, and .10. Today’s temperature fitted to a quartic on yesterday’s temperature.

25. Quantile Regression Computation of the Estimate

26. Quantile RegressionEstimation • The quantile regression coefficients are the solution to • The k first order conditions are

27. Quantile Regression Estimation • The fitted line will go through k data points. • The # of negative residuals ≤ np ≤ # of neg residuals + # of zero residuals • The computational algorithm is to set up the objective function as a linear programming problem • The solution to (1) – (2) [previous slide] need not be unique.

28. Quantile Regression Properties of the Regression

29. Quantile RegressionProperties of the regression • Transformation equivarianceFor any monotone function, h(.), since P(T<t|x) = P(h(T)<h(t)|x). This is especially important where the response variable has been censored, I.e. top coded.

30. Properties • The mean does not have transformation equivariance since Eh(Y) ≠ h(E(Y))

31. Quantile RegressionEquivariance • Practical implications

32. Equivariance • (i) and (ii) imply scale equivariance • (iii) is a shift or regression equivariance • (iv) is equivariance to reparameterization of design

33. Quantile RegressionProperties • Robust to outliers. As long as the sign of the residual does not change, any Yi may be changed without shifting the conditional quantile line. • The regression quantiles are correlated.

34. Quantile Regression Properties of the Estimator

35. Quantile RegressionProperties of the Estimator • Asymptotic Distribution • The covariance depends on the unknown f(.) and the value of the vector x at which the covariance is being evaluated.

36. Quantile RegressionProperties of the Estimator • When the error is independent of x then the coefficient covariance reduces to where

37. Quantile RegressionProperties of the Estimator • In general the quantile regression estimator is more efficient than OLS • The efficient estimator requires knowledge of the true error distribution.

38. Quantile RegressionCoefficient Interpretation • The marginal change in the Θth conditional quantile due to a marginal change in the jth element of x. There is no guarantee that the ith person will remain in the same quantile after her x is changed.

39. Quantile Regression Hypothesis Testing

40. Quantile RegressionHypothesis Testing • Given asymptotic normality, one can construct asymptotic t-statistics for the coefficients • The error term may be heteroscedastic. The test statistic is, in construction, similar to the Wald Test. • A test for symmetry, also resembling a Wald Test, can be built relying on the invariance properties referred to above.

41. Heteroscedasticity • Model: yi = βo+β1xi+ui , with iid errors. • The quantiles are a vertical shift of one another. • Model: yi = βo+β1xi+σ(xi)ui , errors are now heteroscedastic. • The quantiles now exhibit a location shift as well as a scale shift. • Khmaladze-Koenker Test Statistic

42. Quantile RegressionBibliography • Koenker and Hullock (2001), “Quantile Regression,” Journal of Economic Perspectives, Vol. 15, Pps. 143-156. • Buchinsky (1998), “Recent Advances in Quantile Regression Models”, Journal of Human Resources, Vo. 33, Pps. 88-126.

43. Quantile Regression • S+ Programs - Lib.stat.cmu.edu/s • www.econ.uiuc.edu/~roger • http://Lib.stat.cmu.edu/R/CRAN • TSP • Limdep