Essential Issues for Program Evaluation Panel Discrete Choice Models

1. Program Evaluation andPanel Discrete Data Models – Some Considerations on Methodology and ApplicationsCheng Hsiao

2. Essential Issues for Program Evaluation • Panel Discrete Choice Models • General Principle of Estimating Structural Parameters in the Presence of Incidental Parameters • Parametric Approach • Semi-Parametric Approach • Bias-Reduced Approach • Concluding Remarks

3. Definition of Treatment Effect 1. Essential Issues for Program Evaluation Let denote the potential outcomes of the ith individual in the untreated and treated state. Then, the treatment effects on the ith individual are just

4. Two measures of treatment effects are of interest to policy makers: The average treatment effects (ATE) and the treatment effects on the treated (TT)

5. The ATE is of interest if one is interested in the effect of treatment for a randomly assigned individual or population mean response to treatment. The TT is of interest if the same selection rule for treatment continues in the future.

6. The observed data is in the form of , i = 1,…,N, where if the ith individual receives treatment and if not, and , where i = 1,…,N In other words, we do not simultaneously observe and . An individual is either observed with or . Using as a measure of ATE could be subject to two sources of bias: selection on observables and selection on unobservables. (1)

7. If selection into the treatment is random, (1) can be used to estimate ATE (≡ TT). then

8. If (1) will be a biased estimate of ATE or TT. then

9. Suppose we can decompose the outcomes in terms of the effects of observables and the effects of unobservables as where denotes the effects of observable factors , denotes the effects of unobservable factors, j = 0,1, and . Then and

10. Selection on observables Selection on unobservables

11. Conditional Independence Assumption – selection is ignorable after controlling a set of observable confounders then

12. Adjustment Methods 1. If there is no selection on unobservables (i.e. where j = 0,1) 1a. Propensity Score Matching Method (Rosenbaum and Rubin (1983)) Let , then (i) (ii)

13. (ii) → If a subclass of units or a matched treatment-control pair is homogenous in , the treated and control units in that subclass or matched pair will have the same distribution of , then, at any value of a propensity score, the difference between the treatment and the control means is an unbiased estimate of the ATE at that value of the propensity score (treatment assignment is ignorable)

15. Issues of this approach (i) Conditional independence assumption is not a testable hypothesis (ii) Estimates sensitive to how blocks of are constructed

16. Decriminalization and Marijuana Smoking Prevalence: Evidence from Australia Kannika Damrongplasit, Cheng Hsiao and Xueyan Zhao

17. The global sale of illegal drugs – US $150 billion (2001) US drugs policy costs $30-40 billion a year

19. What is Decriminalization Policy? • Reduction of penalties for possession and cultivation of marijuana for personal consumption (i.e. minor possession) • In Australia, it is called “Expiation System” - Still an offence to use or grow marijuana - The offence is expiable by payment of a fine with no imprisonment and no criminal record if the fine is paid.

20. Debates on Marijuana Decriminalization • Supporting Arguments - Criminal offence from marijuana possession is too severe - Allow separation of marijuana market from other harder drugs - Reduce law enforcement and criminal justice resources • Opposing Arguments - Increase marijuana smoking prevalence - Greater use of other illicit drugs

21. Sources of Data • 2001 National Drug Strategy Household survey (NDSHS) - Nationally representative survey of non-institutionalized civilian population aged 14 and above - 26744 total observations - 14008 resulting samples after delete missing data - Treatment group = 2968, Control group = 11040 (ii) Australian Illicit Drug Report (iii) Australia Bureau of Statistics

23. Summary Statistics

24. Non-parametric Model: Propensity Score Stratification Matching • Propensity score is • Under the assumptions, and then and

27. Violation of could happen because of • Our sample size is not large enough to perform a • reliable non-parametric estimation • - For many ranges of propensity score, there is no • overlapping observations • - Within overlapping range, t-squared tests • unambiguously reject the balancing condition • or • (2) Conditional independence assumption is violated

29. Endogenous Probit Switching Model (Model 1)

30. Average Treatment Effect Model 1 & 2: If sample is randomly drawn, Model 3 & 4:

31. Coefficient estimates for marijuana smoking equation

33. Average Treatment Effect and Marginal Effect

34. Panel Data – Allow better control of selection on observables and unobservables Difference-in-Difference Method - outcome of the jth individual after the treatment - outcome of the jth individual before the treatment - outcome of the ith individual who did not receive treatment at time t - outcome of the ith individual at time s

35. Cross-Section vs Panel Discrete Modeling

40. 6. Bias-Reduced Estimator Mean Square Error = Suppose

41. Consider the log-likelihood function of N cross-sectional units observed over T time periods, where denotes the likelihood function of the T-time series observations for the ith individual. For instance, consider a binary choice model of the form,

42. Then The MLE is obtained by simultaneously solving for from (1) (2)

43. The MLE of can also be derived by first obtaining from (2) as a function of , , substituting into the likelihood function to form the concentrated log-likelihood function, (3) then solving (4)

44. When T is finite Expanding the score of the concentrated log-likelihood around , and evaluating it at : (5)

46. is derived by solving

48. Monte carlo experiments conducted by Carro (2006) have shown that when T = 8, the bias of modified MLE for dynamic probit and logit models are negligible. Another advantage of the Arellano-Carro approach is its generality. For instance, a dynamic logit model with time dummy explanatory variable cannot meet the Honore and Kyriazidou (2000) conditions for generating consistent estimator, but can still be estimated by the modified MLE with good finite sample properties.

49. Advantages of Carro (2006) produce: • No need to transform the parameters of interest into (information) orthogonal parameters as done in Cox and Reid (1987, JRSS B) or Arellano (2003). • No need to impose any conditions on the observed data as in the case of Honore and Kyriazidou (2000). In other words, all observed data can be utilized to obtain .

50. Issues: • may not have a closed form solution. Neither is the evaluation of expectation term trivial. Hence, computationally can be tedious. e.g. in the case of logit model,