Econometric analysis informing policies UNICEF workshop, 13 May 2008

1. Econometric analysis informing policiesUNICEF workshop, 13 May 2008 Christian Stoff Statistics Division, UNESCAP, stoff@un.org

2. Outline • Causality in experiments • Confounding factors • Quasi experiments: Difference estimators • Difference-in-difference estimators • Possible questions

3. A quick intro: The ideal situation • Causality means a specific action leads to a specific, measurable consequence • Ideal: Randomized controlled experiment • Subjects are randomly allocated to either control or treatment group • ONLY difference between two groups is treatment

4. The reality • In practice experiments are rare • Subjects are NOT randomly assigned so that sorting out of other relevant factors is difficult • Econometrics provides the tools for controlling these other factors

5. Challenge: Confounding factors • Regress test score on student-teacher ratio • But what about number of students in class still learning English? – Omitted variable? • Omitted variable correlated with explanatory and dependent variable (low student-teacher ratios -> high % English learners -> bad scores) • Thus a policy of increasing no. teachers may not increase test scores because high English learners (%) are the real problem • Solution: Control for differences in English learners (%), i.e. regress test score on student-teacher ratio AND English learners (%)

6. Limits to controlling these factors • Many years of cross-country research • However, countries often have such different settings and the “causal” relationships are only specific to the country and the time period • Therefore in search of quasi or natural experiments between units that are “not too different” • Danger lies in… • Possible correlation between error term and explanatory variable (i.e. treatment not assigned at random) • Teachers try especially hard in areas with programs • General equilibrium effects: when program is enlarged additional factors may arise (external validity)

7. Difference estimators: Using MICS3 • Define unit of analysis (households, districts, provinces, countries) • Selected units gone through policy program (i.e. treatment) AND assignment was “as if” random • If is binary, then no functional form assumption needed; it is simply the difference in the conditional expectations • If can take on multiple values, then the above regression assumes linearity • But often there are pre-treatment differences between control and treatment group…

8. Difference-in-difference estimators: Using MICS2 and MICS3 • Types of datasets: Cross-section, panel and time-series • Includes observations on same units before and after experiment • OLS estimator is the difference in the group means of • Control for district-level context constant over time through fixed or random effects or adjust standard errors for clustering • Advantage over difference estimator: 1. More efficient; 2. Eliminates pre-treatment differences

9. Some possible questions • Education research: • Study drop-out rates and relate it to child labour questions • Study effect of different child disciplining strategies (punishment, praise, etc.) on a child’s “success” in school • Combine MICS data with GIS disaster data and study effect of disasters on school attendance • Combine with policy data between 2000-2005 and evaluate the effectiveness of policies aimed at promoting higher school attendance • Child health: • Effect of different fuel types for cooking on child-health indicators? • Effects of different types of access to water and sanitation on a child’s probability of having diarrhoea or succeeding in school? • How does Vitamin A affect a child’s health? • Impact of different health service facilities • Adult’s knowledge and attitude towards violence: • What is the effect of having information access (TV or radio) on knowledge about HIV or contraception? • What is the effect of having information access (TV or radio) on education methods or attitudes towards domestic violence?