Impact evaluation Applying Analytical Methods: Microfinance Case Study

1. Impact evaluation Applying Analytical Methods: Microfinance Case Study Evaluating the Impact of Projects and Programs Beijing, China, April 10-14, 2006 Shahid Khandker, WBI

2. Micro-finance Impact Evaluation: An example • Micro-finance refers to financial and non-financial services (often small size) for low-income people; • Micro-finance is a response to market failure; • Micro-finance relies on social mechanisms to enforce contract and to reduce the impacts of capital market imperfection and asymmetric information; • Micro-finance usually depends on grants and soft money for institutional development.

3. What does micro-finance provide? Financial services • Loans • Savings • Insurance • Remittance transfer Non-financial services • Awareness and conscious building • Employment and skill development training • Extension and infrastructure development • Legal and regulatory services

4. How does micro-finance provide services? • Collateral-free lending • Small loan size • Small amount of savings • Group-or individual-based lending/savings • Peer monitoring • Targeted lending • Market interest rate policy • Social intermediation

5. Why is micro-finance needed? • Micro-finance helps expand income and strengthen financial sector. • Micro-finance supports poverty alleviation effort. • Micro-finance increases business opportunities through financial innovations that otherwise would not be adopted. • Micro-finance helps decentralize economic power and strengthen community groups. • Micro-finance empowers women and other socially excluded groups

6. Expected benefits of micro-finance Short-term benefits • Consumption and its smoothing • Employment • Income • Poverty and vulnerability Long-term benefits • Household asset and net worth • Human capital • Social capital • Community assets

7. Policy Issues Regarding Micro-finance • Micro-finance borrowing has a high recovery rate and the potential for reducing poverty, particularly among women; • But these programs are dependent on subsidy which might have alternate uses; • Policy issues focus on the efficiency and cost-effectiveness of micro-finance programs to justify their subsidy dependence.

8. Notation • C is credit demand; • Y is outcome variable (consumption, assets, employment, schooling, family planning); • Subscripts: household i; village j; male m; female f; time t; • Want to allow separate effects on outcome of male and female credit; • X represents individual or household characteristics.

9. Measuring Impacts: Cross-Sectional data Credit demand equations: Cijm = Xijmβc + Zijmγc + μcjm + εcijm Cijf = Xijfβc + Zijfγc + μcjf + εcijf Outcome equation: Yij = Xij βy + Cijm δm + Cijf δf + μyj + εyij Note: the Z represent variables assumed to affect credit demand but to have no direct effect on the outcome.

10. Endogeneity Issues • Correlation among μcjm, μcjfand μyj, and among εcijm, εcijf and εyij • Estimation that ignores these correlations have endogeneity bias. • Endogeneity arises from three sources: 1) Non-random placement of credit programs; 2) Unmeasured village attributes that affect both program credit demand and household outcomes; 3) Unmeasured household attributes that affect both program credit demand and household outcomes.

11. Resolving endogeneity • Village-level endogeneity – resolved by village FE; • Household-level endogeneity – resolved by instrumental variables (IV); • In credit demand equation Zijm and Zijf represent instrumental variables; • Selecting Zij variables is difficult; Possible solution: identification using quasi-experimental survey design.

12. Quasi-experimental design • Households are sampled from program and non-program villages; • Both eligible and non-eligible households are sampled from both types of villages; • Both participants and non-participants are sampled from eligible households. Identification conditions: • Exogenous land-holding; • Gender-based program design.

13. Quasi-experimental Design (contd.) • Exogenous land-holding criteria: - Only households owning up to 0.5 acre of land qualify for program participation. In practice there is some deviation from this cutoff. • Gender-based program participation criteria: - Male members of qualifying households cannot participate in program if village does not have a male program group. - Female members of qualifying households cannot participate in program if village does not have a female program group.

14. Quasi-experimental Survey Design:Construction of Zij variables Male choice=1 if household has up to 0.5 acre of land and village has male credit group =0 otherwise Female choice=1 if household has up to 0.5 acre of land and village has female credit group =0 otherwise Male and female choice variables are interacted with household characteristics to form Zij Variables.

15. What this means….. Intuitively, the outcome regression now relates variation in Y to variation in C associated with variation in Z.

16. Alternative Models for Estimation (--) Ordinary Least Squares (OLS) estimation of outcome equation with no correction for weights; (-) OLS on outcome equation with correction for sampling weights; (+) IV (2-stage) estimation of outcome equation with weights correction; (++) IV, weights, village fixed effects.

17. Comparing Results among Alternative Models: Log-log impacts of GB women’s credit on HH per capita consumption

18. Panel Data Analysis: Rationale • Results based on cross-sectional data may not be robust; • Impacts based on cross-sectional data may be short-term; • Cross-sectional data analysis cannot separate credit effects from non-credit effects; • Panel data analysis can assess whether credit has increasing or diminishing returns; • Panel data analysis can assess whether diseconomies or market saturation exists.

19. Concerns with Panel Data Household attrition: • Not a problem if it is random; • Efforts should be taken to minimize it. Households split-off: • As many component households as possible should be traced; • Component households are logically combined.

20. Estimation with Panel Data – Resolving Endogeneity • Household FE resolves both household- and village-level endogeneity. • So IV (quasi-experimental) method is not required for resolving endogeneity. Outcome equation: Yijt = Xijt βy + Cijmt δm + Cijft δf + μyj + εyijt After differencing between two time-points: ΔYij = ΔXij βy + ΔCijmδm + ΔCijftδf + Δεyij

21. Estimation with Panel Data – New Biases • Household-level unmeasured biases that were assumed as time-invariant may actually change over time (that is we should use ηyjt instead of ηyj); • If credit is measured with error, that error gets amplified when differenced between two time points; • This measurement error results in “attenuation bias” of the credit coefficients, meaning that impact estimates will be biased towards zero.

22. Estimation with Panel Data Re-introduction of instrumental variables (IV) Outcome equation: ΔYij = ΔXijβy + ΔCijmδm + ΔCijfδf + Δηyj + Δεyij Credit demand equations after differencing: ΔCijm = ΔXijmβc + (Zijm2γ2 – Zijm1γ1) + Δεcijm ΔCijf = ΔXijfβc + (Zijf2γ2 – Zijf1γ1) + Δεcijf Since credits are cumulative at two periods, ΔC in above equations represents borrowing after first period.

23. Purpose of Instruments • Same instruments as in cross-section analysis, but with a new purpose; • Before, they controlled for unobserved fixed household specific effects; • Now, they control for unobserved non-constant household specific effects, and measurement error; • Household panel structure is now taking care of the fixed effects.

24. FE-IV method over FE method • Durban-Hausman-Wu test was conducted to see if the estimated coefficients of FE or FE-IV model are significantly different. • Results suggest that changes in credit volume used in the FE method are somewhat determined by the time-varying heterogeneity or the measurement errors in credit variables. That means, the FE method may not be as reliable as one would expect. • For comparison, both results are presented.

25. Comparing Results Among Alternative Models: Log-log impacts of women’s current credit on HH per capita consumption

26. Propensity Score method • Run a program participation equation • Predict propensity score for each household • Match non-participant household with participant household • Take the difference of outcomes of interest between participant and non-participant • Weighted mean difference is the average program effect • Use both cross-sectional and panel data analysis

27. Results of PSM with Cross-sectional Data Set: Average treatment effect of female borrowing 1991/92 1998/98 panel • Per capita expenditure: -140.9 455.8* 587.7* • Moderate poverty: 2.30 -5.40* -7.46* • Extreme poverty: 0.30 -6.70* -14.84* Note: * refers to significance level of 10% or better; results shown based on nearest neighbor matching.

28. Selected References Ravallion, Martin. “The Mystery of Vanishing Benefits” The World Bank Economic Review. Volume 15. No. 1. pp. 115-140. Washington D.C.: The World Bank, 2001. Khandker, Shahidur R. “Micro-finance and Poverty: Evidence Using Panel Data from Bangladesh” The World Bank Economic Review, October, 2005, PP: 263-286. Khandker, Shahidur R. “Fighting Poverty with Micro-credit: Experience in Bangladesh” New York, NY: Oxford University Press, 1998. Pitt, Mark M. and Shahidur R. Khandker. “The Impact of Group-Based Credit Programs on Poor Households in Bangladesh: Does the Gender of Participants Matter?” Journal of Political Economy 106 (October): 958-96, 1998.