Assessing the Impact of Microfinance: A Methodological Study Using Evidence from India

1. Assessing the Impact of Microfinance: A Methodological Study Using Evidence from India Maren Duvendack Procedural Paper Presentation 23 May 2008 Supervisors: Arjan Verschoor & Nitya Rao

2. Introduction to Microfinance • What is microfinance? • Provision of financial and non-financial services to low-income households • Microfinance important strategy in the fight against poverty • Importance of microfinance recognised by United Nations and Nobel Prize Committee • No clear empirical evidence yet that microfinance has positive impacts • Impact assessments crucial for donors and microfinance institutions

3. Introduction of Research Project • Challenge of every impact assessment: • Measurement of counterfactual • Elimination of biases (i.e. selection & attrition bias) • Limited number of rigorous impact studies exist • Study intends to focus on methodological challenges of microfinance impact assessment studies • Suggest solutions to bias problem

4. Research Questions • What is the impact of microfinance on the households’ economic and social well-being? • How are microfinance assessment studies measuring the impact of microfinance? • What are the methodological challenges of microfinance impact assessments? • How can a rigorous treatment of biases, in particular drop-outs, improve the accuracy of impact assessment studies?

5. Research Context • Financial exclusion of India’s poor recurring problem for almost 100 years • Access to finance poverty reduction, thus Indian government launched various policy initiatives aimed at financial inclusion • BUT: Most government-run subsidised credit programmes had negative effects • Emergence of microfinance in India mainly due to lack of effective government policies

6. Research Context • Emergence of microfinance in India in the 1990s • Tremendous growth of Indian microfinance in terms of outreach and loan disbursements • BUT: Only 8 impact assessment studies conducted in India • Studies vary significantly in terms of scope and approach • They investigate one or more of the following impacts: • Poverty reduction • Financial services • Women’s empowerment • Studies provide conflicting results,  impact of microfinance unclear • Thus, more systematic approach to impact assessments needed

7. Mediating Processes Outcomes for the agent and/or other agents Behaviours and practices over a period of time Agent The difference between outcomes is the impact Impact Modified behaviours and practices over a period of time Agent Modified outcomes for the agent and/or other agents Mediating Processes Program Intervention Conceptual Approaches • Core elements of conceptual frameworks in impact assessments: • The impact chain model • The units of assessment • The impact type The impact chain model: Source: Hulme, 2000.

8. Conceptual Approaches • Units of assessment: • Individual, enterprise, household, community and institutional level • Majority of studies examine impact at multiple levels • Identification of impact type: • Economic, social or socio-political impacts • Early impact studies mainly investigated economic impacts, using indicators such as income, assets and expenditure • In the 1980s, focus on social impacts, using indicators such as education, health, housing and sanitation • More recently, shift towards socio-political indicators such as women’s empowerment

9. Paradigms of Impact Assessments • Attribution additional challenge of impact assessments • Two main paradigms can be extracted commonly used to demonstrate attribution: Scientific Method Humanities Tradition

10. Scientific Method • Typically attempts to attribute effects of an intervention to its causes by utilising either…

11. Humanities Tradition • Humanities tradition seeks to explain & interpret the underlying processes of an intervention • Dual function: • Triangulation to crosscheck quantitative data • Provides understanding of changes in social relationships • Difficulties in demonstrating attribution due to lack of control group approach • Causality inferred by collecting data on causal chain by interviewing programme participants, then comparison to data from areas which did not have access to programme

12. Methodological Challenges: Biases • Biases common occurrence in impact evaluations adversely effect impact results, thus solution crucial • Typically the following biases occur in the context of microfinance: • Selection bias: self-selection & non-random programme placement • Attrition bias • Only handful of rigorous impact studies exist that control for biases: • Hulme and Mosley (1996) • Coleman (1999) • Pitt and Khandker (1998) • Alexander and Karlan (2007)

13. Selection Bias – Hulme & Mosley, Coleman Hulme and Mosely (1996) study of microfinance programmes in seven different countries • Controlled for self-selection bias but not non-random programme placement bias • Novelty: sampling of prospective clients as a control group • Mixed results, depending on programme design and country context Coleman (1999) study on Thailand, uses village-level fixed-effects to control for non-random programme placement bias • Also, he uses Hulme & Mosley’s (1996) approach of sampling prospective clients as a control group • Difference-in-difference approach employed • Little impact found, more importantly microfinance led to vicious circle of bad debts

14. Would not be eligible Not eligible Would be eligible Participants Eligible but do not participate Selection Bias – Pitt & Khandker • Until today, most rigorous attempt at controlling for selection bias • Quasi-experiment & eligibility requirements used to measure programme impact • Primary eligibility criterion: landownership “Treatment” Village “Control” Village Source: Armendáriz de Aghion and Morduch, 2005. • Overall findings: microcredit has positive impacts • BUT: accuracy of results disputed due to lax enforcement of eligibility criteria • Econometric debate between Pitt & Khandker and Morduch, not resolved until today

15. Selection Bias – Solution? • Propensity score matching (PSM) popular method used to eliminate selection bias • Works by matching participants to non-participants based on predicted probability of programme participation or the “propensity score” • Basis for matching: observable characteristics  drawback • Underlying assumption: no selection bias due to unobservables • Combine PSM with difference-in-difference, picks up on unobservables but baseline data set required • PSM results good approximation to those obtained under experimental approach

16. Attrition Bias • Drop-out rates estimated to be between 3.5% to 60% in microfinance programmes worldwide • Two different types of clients exiting: • Graduates • Drop-outs • Attrition bias neglected by majority of studies, Alexander and Karlan (2007) one of the few recognising its importance • Solution to attrition bias: • Better sampling • Systematic interviews with drop-outs

17. Methodology – Research Design • Mainly a quantitative study with selected qualitative elements • Questionnaire survey of 500 households • Semi-structured interviews with selected key borrowers, in particular drop-outs • Study proposes to employ propensity score matching (PSM) as a means to control for selection bias as well as attrition bias  novelty in the context of microfinance • Requirement: sampling of participants and non-participants as well as drop-outs

18. Methodology – Overview (1)

19. Methodology – Overview (2)

20. State: Andhra Pradesh Region: Telangana District: Khammam Mandal: tbd Village: tbd Household: tbd Methodology – Sampling Procedure • Study proposes to employ multistage cluster sampling, as illustrated by figure • Sampling in stages: first, identify large areas, then narrow them down by selecting smaller areas within those larger ones • Research location: Andhra Pradesh • Sample selection criteria: • Mature microfinance programmes preferred, at least 5 years of operation • Participants: 4-5 loan cycles required

21. Methodology – Ethics • Oral and/or written consent of research participants shall be obtained before embarking on data collection • Data collected shall be kept confidential and will be anonymised • Reliance on research assistant and translators is expected, they shall be treated with the utmost respect and their expenses shall be covered by the researcher

22. 2008 2009 2010 Jan Mar May Jul Sep Nov Jan Mar May Jul Sep Nov Jan Mar May Jul Sep Nov Preparation Procedural Paper Fieldwork Recce 23 May Procedural Paper Presentation Preparation Fieldwork Fieldwork in India Data Analysis Writing-Up Timeline

23. Q & A Session