Pradeep Panda, PhD Micro Insurance Academy, New Delhi Ellen Van de Poel

1. Reconciling Research and Implementation Needs in Community-Level Cluster Randomised Trials:An EC-FP7 Research Project Pradeep Panda, PhD Micro Insurance Academy, New Delhi Ellen Van de Poel Erasmus University Rotterdam

2. Introduction • Micro-Insurance Academy is an Indian charity dedicated to training, research, technical assistance and advisory services for micro-insurance units serving the poor • With our partners, we are running 3 separate, but similar, RCTs in northern India • Each RCT will establish a Community Based Health Insurance (CBHI) scheme – a small-scale, member-operated health insurance scheme, offering limited coverage of defined events • Each scheme will be evaluated for its effect on healthcare utilization and healthcare financing • Now 30 months into this 5-year project – all CBHI units went live in 2011

3. 1. What do we Know about Micro Health Insurance (MHI)?

4. 1. What do we Know about Micro Health Insurance? • Limited robust evidence on causal impact of MHI schemes: • Of the 22 most robustly studied trials, only 2 use a randomised trial methodology… • … and only 7 use any form of counterfactual! • The remaining evaluations are prone to bias • No evaluation has examined a holistic list of outcome indicators • And comparability of results across trials may be limited, as all kinds of different schemes have been called “MHI:” national public schemes, private for-profits products, operator run pre-payment programs, and non-profit mutual finds!

5. 2. Overview of Trials

6. 2.1 Motivation for Trials • These trials aim to close knowledge gaps on MHI in three ways: • Use the “gold standard” randomised trial methodology to boost internal validity • Give clear and detailed descriptions of scheme setting and operation to clarify limits of external validity • Evaluate a holistic range of outcome indicators to broaden the knowledge base

7. 2.2 Scheme Areas & Target Populations 3 separate CRTs at 3 separate sites: Uttar Pradesh Bihar 1. Kanpur Dehat 2. Pratapgarh 3. Vaishali

8. 2.2 Scheme Areas & Target Populations • At each site, a particular type of MHI scheme is set up: • Community Based Health Insurance (CBHI) • CBHI schemes are locally based, mutual, not-for-profit programs: • All schemes are ownedby members • All schemes are managedby members • All premiums and coverage types are set by members • This is why we call it Community Based!

9. 2.2 Scheme Areas & Target Populations • Implementation at each site is managed by an NGO implementing partner • Each NGO has a network of Self Help Groups (SHGs): village level MFI groups • At each site, there are 1400 – 1600 SHG members • Members and their families can take part in the trial

10. 2.3 Description of Treatment Program • 4 sequential modules create an eventual impact: 1. Design Benefit Options workshop (x1) Awareness Tools workshop (x1) Design workshop (x1) 2. Awareness Insurance Education (x3) CHAT (x2) Finalization (x1) Enrollment (x1) 3. Launching Selection of Officers Training of Officers Installation of MIS Stakeholder Events 4. Live Scheme Submission of Claims Processing of Claims Payout of Claims Key Outcome Indicators

11. 3. Overview of Research Design

12. 3.1 Outcome Measures & Tools 1. Effects of CBHI on Healthcare Utilization Levels Key indicators depend on coverage of insurance packages at each trial. Will be drawn from: Outpatient Care Usage Rates Hospitalisation Rates Involuntary Non-Treatment Rates Maternity Care Usage Rates Use of Transport

13. 3.1 Outcome Measures & Tools 2. Effects of CBHI on Healthcare Financing Key Indicators include: Instance of catastrophic health expenditure (>10% HH Income) Instance of asset sales, savings liquidation, etc. Total Healthcare Spending Financial Exposure Index (Under Development) 3. Physical Accessibility of Healthcare Village-wise Health Care Index

14. 3.1 Outcome Measures & Tools 3 mutually supportive and integrated research streams: This presentation shows have quantitative and spatial baseline have driven experimental design…. Quantitative • Household surveys • Healthcare Provider surveys • Exit interviews • Income surveys • Insurance Understanding surveys Qualitative • FGDs - SHG members • FGDs - Heads of SHG households • KIIs - SHG leaders • KIIs - Local Healthcare Providers Spatial • GPS Mapping • Satellite Imaging • GIS Imaging

15. 3.2 Important design aspects of the RCT • Staggeredimplementation • Each wave a random third of the target populationgetsoffered CBHI • Clustered trial • SHGs are groupedintoclusters • Clusters are randomized in 3 treatment groups • En-blocaffiliation • All hhs within a SHG needtojoin the CBHI

16. 3.3 Defining an Implementation Friendly Unit of Randomisation • combine Quantitative and Spatial data, and map locations of trial participants • “3 Rules” to integrate implementation and research needs when forming clusters: • Non-Divisibility: A village cannot be divided over different clusters • Equal Size: Clusters must contain (roughly) equal numbers of SHG members • Continuity: Each cluster must be geographically continuous

17. 3.3 Defining an Implementation Friendly Unit of Randomisation This transforms villages into clusters….

18. 3.4 Generating Power with a Fixed Population Size Then MDES is calculated: If it is too low, one of the three rules must be broken somewhere to improve power

19. 3.5 Randomization • Matching prior torandomization was considered (creatingsimilar triplets of clusters) but didnotimprovebalance on observables • Simple randomization was chosen as it does not affect power • Lessbalance on ethnicity/caste & health care supply

20. 3.6 Limitations • Externalvalidity: • Only effect of CBHI on SHG • Selection of implementation partners • Internalvalidity: • Self-selection of SHGs in CBHI -> intentiontotreat effect • Contaminationeffects • (health related) attrition • Multiple treatments (throughparticipatory design of CBHI)

21. Thank You!