Arianna Legovini, Lead Impact Evaluation Specialist Edit V. Velenyi, MIEP Coordinator

1. Arianna Legovini, Lead Impact Evaluation Specialist Edit V. Velenyi, MIEP Coordinator Africa Impact Evaluation Initiative (AIM) Malaria Impact Evaluation Program (MIEP)

2. “The impact ofmalaria extends far beyond the realm of public health and exacts a heavy toll on human and economic prosperity...” • Malaria kills an estimated 1 million people every year • Costs Africa US$12 billion annually in lost GDP • 90 percent of malaria deaths worldwide are in Africa • Yet malaria is both preventable and treatable • Universal coverage would bring the scourge to a halt

3. Strong epidemiological experimental evidence: prevention and treatment work • Lacking behavioral and economic evidence • What will induce people to have their children sleep under a net? Or take the child for treatment within 24 hours? • What are the socio-economic returns to malaria prevention and treatment? • Labor participation and productivity, child cognitive development • Which interventions are most cost-effective? • Private delivery mechanisms • School based malaria interventions

4. AIM - Africa Impact Evaluation Initiative • 80+ Experimental or Quasi Experimental Evaluations • MIEP - Malaria Impact Evaluation Program • Inaugural Workshop, Cape Town May 2007 • Second Workshop Asmara Feb 2008 • Countries: Eritrea, DRC, Kenya, India, Nigeria, Senegal, Zambia • Areas of investigation • Alternative delivery mechanisms and PPP • Complementary interventions (IRS, test kits) • Preventive treatment (IPTs) • Innovative financing for ACTs

5. Improve quality of operations • Separate process from quality of intervention • Test alternatives and inform design in real time • Increase project effectiveness • Answer the “so what” questions • Build relationship with client • Come with options not solutions • Find out together what works best • Assist client adopt better way of doing business and taking decisions

6. Quality assurance • Client-driven & operationally relevant • Feedback loop into policy design Lead researcher & Field Coordinator Working Groups Biometrics Technical Advisory Group Researchers & Specialists Cognitive MIEP Research Team Socio-Economic Knowledge, Attitudes and Practice MBP Project Team Coordinating Unit IE Leads Sector Lead Program Coordinator Team Support Client Government Cost-Effectiveness

7. Alternative delivery mechanisms and PPP • Complementary interventions (IRS, test kits) • Preventive treatment (IPTs) • Innovative financing for ACTs

8. MIEP Portfolio BNPP Recipients • DRC • Nigeria

9. Number one public health problem in Nigeria • 99% of the population in endemic areas (MARA/ARMA 2002) • High Federal and States’ Governments commitment to controlling the disease • 2004 National Economic Empowerment and Development Strategy • development of strategic plans for malaria • Presidential Initiative for Accelerated Achievement of the MDGs • malaria control as a key component of health-related MDGs • 2004 National Health Policy • malaria control as a priority health program

12. Explore innovative modes of delivery and complementary interventions to address limitations of the health system • Weak / insufficient public sector delivery • Human resource constraints • Heterogeneity in the quality of private sector delivery • Insufficient coordination between the public and private sectors

13. Consultations and Capacity Building – National and States • Cape Town Inaugural Workshop: May 2007 • Abuja Design Workshops • MBP7: August 2007 • MBP 4: September 2008 • State Field Visits • Anambra: October 2008 • Gombe: October 2008 • Akwa Ibom: December 2008 • Jigawa: December 2008

14. Nigeria Malaria IE Cycle and Timeline

16. JIGAWA GOMBE ANAMBRA AKWA IBOM

17. STEWARDSHIP National Policy Making ResourceAllocation System Harmonization 1. FMOH / NMCP Program Coordinator Evidence Based Policy Making State-Level Policy Making Implementation Decision 2. STATE MOH Program Managers Supply 3. LGA MBP Focal Person Program Implementation Service Delivery & Reporting Wards WDC Wards WDC Wards WDC ACT Supply (LLIN – BCC) M&E Public Facility (K) Private – PMV T1 Community – RMM T2 Vulnerable Target Group: U5 Children & Pregnant Women IE

18. Counterfactual: Public Sector (K) • Private Sector Intervention Arm: PMV (T1) • What is the added value of using trained Patent Medicine Vendors (PMVs) to increase timely treatment of uncomplicated malaria with ACTs? • Community-Based Intervention Arm: RMM (T2) • What is the added value of using trained Role Model Mothers (RMMs) to increase • increase timely treatment of uncomplicated malaria with ACTs? • effective coverage of LLINs by the vulnerable target groups? • Cross-Over / Factorial Design (T1 + T2) • What is the joint effect of using trained PMVs and RMMs …?

19. Uniform Intervention Across States • Supply Side: Training • Variations • Supply Side: Incentives for PMVs and RMMs • Monetary v. Non-Monetary • Demand Side Approach: Communication • BCC Component in Training for RMMs to Improve Use LLINs • BCC (Community-Based Orientation)

20. Effect of Trained PMVs and RMMs on coverage and use of ACTs • Target Population - U5 Children: 470,770 • MB Program Funds: 6 million • Design: Covers whole state, including all 11 LGAs and 114 Wards • Intervention Arms: 1) PMV 2) RMM 3) PMV + RMM 4) Control • Intensity: 114 Trained PMVs and 114 Trained RMMs • HH Survey: 2,280 HH | Community Survey: 228 | Facility Survey | MIS RMM in 57 Communities PMV in 57 Wards PMV in 57 Wards RMM in 57 Communities Pure Control in 57 Wards

21. Effect of Trained PMVs & RMMs on coverage and use of ACTs, and RRMs’ on LLIN use • Target Population: U5 (836,400) and Pregnant Women (209,100) • MB Program Funds: 9.5 million • Design for State (21 LGAs and 329 Wards) • Intervention Arms: 1) PMV 2) RMM 3) PMV + RMM 4) Control • Intensity: 172 Trained PMVs and 172 Trained RMMs (2 per Ward) • HH Survey: 2,064 HH | Community Survey: 172 | Facility Survey | MIS RMM in 43 Wards PMV in 43 Wards PMV + RMM in 43 Wards Pure Control of 43 Wards

22. Effect of incentives for PMVs & RMMs on coverage and use of ACT, and RRMs’ on LLIN use. • Effect of BCC on coverage and use of ACTs, and RMMs on LLIN use. • Target Population: U5 (784,042) Pregnant Women (196,010) • MB Program Funds: 8.5 million • Design for State (31 LGAs and 329 Wards) • Arms: 1) Incentives 2) BCC 3) Control • 329 Trained PMVs and 329 Trained RMMs (Using 1 each for Focal Village per Ward) • HH Survey: 3,290 HH | Community Survey: 329 | Facility Survey | MIS Incentives 110 Focal Villages (1 FV per Ward) BCC 110 Focal Villages (1 FV per Ward) Control (Training Only) 109 Villages (1FV per Ward)

23. Upgrade Routine MIS: Case Management Reporting • Training and Rollout of Case Management Tool to Health Facilities • Management Tool (McKinsey) • Costing Tool • Cost of Standard Delivery • Cost of Innovative Arms (Incremental Costs) • System Start Up (Intervention Specific M&E Tool) • Intervention Cost (Commodity and Training) • Management Cost (Operating the New System) • Third Party Audit • Focal Person and State Manager • Contracting of NGO/CSO

24. Routine Reporting • Upgraded MIS • Customized MIS Tools • Random Audits • Household Survey • Baseline CWIQ (November - December 2008) • Developed Malaria Section for CWIQ • Follow up Dedicated Survey (November – December 2009) • Community Surveys • Component of HH Survey • Facility Surveys • SFH Facility Census (November 2008) • Facility Survey (November 2009)