Thursday, September 1, 2011 Milles Collines Hotel, Kigali

1. Final evaluation of the kabehomwana (expanded impact) project, 2006-2011districts of gisagara, nyaruguru, kirehe, ngoma, nyamagabeand nyamasheke,Republic of RwandaEric Sarriot, MD, PhD & the Final Evaluation Team Preliminary Findings Thursday, September 1, 2011 Milles Collines Hotel, Kigali

2. Evaluation Objectives Evaluate achievement of EIP objectives Examine quality of interventions and approaches Examine in context questions of equity and sustainability

3. Human Capacity Investment • Presence in the district, Capacity Building and Partnership at District and Sub-District Levels • Training reinforced by coaching

4. Level of Effort in Training 48 types of training, 700 people in training on average for each month of the project; 70 people in training each work day of the project!!! Justine-9 people trained/project hour

5. Methodology – Final Qualitative District Study • 3 Teams ; 6 Districts; August 15-26, 2011 • Hospital: Medical Directors, Supervisors • +DHO • Health Centers: Community Health In-Charge • + Titulaires • Cooperatives • Cell Coordinators and CHWs • Mothers seeking care at HC level

6. Preliminary Findings Child Health Achievements

7. Child Sleeping Under ITN EIP FINAL EIP BASELINE DHS 2010 DHS 2005 Nota: RDHS 2005-2010; Prevalence of Fever: 26% to 16%

9. Appropriate Treatment / Fever EIP = Presumptive Treatment + RDT + when applicable RDHS = Presumptive Treatment

10. Control of Diarrheal Diseases EIP FINAL EIP BASELINE Nota: RDHS 2005-2010; Prevalence of Diarrhea: 14% and 13%

11. Pneumonia / Care Seeking & Treatment EIP FINAL DHS 2010 DHS 2005 EIP BASELINE Nota: RDHS 2005-2010; Prevalence of ALRI symptoms: 17% to 4%

13. Other District Indicators (KPC)

14. End of Project: 69% of mothers have ‘ever used’ a CHW; and 40% of children with an illness in the past 2 weeks (KPC)

15. Conclusions on Achievement of Child Health Objectives + - • Project and District Partners have achieved major improvements • Sick child management (Pneumonia, Malaria) • Improvements in diarrhea management and prevention • Contribution to national gains • Ways to go in diarrhea management • Attribution of results challenging • Role and value of specific strategies… Questions

16. 1- Expanding CCM in 6 Districts Strategic Contributions

17. By the End of Project • 6,168 binomes were involved through • 660 CHW peer groups (“Care Groups”), • 88 Sectors / 84 Health Centers in 6 Districts • EIP adjusted plans and strategies to align to and support national strategy • All 3 conditions considered, the districts of intervention of EIP reported 183,959 treatments to the CHD (out of 567,981* for the country; or 32%) over the last 4 quarters

18. Recent EIP Trend in Utilization of CHWs and Referrals

19. Community Drug Supply * Excluding Nyamagabe for Primo and RDTs and Nyamashake for RDTs

20. CHW/binomes Retention (Jan-June 2011) • Investigated highest reported dropouts • Very Reasonablenatural attrition

21. Conclusions and Achievement of expanding CCM Conclusions + - • Rapid scale up of CCM behind sick child treatment indicators – Substantial contribution to national strategy • High utilization rate • Strong partnership with and buy in from districts and health centers • Supervision happens • Drugs are there • Additional contribution to RDT purchase • Challenges in quality: • Supervision sub-optimal • EIP still involved in drug supply • RDT effect on utilization • Epidemiologic trend and LOE of CHWs • Diarrhea care seeking

22. 2- Quality of Care Strategic Contributions

23. Achievements (beyond training, and drug supply) • Initiated Quarterly Feedback Meeting at HC level with Cell Coordinators • The “C-IMCI Bulletin” as an evolving learning tool • Emphasis on Standards at HC Level – improved quality of health care delivery care in the health facilities (17 indicators in com-Imci) • Coaching role and presence of QA and M&E teams of EIP

24. Illustration of Bulletin Analysis

25. Conclusions on other elements of the QA Component of EIP + - • In addition to training, EIP has made important contributions • Use of information at local level (along with support in reporting); focus on identifying performance gaps • Appreciated proximity coaching / accompaniment presence • Based on an evolving learning approach • Along with PBF, this contribution has laid important basis for evidence based approaches. • Further work will be required to institutionalize full cycles of quality improvement for community health( QA team for Comm.level)

26. 3- Community Mobilization Strategic Contributions

27. Health Promotion Strategy • Messages and Dissemination of Messages – From 2009 on • Support to Campaigns • Production of Communication Support • Engaging Community Leaders • Organizing and Mobilizing CHWs… the Care Groups • Beyond Messages: The Care Group Concept and Practice

28. THE ‘CARE GROUPS’ CHW Peer-Support and Collaboration Groups

29. Care Groups and Health Promotion Approach of c-IMCI • EIP was flexible and adapted the Care Group model to create a CHW Peer Group model: • Requiring very input, scaled through district mechanisms • Aligned with MOH policies • CHW, Cooperative and Community In-Charge identify positive elements and support from these peer groups • Links to livelihood and general development

30. Signs of over-burdening the Cell Coordinators in Reporting and Meetings - Illustration

31. Conclusions

32. EIP – KabehoMwana • Major Contribution to National Health Improvements of last 5 Years • Helping Launch and Scale CCM – 183,000 treatments in the last year • Alignment and Harmonization • Partnership of the consortium “worked”, internally and for the GOR • Presence and the role played by the Program at Kigali level down to the ield (District and below down to Community) • Established critical building foundation for quality monitoring and performance improvement • Implemented the Com Health model for integrating the objectives of the different types of CHWs

33. Enabling environment for EIP • National Community Health Determination by MoH • Highest level involvement +++ by the GOR/MoH

34. Questions on Sustainability • Ensuring the basics: drugs and supervision • Can CHW Peer-Groups provide Leverage for: • Supervision, including peer-supervision, including CCM for cell coordinators • Health promotion will continue even at end of KM • How will actors behave through PBF when the revenues are no longer expanding? • How to maintain focus while addressing enduring and new priorities (neonatal health)?

35. Suggestions / Recommendations

36. To the EIP Consortium PartnersConcern – The IRC – World Relief • For new projects and initiatives in any of the 6 districts: • Based on the lessons of the project, and together as NGOs, promote, hold yourselves accountable to, and share with the stakeholders a code of good practices for enhancing sustainability in follow-on activities.- CG, QI-bulletin, M&E feedback analysis • Potential for joint funding as a consortium

37. To the Community Health Stakeholders in Rwanda1- The Basics • Drug Supply: • First ensure availability of community health drugs and commodities without gap • Then, integrate community drugs’ supply within facility-level supply management • Continue without fail efforts to improve technical supervision / on-the-job continuous training for CHW/CCM Performance • Notably support MOH/CHD current initiatives (practicum, observation)

38. 2- Supervision • Continue Developing Supervision for Community Health as Distinct from Quality Control • Resolve the problem of disbursement of available funds for supervision at HC level • Identify and promote innovative solutions to supervision challenges; give value and visibility to successful examples • Encourage Supervision at Community Level through improving community-PBF guidelines

39. 3- Suggestions on the CHW peer-group experience (Care Group) • Develop low-cost mechanisms to encourage Cooperatives / Districts to develop Care Group-type approaches to strengthening Health Promotion in new districts • Consider Cooperatives’ and Health Centers’ experience sharing across districts about their organization of health promotion, for example through a National Nutrition or c-IMCI Forum, or through structured exchange visits

40. 4- Suggestions on Supporting Performance and Quality of Community Health • Promote institutionalization of Feedback Meetings as distinct from data gathering efforts beyond the 6 districts [Emphasize Decision Making and Information Use] • Health Information: allow continued experimentation with the C-IMCI Bulletin but revise data flow to reduce duplication and favor institutionalization.Bear in mind that the objective is NOT to send more information upstream (already in SIS and SISCom) but to provide real-time local information for rapid decision making (decentralization).

41. 5- CHWs, their support and retention as health agents • Ensure that CHWs maintain a clear focus on essential tasks in CCM and MCH [avoid over-burdening] • Develop District Capacity to monitor CHWs’ level of effort to (a) maintain and preserve achievements on first tasks when (b) additional roles are vestedupon them [i.e. hygiene, additional nutrition work, neonatal health, first aid…]. • Establish a reasonable pace (~24 months?) for reviews of protocols and standards vs. needs and requirements on CHWs.

42. 5- CHWs -- continued • Consult Medical Directors and Supervisors to clarify role of Cell Coordinator Binomes in supervision of CCM. • As needed, review/ adjust roles of cell coordinators between binomes and social affairs • In-kind support to CHWs (light, boots, raincoat), especially binomes, should be more widely made available either through: Cooperative initiatives, MOH / Community Health in Charge, preferably not, new external partners. • Build Management and Financial Management Capacity in Cooperatives

43. 6- Community Health and Cross-sector synergies: The donors and the GOR may want to consider the very active CHW cooperative groups and Peer-Care Groups for livelihood and food security grant funding • Donors and partners of the GOR should direct funding from other sectors, such as Food Security or Economic Development, through the CHW Cooperative or Peer-Care Group structures to further promote their strategic objectives and their non-health sector contributions to the fight against malnutrition. • Donors should consider using grants under contract agreement to build civil society capacity (CHW cooperative for example) to continue the low cost but high-impact CHW activities.

44. 7- Further Studies - continued A reanalysis of RDHS data (2005, 2007-2008, 2010) should be carried out to compare changes in districts of EIP implementation with other districts to evaluate the impact of EIP. • This would provide the MOH with a new baseline for further initiatives in the 6 districts and outside of EIP’s area, as well as cost-effectiveness benchmarks to negotiate with partners.

45. Thank You .