RBM Monitoring and Evaluation Reference Group MERG John Miller WHO Roll Back Malaria Department Monitoring and Evalu

1. RBM Monitoring and Evaluation Reference Group (MERG)  John MillerWHORoll Back Malaria DepartmentMonitoring and Evaluation

2. Overview RBM Monitoring and Evaluation Reference Group (MERG) MERG Task Forces, Progress & Products http://rbm.who.int/merg Malaria monitoring and reporting Global Malaria Report 2004

3. Monitoring and Evaluation Reference Group (MERG) Acts as advisory body to RBM Technical focus on global indicators to ensure consistency Maintains communication with ROs on process monitoring and country-specific M&E issues but does not address these issues as part of its primary mandate Geographic focus on Africa Functions: Technical guidance on selection and definition of indicators for national, inter-country and global reporting Advising on prioritization of tasks and recommendations on appropriate data collection methods, analysis and dissemination of recommendations Identifying critical technical questions on M&E and organizing smaller task forces to address these Monitoring changing M&E needs as country programmes and the RBM initiative itself, mature Supporting coordination of M&E activities among other RBM working groups and partners Identifying and recommending strategies for addressing capacity building needs at all levels

4. Monitoring and Evaluation Reference Group (MERG)  First meeting in Arlington, VA, USA, May 2003 Chairs: WHO and UNICEF; Secretariat: MACRO includes representatives from RBM partners, Regional Offices (UNICEF and WHO) and experts assists with technical consensus on selection of indicators, standardization, and clear guidelines for data collection/ sampling with Task Forces: mortality (chair: UNICEF, first meeting July 2003) morbidity (chair: WHO, first meeting Oct 2004) anemia (chair: WHO, first meeting Oct 2003) survey tools (chair: Macro DHS, first meeting Feb 2004) M&E capacity building in countries (chair: Malaria Consortium) Second meeting held in Kampala, Nov 2003 Third meeting held in Geneva, May 2004 Fourth meeting currently underway in New York City, 15-16 November 2004

5. MERG Mortality Task Force First meeting, UNICEF HQ, New York City July 16, 2003 Focus on monitoring of mortality impact of malaria control among African children Recommendations: Primary impact indicator should be all-cause child mortality as measured by household surveys e.g. typical DHS of 7,000 women would enable statistically significant detection of child all-cause mortality reduction of 15% or more VA, HIS data on malaria mortality and VR tends to underestimate impact of malaria control Greater emphasis must be placed on coverage indicators before embarking on impact measurement Given current coverage levels and rates of increase in coverage, annual reporting on mortality (other than deaths occurring in health facilities) is not realistic Impact on malaria-specific mortality may be estimated from measured trends in all-cause under-5 mortality rate in combination with measured coverage of the three key interventions Collaboration with CHERG and CDC to review published literature and develop baseline 2000 estimates of malaria deaths among African children CHERG = Child Health Epidemiology Reference GroupCHERG = Child Health Epidemiology Reference Group

6. MERG Anaemia Task Force Meeting 27-28 October 2003, Geneva Data from malaria intervention trials support the use of childhood anaemia as an indicator of malaria burden and RBM impact in areas of stable malaria transmission Childhood anaemia is best monitored through childhood surveys More timely and smaller sample size required than all-cause mortality Should be measured as Hb level, using Haemocue, in children aged 6-59 months Surveys should ideally be conducted during or immediately after the rainy season; for impact measurement an interval of 2 years (range 1-5 years) is optimal Key indicator is Hb<8 gm/dl Outstanding issues: Additional measurements, such as (sentinel) clinical surveillance Anaemia in pregnant women as supplementary indicator For most endemic settings, limitation of analysis to 6-24 month age group How to interpret trends in anaemia in view of confounders such as malnutrition, HIV/AIDS, geohelminths, etc Use of anaemia as indicator in areas of unstable and low malaria endemicity Value of child/adult anaemia ratios instead of absolute anaemia prevalence as indicator of malaria burden

7. MERG Survey Task Force Meeting 10-11 February 2004, Calverton MD, USA Timely because: UNICEF in process of revising the MICS survey for the next round. The next round of MICS will include the full MIS package of questions for children under 5 in relevant countries GFATM funds beginning to flow to countries; WHO and UNICEF being approached for advice; We get many requests for standard methods!! Products: Malaria Indicator Survey (MIS) Package for household level coverage assessments Stand-alone survey with focus on core coverage indicators Also available as scaled-down "add-on" module Package includes: Questionnaires (Household and women�s) Rationale for each question Interviewer's and Training manual Guidelines on Core Malaria Indicator (available as HANDOUT) Guidelines on sample design and size estimations (long and short versions) Data tabulation plan CSPro data management tool Guidelines to programme managers on use of package

8. MERG Morbidity Task Force Meeting held Oct 2004 to review draft methods and country-level estimates Incorporates existing (MARA for Africa) and new (for outside of Africa) population at risk estimates, population denominators (GPW 3, GRUMP) and standard UN population age distributions Applies fixed incidence rates by endemicity, age, and location (geographic and urban/rural) Includes work of Child Health Epidemiology Reference Group (CHERG) and LSHTM on malaria morbidity estimates in African children Estimates adjusted for coverage of interventions and reported HMIS where relevant Large uncertainty with sensitivity analyses and most estimates are very imprecise Task Force recommendations are being implemented and Task Force members are contributing to a next version. Triangulation of estimates with WHO Global Burden of Disease project

9. MERG Capacity Development Task Force Purpose: Prepare a conceptual framework for strengthening monitoring and evaluation capacity at country and subregional levels Status: Awaiting funding to conduct needs assessment Subcontract signed with MACRO in March 2004 Work scheduled for May-August 2004 Review approaches and agree on methodology Review existing M&E documentation AFRO M&E Mission reports, 2003 Reaping reports, 2003/2004 Series of rapid country assessments (three) Draft framework Workshop in Harare July/August 2004 AFRO, MC, MACRO (and other interested partners) Review findings and discuss draft framework Finalise and present to MERG in November 2004

10. Monitoring and Reporting Efforts for M&E Examples Africa Malaria Report 2003 1st Abuja Summit Progress Report 2004 Global Malaria Report 2004 Intensified regional office and country feedback via standard country profiles (examples as HANDOUTS)

11. Questions or comments Please see the MERG site http://rbm.who.int/merg

12. Input � Process � Outputs � Outcomes � Impact Basic Malaria M&E Framework This slide depicts a basic M&E framework used in the GFATM/WHO M&E Toolkit (top portion) and presents a brief overview of information that is currently being collected for the global malaria report and/or for which standard indicators have been developed. Notably, emphasis is being placed on understanding coverage of RBM�s key interventions for ITNs (both HH possession and use among target populations), fever treatment with antimalarials among Africa children, and IPT. Standard tools (MIS package: available as a handout) have been developed for this purpose and we are trying to promote their use!!! This slide depicts a basic M&E framework used in the GFATM/WHO M&E Toolkit (top portion) and presents a brief overview of information that is currently being collected for the global malaria report and/or for which standard indicators have been developed. Notably, emphasis is being placed on understanding coverage of RBM�s key interventions for ITNs (both HH possession and use among target populations), fever treatment with antimalarials among Africa children, and IPT. Standard tools (MIS package: available as a handout) have been developed for this purpose and we are trying to promote their use!!!