1 / 23

M&E in HNP Operations: Lessons Learned in South Asia

M&E in HNP Operations: Lessons Learned in South Asia. Benjamin Loevinsohn and Aakanksha Pande October 26th, 2006. Review of M&E in South Asia HNP Operations. A. Background & Methods B. Results C. Recommendations. Three Aspects of M&E.

brandice
Download Presentation

M&E in HNP Operations: Lessons Learned in South Asia

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. M&E in HNP Operations: Lessons Learned in South Asia Benjamin Loevinsohn and Aakanksha Pande October 26th, 2006

  2. Review of M&E in South Asia HNP Operations A. Background & Methods B. Results C. Recommendations

  3. Three Aspects of M&E • Measuring the results of World Bank financed operations – fiduciary obligation • Strengthening the capacity of governments to monitor the performance of their HNP sectors – the long-term solution • Impact evaluation – rigorously testing whether innovations actually make a difference

  4. Sampling • 12 South Asia HNP operations were selected using stratified random sampling. • Since India represents 60% of HNP operations, 7 of the 12 projects were from India • Indian sample was also stratified to ensure a balance between disease specific projects and state health systems projects

  5. Design of Study 12 projects/ 60 indicators Interview with TTLs PAD Aide Memoire PSR/ISR ICR Sources of Data AP BPL SR Reviewers Kappa Statistic

  6. Review of M&E in South Asia HNP Operations A. Background & Methods B. Results C. Recommendations

  7. Summary of Results • Indicators could be better selected (too many & too focused on inputs) • Data collection plans ok but could be improved • Data collection plans rarely implemented • Even when data is collected, not often analyzed. • Even when analyzed, not often used • Few controlled evaluations, even though many opportunities • Little attention to building M&E capacity

  8. Indicator Selection • Indicators are usually defined in a measurable way – 80% of the time. • Uncertainty about the selection of indicators, reasonable grounds for disagreement • Task teams need to keep a lot of people happy • Many indicators are “upstream” more than would make sense for attribution • Many indicators are chosen for each project avg. = 26, range 5 to 47 • Having more indicators leads to less data

  9. Evidence of “Indicator Inflation”

  10. Data Collection Plans • 33% of indicators had 5 attributes: • defined in a measurable way (80%) • clear method of collection (90%) • clear target (53%) • clear schedule (53%) • clear responsibility (73%) • Individually these attributes were clearly important to people

  11. Implementation of Data Collection Plans

  12. Data Collection • Baseline data rarely collected • Overall data collection plans were not implemented as designed • Inconsistent approach to data collection – sad examples • But, good examples of data collection do exist

  13. Data Analysis • Even when there was data, evidence of analysis existed only half the time • For only 27% of indicators was there any evidence of data being analyzed • Even when there was analysis, there was evidence of action 27% of the time

  14. M&E Capacity • Analysis of M&E capacity appears in PADs a third of the time • Little agreement among observers about presence of M&E capacity building plans (kappa = 0.18) • Lack of clarity on plans, implied not specified • Much of the effort focused on HMIS • Capacity building on M&E seen as important by World Bank staff

  15. Evaluation of Innovations • Half of projects contained at least 1 innovation • Innovations not clearly defined, sometime implicit • Explicit mechanism for evaluating innovation in 8% of projects • Only one project had a controlled design

  16. % of Children under 5 who were moderately or severely malnourished - BINP

  17. % of Children under 5 who were moderately or severely malnourished - BINP

  18. Little Progress of M&E Over Time

  19. Review of M&E in South Asia HNP Operations A. Background & Methods B. Results C. Recommendations

  20. Recommendations • Use of an M&E checklist during preparation and implementation • Select a set of indicators that TTLs can choose from • Provide technical assistance to task teams where needed • Dedicate staff to implement M&E • Build the M&E capacity of task teams through training

  21. Recommendations • Deal with all aspects of M&E: (i) M&E of the project; (ii) capacity building of Government; (iii) impact evaluation • Clear & consistent messages from management about importance of M&E i.e. Results-Based Management

  22. M&E Checklist(page 22 of Report) • Meant to be a guide for task teams • List of actions that should be considered during preparation and implementation • Part of QER package • Has been adapted for water supply, rural development, & education.

  23. Life is short. The art is long. --Hippocrates

More Related