1 / 63

Monitoring and Evaluation

Monitoring and Evaluation. Session Objective. To increase participants understanding of the concepts used in designing M&E Frameworks and Plans To build participants competence in designing Program M&E Plans. Expected Results.

jshankle
Download Presentation

Monitoring and Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Monitoring and Evaluation

  2. Session Objective • To increase participants understanding of the concepts used in designing M&E Frameworks and Plans • To build participants competence in designing Program M&E Plans

  3. Expected Results • At the end of the sessions participants will know about Program frameworks, M&E frameworks and the difference between the two frameworks • Participants will be able to identify and select appropriate indicators for a program. • Participants will be able to produce a program monitoring and evaluation framework.

  4. Key Terms forMonitoring and Evaluation

  5. Objectives • Explain the difference between monitoring and evaluation. • Introduce the most common M&E terms. • Review examples of each term.

  6. Monitoring versus Evaluation Evaluation Data collected to answer specific questions Periodic Focus on outcome, impact How effective were our activities? Monitoring • Data collected on program activities • Ongoing, routine • Focus on activities and output, compared to target Are we doing the work we planned?

  7. Monitoring or Evaluation? • Local researchers conduct a study to determine if there are more people with possible TB symptoms coming to clinics as a result of a media campaign to promote TB screening. Evaluation

  8. Monitoring or Evaluation? • A district manager reports on how many nurses were trained on interpersonal communication skills for her quarterly donor report. Monitoring

  9. It Starts with QUESTIONS • Monitoring and evaluation answer different questions. • If we do not ask good questions about our activities, we will not get useful data!

  10. What is a GOAL? • The ultimate result of efforts at a broad, population level. • Achieved over the long term (years) and through combined efforts of multiple programs. • Decrease morbidity and mortality due to TB in Country X. • Reduce prevalence of TB by 50%. • Eliminate stigma of TB in our communities.

  11. What is an OBJECTIVE? OBJECTIVES GOALactivities • How the results of your short-term program activities contribute to the big goal. • Several objectives can relate to the same goal.

  12. Objective Examples • Aggressively advocate to increase budget by 8% each year for the next four years. • Double the percentage of secondary school students who can correctly identify TB symptoms by 2015. • Design and pilot a treatment support program for newly released prisoners with TB by 2015.

  13. INPUTS • Resources needed to plan and implement XX • “Raw materials” of an XX project Examples • Money • Staff • Policies, guidelines • Equipment • Partners

  14. ACTIVITIES • The work that we do, what we implement • Also called “processes” Examples • Training events • Meetings • Events • Outreach • Home visits

  15. OUTPUTS • Immediate results of activities • What we can measure/count right after the activity Examples • Number of people trained • Number of brochures produced • Number of policymakers reached with advocacy activity

  16. OUTCOMES • “Ripple effects” of XX activities • What changesafter outputs are produced Examples • Increased funding for TB after lobbying meeting • Short: Improved attitudes toward TB patients among DOTS nurses after a training • Medium: Increased satisfaction of TB clients • Long: TB clients stay in treatment longer

  17. INDICATORS • How we define our activities, outputs, or outcomes • Signs or evidence we watch for to see if we have reached them

  18. IMPACT • More related to goal • Very broad-scale result over long term Examples • Higher rate of treatment success • Reduction in deaths among MDR-TB patients

  19. Introduction… • Write the administrative/engineering/health problem addressed by a major public health program in your country • Write at up to three specific objectives of a engineering/health program that addresses the above mentioned health problem. • Write down two indicators the program mentioned above uses to monitor it progress or performance towards its objectives.

  20. The Project cycle

  21. Project CycleJan Davis and Robert Lambert, Engineering in Emergencies, p.63. Project CycleJan Davis and Robert Lambert, Engineering in Emergencies, p.63.

  22. Monitoringis the routine process of data collection and measurement of progress toward program objectives. Evaluationis the use of social research methods to systematically investigate a achievement of a program’s results What is Program Monitoring, Evaluation?

  23. Differences and Links between Monitoring and Evaluation SLIDE 7

  24. Comparison Between M&E Adapted from UNICEF, A UNICEF Guide for Monitoring and Evaluation: Making a Difference? New York, 1991, p.3

  25. MONITORING MONITORING is the continuous, systematic and critical review of operations in order to measure their evolution and adjust them according to circumstances and project’s objectives.

  26. Building a Monitoring system

  27. BUILDING A MONITORING SYSTEM Intervention objectives BUILDING A MONITORING SYSTEM

  28. Impact Outcomes Outputs Activities Inputs Source: Adapted from ADB (2006) Introduction to Results Management, p. 7 World Bank (2001) PRSP Sourcebook, p. 108. What should a M&E System Measure? The Results Chain Indicative Example: Effects on dimensions of well-being Improve literacy Access to, use of, and satisfaction with services School enrollment rates Goods and services produced Number of schools built; textbooks, etc. Tasks undertaken to transform inputs to outputs Building of schools Distribution of textbooks, etc. Financial, human and material resources Spending on primary education

  29. Some examples of outcome indicators Source: ADB. 2005. Practice Note on Results-based Country Strategies and Programs, Annex 2.

  30. Key Questions • What is the purpose of carrying out M&E • Who needs, uses M&E Information • Who carries out M&E? • How is M&E carried out? • When should M&E be carried out?

  31. What is the purpose….? • Improve program implementation • Data on program progress and implementation • Improve program management and decision making • Inform future programming • Inform stakeholders • Accountability (donors, beneficiaries) • Advocacy

  32. To Improve program implementation… To Inform and improve future programs Inform stakeholders Managers Donors Governments Technocrats Donors Governments Communities Beneficiaries Who needs, uses M&E Information?

  33. Who conducts M&E….? Program implementer Stakeholders Beneficiary Remember .. M&E Technical skills Participatory process

  34. How to carry out M&E…? Key Features • Program Framework: Analyze and systematically lay out program elements • Identify key elements to monitor and evaluate. • Determine and describe the measures to be used for monitoring and evaluation • Develop M&E Framework and action plans, including data collection and analysis, reporting and dissemination of findings.

  35. Program Framework What do you know about your program….?

  36. Program Framework • Systematic lay out of the program elements and path showing what the program plans to: do ……………..achieve!

  37. Program Framework • Based on a theoretical, empirical model, or general understanding Public health Problem Population, system level factors that cause the public health problem Action/interventions that can change the factors and ultimately alleviate/eliminate the problem

  38. Results Framework Improved Health Status Impact: Improved (Sustained) Use of Key Health Services and Practices/Behaviors Strategic Objective: Increased quality of… Increased availability/ access to… Improved social / policy environment… Intermediate Result: Strategies: Strategies: Strategies: Strategies (Sub IR):

  39. Example: Result Framework for a Family Planning Project GOAL: REDUCED FERTILITY SO: Increased FP use and improved FP/RH practices Increased knowledge of, improved attitudes toward, and acceptance of key services and behavior Increased quality of FP counseling and services for Increased availability/access to FP/RH Improved social and policy environment for FP Strategies: • Increase availability of educational materials at clinic and community level • Community mobilization (using PRA and PDI) including men • Implement mass media strategy • Mobilize opinion leaders at national and local level • Design/ implement supportive supervision System • Train service providers (in-service in FP counseling and management of side effects) • Remodel clinic to allow for privacy • Design and implement quality improvement program • Strengthen logistics management • Mobilize private sector providers • Mobilize CHWs/CBDs • Encourage socially marketed pills Advocate for community based distribution of pills Promote addition of Depo injections to EPI outreach strategy Pilot social marketing of pills

  40. Basic Logic Model

  41. Case 1: To decrease maternal mortality, a 10-year program plan to improve to train midwifes to Delivery and ANC services at health facilities, and to train and deploy CHWs to increase the community’s awareness about, and use of the improved services at the health facilities. Case 2: To reduce high fertility, a 5-year program plans to work with the Government to change policies in order to allow and promote use of modern family planning methods, train family planning providers to provide better FP services, and to launch public campaigns that promote family planning methods. Case 3: To reduce HIV infection among adolescents, a five-year program plans to implement income generation activities for the youth, provide and promote universal secondary education, and build adolescent-friendly reproductive health service delivery points.

  42. Exercise • Identify and state is the Public Health problem implied in the case study. • What are population level factors will the program target to change in order to alleviate the public health problem • Prepare a Program Framework for the scase study

  43. Monitoring and Evaluations Framework

  44. M&E Questions • Monitoring questions • What is being done? • By whom? • Target population? • When? • How much? • How often? • Additional outputs? • Resources used? (Staff, funds, materials, etc.)

  45. M&E Questions • Evaluation Questions? • Is the content of the intervention or the activity being delivered as planned? • Does the content of the intervention or the activity reflect the requisite standards? • Have the intervention achieved the expected results?

  46. What do we need to answer these questions…?INDICATORS …to take measurements.

  47. Indicators: Definition • Markers that help to measure change by showing progress towards meeting objectives • Observable, measurable, and agreed upon as valid markers of a less well-defined concept or objective • Indicators differ from objectives in that they address specific criteria that will be used to judge the success of the project or program. See comment for examples

  48. Type and Level of Each Indicator • Type • Input/Process (Monitoring) • Outcome / Impact (Evaluation) • Level • Global level • Country level • Program level

More Related