Public Policy Process: Monitoring and Evaluation plans, frameworks, methods & process

1. Public Policy Process:Monitoring and Evaluation plans, frameworks, methods & process Ghanashyam Bhattarai bhattaraig.62@gmail.com

2. Monitoring plan, framework, method & process • continuous assessment aimed at providing early information of the ongoing activities • regular collection and analysis of information to track the progress of program implementation against pre-set targets and objectives and • action taken to correct the deficiencies

3. Monitoring plan, framework, method & process Characteristics • Clarifies program objectives, • Links activities and resources to objectives, • Translates objectives into performance indicators and • sets targets, • Routinely collects data on these indicators, • compares actual results with targets

4. Monitoring plan • A monitoring plan outlines: • underlying assumptions • activities, outputs, and outcomes relationships • baseline data • monitoring schedule • data sources • associated cost estimates • information on dissemination and utilization plan

5. Monitoring plan • Key consideration: • Resources • Capacity • Feasibility • Reliability • Timeline • Ethics

6. Monitoring Framework

7. Logical Framework

8. Evaluation plan, framework, method & process • Evaluation: • process of determining worth or significance of a development activity, policy or program (determining relevance of objectives, efficacy of design and implementation, efficiency or resource use, and sustainability of results)

9. Evaluation plan, framework, method & process • Evaluation: • making of a judgment about the amount, number, or value of something; assessment. • systematic and objective assessment of an on-going or completed project, program, or policy, and its design, implementation and results.

10. Evaluation Method and Process Requirements; • robust, reliable and repeatable • Simple to operate in terms of technology and user skills • cheap to operate and of a high out put in a short time • Data able to be related to sensitivity responses in the field

11. Evaluation plan • Evaluation is carried out for assessing • Relevance • Efficiency • Effectiveness • Impact • Sustainability • of a plan, program and/ or policy implementation

12. Evaluation plan • Answers are sought on who will • involved in design, implementation and reporting • clarify scope • develop the questions • select indicators • determine data collection methods • analyze and synthesize the information you obtain • interpret the findings, provide feedback, and make recommendations • communicate

13. Evaluation plan • A written document detailing the program model being evaluated • Description and justification of the approach • Instructions for the evaluation/ a guide for each step of the evaluation process It Creates a shared understanding helps decide on how to address the objectives helps identify methods for getting information helps identify issues and strategies to overcome challenges helps determine reasonable and realistic timeline

14. Evaluation Framework • Purposes of framework are to • summarize the essential elements of program evaluation, • provide a framework for conducting effective program evaluations, • clarify steps in program evaluation, • review standards for effective program evaluation, and • address misconceptions regarding the purposes • methods of program evaluation.

15. Evaluation Framework • Conceptual Frameworks • Results Frameworks • Logical Frameworks • Logic models

16. Conceptual Frameworks • maps pathways to achieve results, and • constitutes logical framework for evaluation

17. Results Frameworks • A results framework is both • planning tool and • management tool • Provides program-level framework for mangers

18. Results Frameworks • A results framework is both a planning and management tool. It provides a program-level framework for mangers WHAT WE WANT DO? WHY HOW?

19. Evaluation Framework • Logical Frameworks • Core of the Logical Framework is "temporal logic model" that runs through the matrix as • Vertical (Intervention) Logic: specifies project goals, clarifies relationship and specifies externalities • Horizontal Logic : relates to measurement of effect, resources used, specific indicators and sources of verification

20. Evaluation Framework Logical Frameworks

21. Evaluation Framework • Logic models • Describes logical linkage among program resources, activities, outputs audience and short, intermediate and long term outcomes to the related the program. Logic model is a narrative or graphical description of processes in the real life. It illustrate sequence of cause and effect relationship • Tracking progress against well defined outputindicators provides a clear basis for monitoring progress; verifying purpose level and goal level progress.

22. Evaluation Method and Process • Phases of Evaluation: • Planning Phase • Implementation Phase • Completion Phase • Post completion Phase • EvaluationMethods • Quantitative Evaluation • Qualitative Evaluation • Mixed Methods

23. Evaluation Method and Process EvaluationMethods Quantitative and qualitative methods provide important information for evaluation These methods are rarely used alone; combined, they generally provide the best overview of a project

24. Quantitative Evaluation • Provide information that can be counted to answer such questions as • How many? • Who was involved? • What were the outcomes? and • How much did it cost?” • Data collected by surveys or questionnaires, pretests and posttests, observation, or review of existing documents and databases

25. Qualitative Methods • Qualitative data answer such questions as • What is the value added? • Who was responsible? and • When did something happen? • Qualitative data are collected through • direct or participant observation, • interviews, • focus group discussions, • case studies and • written documents.

26. Qualitative Methods • Analysis of Qualitative data • examining, • comparing and contrasting, and • interpreting the patterns

27. Qualitative Methods • Analysis include • identification of themes, • coding, • clustering similar data, and • reducing data to meaningful and important points or • other approaches

28. EvaluationMethods Mixed Methods Based on the nature of participation by stakeholders may need both qualitative and quantitative methods because of the diversity of issues addressed (e.g., population, type of project, and goals). The choice of methods should fit the need for the evaluation, its timeline, and available resources

29. An Exercise on Logframe: A half of round 1000 students in Grade 5 & 6 of a community were not found to satisfactory in their reading proficiency and the school drop rate was very high. By the time the students reach Grade 10 the number of students is almost half. Department of Education is planning to improve the percentage of school going students at Grade 10 by at least 10% and also improve proficiencies of the students and intends to introduce an intervention at Grade 5 and 6 and. As an effort to improve it, Government has asked DoEd to prepare plan and implement it. DoEd has prepare a programme to organize five summer camps with 100 students of Grade 5 & 6 in each camp. The community members have expressed their concerns on the performance of their wards but have not been able to give time for them. The government has assured of the required budget in upcoming budget for the programme and has instructed DoEd to submit a plan for implementation

30. Logical Framework Example

32. Thank You