210 likes | 455 Views
Evaluation Quality Assessment And CPE Methodology. Division for Oversight Services Evaluation Branch. Phnom Penh - November , 2011. The Challenge. How to Reconcile: The decentralized nature of the Evaluation Function at UNFPA
E N D
Evaluation Quality Assessment And CPE Methodology Division for Oversight Services Evaluation Branch Phnom Penh -November, 2011
The Challenge How to Reconcile: • The decentralized nature of the Evaluation Function at UNFPA • A corporate system that ensures good quality and credibility of its evaluations
Why an EQA? • To provide an independent (EB at DOS) assessment of the quality and usefulness of reports for senior managers (HQ and COs) and also for the information of the Executive Board (reporting) • To strengthen evaluation capacity by providing practical feedback and recommendations to evaluation managers • To contribute to organizational learning and Evidence-Based Management
What is EQA? • a detailed and comprehensive assessment of Evaluation Reports produced in/by UNFPA • a powerful information resource about UNFPA evaluations across a variety of settings (database) • a means of connecting the Evaluation Function within UNFPA’s decentralized system
EQA grid: 9 Core Elements • Structure and Clarity of Reporting • Executive Summary • Design and Methodology • Reliability of Data • Findings and Analysis • Conclusions • Recommendations • Meeting Needs
What Makes a Good Evaluation Report? • A credible report • That addresses the evaluation purpose and objectives • Based on evidence • And therefore can be used with confidence
What are our findings? Example 2: The scope of the evaluation is not sufficiently focused especially in view of the fact that: (i) this evaluation follows on the MTR which took place a year earlier; (ii) the poor quality of the CPAP P&T tool where baseline (and targets) were highly limited and none of these indicators have been tracked sufficiently to measure progress throughout the CP implementation. Finally, ToR should not request that CP’s impact is assessed (this evaluation is performed during the 4th year of a 5-year CP). The methodological design (and its limitations) is well explained and evaluators’ findings appear to be based on credible data and, in turn, support a rigorous analysis presented in a clear manner. Although recommendations stem logically from findings they are not presented in a priority order, nor are the main ones (solely) presented in an Executive Summary which is way too long and detailed and, as a result, does serve its purpose of providing an overview of the evaluation, scope, objectives, methodology and results.
Next Steps (1) • Quality Control: annex EQA grid (and explanatory note) to all evaluations ToRs • Memo sent to Regional M&E Advisers (PD) and to Reps as well as ROs (DOS) • Evaluation Reports & EQAs are made available on the Database (internet)
Evaluation Database http://web2.unfpa.org/public/about/oversight/evaluations/
Evaluation Database Reports are searchable by: • Region • Country • Language • Year of Evaluation • Focus Area • Keywords
Next Steps (2): DOS CPE METHODOLOGY • Develop a pilot methodology for Country Programme evaluations that could then be replicated by/in other Country Offices using the Cameroon AND Bolivia pilot cases as illustrations of this methodology • Contribute to raising the quality of CPEs and consequently the use of evaluation evidence based results and lessons learned in the preparation of subsequent programming cycles.
Phases of the Evaluation Process We are here
The evaluation criteria in a country programme evaluation COUNTRY PROGRAMME EVALUATION Component 2 Analysis of the strategic positioning Component 1 Analysis of the focus areas Strategic Alignment Relevance Efficiency Responsiveness Evaluation criteria Evaluation criteria Effectiveness Added value Sustainability
BOLIVIA LACRO Lower middle-income Country CAMEROON Africa Lower-income Country Piloting the Methodology Different Contexts and levels of development Different Challenges : Maternal Mortality : Decreased in Bolivia: 416/1989 – 229/2003 Increased in Cameroon: 430/1998 – 669/2004 Different Approaches Both Presenting a New CP to the EB – June 2011
Country Programme Evaluation : Objectives To provide an independent evaluation of the progress or lack thereof, towards the expected outcomes envisaged in the UNFPA programming documents. Where appropriate, the evaluation will also highlight unexpected results (positive or negative) and missed opportunities; To provide an analysis of how UNFPA has positioned itself to add value in response to national needs and changes in the national development context; To present key findings, draw key lessons, and provide a set of clear and forward-looking options leading to strategic and actionable recommendations for the next Programming cycle.
Outputs • Bolivia : Field work: June 11 - 30 • Cameroon: Field work: July 11 – 29 • Final Reports: October 2011 • CPE Methodology : December 2011 • Training in RO Jo’burg: March 2012