M&E in the GEF

1. M&E in the GEF Carlo Carugi Sr. Evaluation Officer Expanded Constituency Workshop Belize City, March 2011

2. Overview • Monitoring and evaluation (M&E) and results-based management (RBM) in GEF-5 • M&E policy for GEF-5 • M&E Minimum Requirements • Involvement of focal points • Questions on NCSA evaluation • Evaluation planning for GEF-5

3. M&E and RBM in GEF 5

4. M&E and RBM • Monitoring is one of the main instruments of Results Based Management • Evaluation is a “reality check” on monitoring and RBM • Monitoring & RBM tell whether the organization is “on track” • Evaluation could tell whether the organization is “on the right track”

5. M&E in the GEF Two overarching objectives: • Promote accountability for the achievement of GEF objectives through the assessment of results, effectiveness, processes, and performance of the partners involved in GEF activities. • Promote learning, feedback, and knowledge sharing on results and lessons learned among the GEF and its partners as a basis for decision making on policies, strategies, program management, programs, and projects; and to improve knowledge and performance.

6. RBM in GEF-5 • Portfolio level: tracking progress toward achieving outcomes • Standardized terminology: achieving coherence across Focal Areas • Learning: integrating lessons in management decisions • Feedback: Coherent framework for improved decision-making

7. Project and Program Design Implementation Evaluation LFA/Results framework M&E Plan Monitoring of progress; midpoint course correction as needed Terminal Evaluations Lessons Learned Management, monitoring, and learning Lessons learned; Good practices Tracking Results Adapted from the World Bank’s Results Focus in Country Assistance Strategies, July 2005, p. 13

8. GEF Strategic Goals Institutional Level (top-down) GEB Impacts Outcomes Outputs Focal Area Goal Focal Area Objectives Operating Level (bottom-up) Project Objectives RBM Framework for the GEF

9. Current Reporting Requirements: After Project Approval • Project Level: • Report on Project Start, Project Delays, Project Cancellations • Submission of Project Implementation Report (PIR) • Submission of Tracking Tools • Agency: • Portfolio Overview Report • Focal Area Reports Information should be made available to Focal Points Input to Annual Monitoring Report

10. Annual Monitoring Report • The Annual Monitoring Report (AMR) is the principal reporting instrument of the GEF Secretariat’s monitoring system • Provides a snap shot of the overall health of the GEF’s active portfolio of projects each fiscal year • Report is based on Agency PIR submissions

11. New policy approved by GEF Council in November 2010 M&E Policy in GEF 5:

12. What’s New? Highlights • Reference to GEF Results-based Management (RBM) • Clarification of roles and responsibilities • Stronger role for GEF Operational Focal Points in M&E • Strengthened knowledge sharing and learning • Inclusion of programs and jointly implemented projects • Baseline data for M&E to be established by CEO endorsement • New Minimum Requirement on engagement of GEF Operational Focal Points in project and program M&E activities

13. Separate reporting lines for Monitoring (through Secretariat) and Evaluation (through Evaluation Office)

14. Follow up to Evaluations • A management response is required for all evaluations and performance reports presented to the GEF Council by the GEF EO • GEF Council takes into account both the evaluation and the management response when taking a decision • GEF EO reports on implementation of decisions annually (Management Action Record) • In the case of Country Portfolio Evaluations countries have the opportunity to provide their perspective to Council as well

15. Knowledge Sharing Knowledge management is a process for improving performance by learning • M&E contributes to knowledge building and organizational improvement • Findings and lessons should be accessible to target audiences in a user-friendly way • Evaluation reports should be subject to a dynamic dissemination strategy • Knowledge sharing enables partners to capitalize on lessons learned from experiences • Purpose of KM in the GEF: • Promotion of a culture of learning • Application of lessons learned • Feedback to new activities

16. M&E Levels and Responsible Agencies

18. Evaluation Criteria Did I pay or do too much? Did I get value for money (or efforts)? Results Is it any good or useful? Will it last? Sustainability

19. Monitoring Criteria: SMART indicators

20. M&E Minimum Requirements

21. M&E Minimum Requirement 1 Design of M&E Plans • Concrete and fully budgeted M&E plan by CEO endorsement for FSP and CEO approval for MSP • SMART indicators • Projects should align with GEF focal area results frameworks • Baseline data for M&E by CEO endorsement • Mid Term Reviews (where required or foreseen) and Terminal Evaluations included in plan • Organizational set up and budget for M&E

22. M&E: Minimum Requirement 2 Implementation of M&E Plans Project/program monitoring and supervision will include execution of the M&E plan: • SMART indicators for process/implementation • SMART indicators for results • Baseline for the project fully established and data compiled to review progress • Organizational set up for M&E is operational and its budget is spent as planned

23. M&E: Minimum Requirement 3 Project/Program Evaluations: • All full sized projects and programs will be evaluated at the end of implementation. • Evaluations should: • Be independent of project management or reviewed by GEF Agency evaluation unit • Apply evaluation norms and standards of the GEF Agency • Assess, as a minimum, outputs and outcomes, likelihood of sustainability, compliance with M&E minimum requirements 1 & 2 • Contain: data on the evaluation itself (including TORs); basic project data, lessons • Should be sent to GEF EO within 12 months of completion of project/program Guidelines for evaluating MSPs/EAs will be developed

24. New Minimum Requirement 4 Engagement of Operational Focal Points • M&E plans should include how OFPs will be engaged • OFPs to be informed on M&E activities, including Mid Term Reviews and Terminal Evaluations, receiving drafts for comments and final reports • OFPs invited to contribute to the management response (where applicable) • GEF Agencies keep track of the application of this requirement in their GEF financed projects and programs

25. Involvement of operational focal points

26. Role of GEF Focal Points in M&E • Keep track of GEF support at the national level. • Keep stakeholders informed and consulted in plans, implementation and results of GEF activities in the country. • Disseminate M&E information, promoting use of evaluation recommendations and lessons learned. • Assist the Evaluation Office, as the first point of entry into a country: • identify major relevant stakeholders, • coordinate meetings, • assist with agendas, • coordinate country responses to these evaluations.

27. Support to GEF focal points • GEF-5 Cross-cutting capacity development strategy: • Fifth component: enhancing capacities to monitor and evaluate environmental impacts and trends • This should be identified as a priority in the NCSA capacity development action plan • The capacity development plan should be formulated as a medium size project, or it should be integrated into a broader proposal that would be formulated as MSP or FSP – if MSP it should have 1:1 cofunding • Development of regional partnerships could be considered • Funding from $44m set-aside for capacity development

28. Questions on NCSA evaluation

29. Evaluation of National Capacity Self-Assessments (NCSAs) • Support to NCSAs was one of the approaches to implement the GEF capacity development strategy and UN conventions guidance to GEF • NCSA aimed to identify country level priorities and needs for capacity development to address global environmental issues, holistic and long-term approach, country driven and led • As of August 2010: • 153 NCSAs approved ($28.7 million), 119 completed (UNDP: 76%; UNEP: 23%; WB: 1%) • 23 approved second phases to implement NCSAs recommendations (more in GEF5) • Global Support Programme for NCSA (completed) • Evaluation under preparation, report expected for the November 2011 GEF Council

30. Key questions for consultation/discussion Issues that will be tackled in the NCSA evaluation and that can be raised with us at this meeting: • To what extent have NCSAs been relevant to your country’s needs and priorities? Have they been relevant to support the implementation of conventions? • What was the process of NCSA preparation? Who participated? • What are the main achievements and results of the NCSAs? • Was capacity development improved during the implementation or NCSAs? Any specific examples? • What is the sustainability of the capacity developed? Any specific examples? • Other issues to be included?

31. Evaluation planning for GEF-5

32. Vision for GEF-5 • Consolidation and strengthening of the four streams of evaluative evidence: • Country Portfolio Evaluations: up to 15 during GEF-5 • Impact Evaluations: main effort on International Waters and additional impact work on other focal areas • Performance Evaluations: APR continued and strengthened as well as independent process reviews • Thematic Evaluations: focal area strategies and adaptation • These streams of evaluative evidence will enable a timely OPS5 for which less additional work should be needed than for OPS4

33. OPS5 will include: • Verification and ratings of outcome and progress toward impact • Coverage of the reform process: GEF project cycle and modalities, direct access, STAR, paragraph 28 • Increased understanding of the catalytic role of the GEF • Trends in ownership and country drivenness • Trends in global environmental problems and relevance of the GEF to the conventions • More in-depth look at the focal area strategies, including sustainable forestry management • Better understanding of the longer term impact of the GEF

34. Country Portfolio Evaluations • The LAC region has been visited in 2006-2007 • In this constituency: Costa Rica (report # 32) • 1st pilot experience of country portfolio evaluation • The new CPE cycle has started in the LAC region: • Ongoing CPE in Nicaragua and in OECS member countries • Ongoing Country Portfolio Studies parallel to country level evaluation work by UNDP Evaluation Office in El Salvador and in Jamaica • Two more countries are foreseen in the LAC region (still to be determined)

35. Discussion • Any comments on the issues that will be tackled in the NCSA evaluation? Other issues to be included? • Q&A on the new GEF M&E Policy • Ongoing and future CPE evaluations in the LAC region Thank you!