Country-led evaluations of the Delivering as One Programme Country Pilots

1. Country-led evaluations of the Delivering as One Programme Country Pilots Framework Terms of Reference Kigali– October 2009

2. Why an evaluation? • An evaluation is an objective assessment, as systematic and impartial as possible. • It should provide credible, reliable and useful findings, recommendations and lessons for decision-making and learning. • It is important to evaluate pilot initiatives to understand what is working, what is not and why. • To take well informed decisions on continuation and/or scaling up.

3. Principles Evaluations should be: • Independent • Credible • Useful • Owned by national stakeholders

4. Country-led Evaluations of DAO • UN General Assembly Resolution 62/208 Decision 139: ‘encourages the Secretary-General to support programme country pilot countries to evaluate and exchange their experiences, with the support of the United Nations Evaluation Group; and emphasizes, in addition, the need for an independent evaluation of lessons learned from such efforts, for consideration by Member States, without prejudice to a future intergovernmental decision. • GA System-wide Coherence Resolution 63/103, encourages the SG to support programme country pilot countries to undertake expeditiously their own country led evaluations with the participation of relevant stakeholders and with the technical support of the UN Evaluation Group.

5. Approach to Country-led Evaluations • One for each pilot country • Each one to be considered in its national context • Each to build on its evaluability study • National ownership of evaluation • Management of the evaluation process should be under the guidance of national governments • Evaluation results should orient decision-making at national level

6. Approach to Country-led Evaluations • Avoid conflict of interest • Management group should not have been involved in the implementation of DAO • Conducted by independent consultants • Evaluators must not have been involved in policy-setting, design or overall management of the DAO initiative • External quality assurance panel to enhance credibility • Expert panel with international reputation

7. Purpose • Assess the contribution of the UN System to the achievement of national development results. • Assess the progress made against the strategic intent of Delivering as One (DAO) • Record achievements • Identify areas for improvements and remaining challenges, and • Distil lessons to inform decision-making processes at the national level.

8. Scope • Focus on national development priorities, strategies and plans and the way the UN responded to address those priorities • Focus on the implementation of DAO process as expressed by One Programme, Leader, Budget, Office, Voice • Cover operational activities of all UN agencies under One Programme and those that affected the implementation of DAO since 2006

9. Evaluation Matrix

10. Evaluation Criteria • Relevance: Responsiveness to the needs and priorities of the countries • Effectiveness: Achievement of development results and implementation of better processes to achieve those results • Efficiency: Reduction of transaction costs for countries and the UN in comparison to previous arrangements • Sustainability: The probability of benefits to continue over time

11. Evaluation Questions (examples) • Relevance: To what extent did the new programme respond to national priorities and needs? • To what extent does the One Programme mainstream and reflect recommendations from UN conventions, resolutions and treaty bodies, as well as national priorities on gender equality, women’s empowerment and human rights? • Effectiveness: What has been the progress towards the achievement of DAO outcome areas? • To what extent has the UN RC, as One Leader, been able to exercise enhanced authority, responsibility and accountability? • To what extent has the one budget resulted in a more effective allocation and use of funds to the one programme components?

12. Evaluation Questions (cont.) • Efficiency:To what extent has the one programme generated positive synergies beyond individual interventions to increase efficiency? • Has the DAO process led to reduced transaction costs for the national government and the UN? • Sustainability: To what extent has the one programme been integrated into government systems and ensure sustainability of results? • To what extent has the gender and human rights components of the One Programme been integrated into government systems to ensure national ownership and strengthen capacities for results on these areas?

13. Methodology • The methods and tools will match the evaluation questions • Mixed-methods approach (quantitative and qualitative) • Limitations: absence of benchmarks. Other limitations to be determined country by country

14. Evaluation Process Phase IV: Using the Evaluation Phase I: Initiation Phase II: Preparation Phase III: Conduct • Check evaluability • Establish Management Group • Establish Quality Assurance • Finalize the TOR • Recruit Evaluation Team Inception Report Collect & analyze data Draft report Quality Assurance Stakeholder consultation Final reporting Management response Using results Public disclosure Stakeholder engagement is critical in all phases of evaluation

15. Institutional Arrangements • Evaluation management group • Independent evaluation team • Quality assurance panel • Stakeholders • Reference Group

16. Institutional ArrangementsDiagram Stakeholders Stakeholders EVALUATION MANAGEMENT GROUP NAT. GOV - UN NATIONAL GOVERNMENT UNITED NATIONS COUNTRY TEAM EVALUATION TEAM REFERENCE GROUP QUALITY ASSURANCE PANEL

17. Evaluation Management Group • Commissions the evaluation • Drafts the specific TOR, manages the evaluation process, identifies evaluation team and QA panel • Should not have conflict of interest or being directly involved in the implementation of DAO • Members are representatives of the national government, ideally includes the state institution responsible for evaluation, and by the UN

18. Evaluation Team • Conducts the evaluation according to the specific TORs • Members are independent and experienced evaluation experts • Should be gender balanced and draw on regional and national evaluation capacity

19. Quality Assurance Panel • Enhances the quality of evaluations by reviewing deliverables of the evaluations including TORs, inception reports, draft and final evaluation reports • Members should be evaluation experts representing regional evaluation associations, evaluation networks and UNEG

20. Stakeholders • The stakeholders include the parties whose contribution is being evaluated • National implementation agencies such as line ministries sub-national and local government institutions • The United Nations Country Team and DOCO • Should support the work of the evaluation team providing all the information and logistic support as required

21. Reference Group • The reference group will be a consultative body that serves as a sounding board for the EMG decisions on the evaluation • The reference group consists of stakeholders and interested parties such as civil society organizations, private sector, media, academia and international partners working in the country

22. Use of Framework TOR • The Framework Terms of Reference provides: • Principles • Criteria • Sample evaluation Questions • Credible institutional arrangements • The country led evaluation will allow adjustments to the pilots based on evidence • Using the Framework TOR for the country-led evaluations will allow countries to learn from each other