The independent evaluation office of the united nations development programme reflections on contribution to transpare

1. The independent evaluation office of the united nations development programme reflections on contribution to transparency, learning and accountability Indran A. Naidoo Director June 2013

2. Setting the stage Demand from Members states and management. The Evaluation Policy of the UNDP clarifies roles and responsibilities for evaluation The Evaluation Office (EO) of the UNDP, the largest in the UN system, produces independent evaluations at the corporate, programme and country level. It extends its influence by managing the UNEG Secretariat, producing guidance and standards, engaging with networks and supporting evaluation capacity building across the globe

3. 1. Mandate and functions • 1999EO was established 2004 GA 59/250 Resolutions for UN System 2005 UNEG Norms & Standards • 2006 UNDP Evaluation Policy and Independence • 2011UNDP Evaluation Policy (revised) • Reports directly to the Executive Board of the UNDP • Supports the Administrator in her substantive accountability function • Operates within the UNEG Norms and Standards and ethical guidelines, and engages stakeholders in the conduct of evaluations to ensure transparency, learning and accountability

4. In practice this means… Evaluations are planned and answer questions within the Strategic Plan - has been RELEVANCE, EFFECTIVENESS, EFFICIENCY and SUSTAINABILITY DO UNDP INTERVENTIONS MAKE A CHANGE? EO staff lead all evaluations, drawing on advisory panels and using engagement processes with evaluands and stakeholders as a means to enhance credibility. As an independent office, the Director signs off on all evaluations.

5. 2. Contributions to other organisational tier support and professionalasation)

6. For evaluators it requires … Commitment to evaluator professionalisation Ability to engage in a supportive yet independent manner High levels of methodological skill and content expertise Strong strategic and communication skills

7. The Evaluation Office Has produced over 100 evaluations since 2000, and 80 country level evaluations. All have a management response. Management uses evaluations to review policies, programmes and approaches – shown in 92% uptake on recommendations. Independence respected, and evaluation on the agenda at key top management meetings. Board allocates significant time to engage with evaluation findings, management responds fully.

8. Transparency and stakeholders The EO engages fully as evaluation processes are as important as the “big report” Learning can come through accountability processes, these are not opposed to each Stakeholder workshops held at country level, led by EO, with government as key stakeholder.

9. Where UNDP operates Globally, across a broad mandate In complex situations where interventions are difficult to embed and hard to measure Implications for evaluation: difficult to work out additionality in terms of attribution or contribution Context plays an important role, not easy to superimpose systems to measure results, but not impossible. Involves high levels of engagement to agree on what constitutes success

10. UNDP Programmes and Operations • Non-Project operations (advisory, advocacy, standard setting/normative, coordination, mobilization) National Goals and Priorities • UNDP Focus Areas • Poverty and MDG • Democratic governance • Crisis prevention and recovery • Environment and sustainability development • Programs • Global • Regional • Country • Others United Nations Development Framework Projects 177 countries in 5 Regions

11. What does this mean? Evaluations need to be context specific, yet meet evaluation norms and standards. At the country level they inform the next UNDP programme, and draw on lessons from the last programme period Aggregation for synthesis needs to consider the question of scale, variability and the challenges of validity Stakeholder workshops demonstrate to government UNDP commitment to transparency and accountability

12. Independence, a credibility question Credibility goes beyond the tools and methods. It rests on the leadership that directs evaluation processes. This comes about through transparent and logical evaluation plans and processes which ensure engagement opportunities throughout and across the spectrum to reduce bias Results must be engaged publically Independence is central for the reasons above, to ensure credibility and authenticity of reports

13. Transparency • Defined in UNEG norms and standards • Consultations with Stakeholders • TORS > Inception Report on Scope, Design and plan for data collection and analysis > Stakeholder Meetings > Draft Reports • The Audit Trail <> significance for mutual understanding and final decisions by EO • Public Access • All UNDP plans and evaluations in the Evaluation Resource Center (l) • EO evaluations in the EO website • Management response and tracking system (ERC) • Ratings on quality of decentralized evaluations

14. Independence - Structural Executive Board UNDP Administrator Evaluation Office Director Functions, and staff - organizationally independent from operations and policy units and decision making. • Executive Board • Director reports to the Executive Board (2 terms and no re-entry into UNDP) • Board approval of programme of work and budget (independent of programme budget) • Reporting • Evaluation reports are the responsibility of the Director • Transmitted directly to the Board following review and comment by management • Senior managers safeguard the independence: EO has access to all records and information

15. Examples from …2013 As of June 2013 the Evaluation Office of UNDP presented eight independent evaluations to the UNDP executive board, which will contribute towards the development of the new UNDP Strategic Plan. Five Regional Programme Evaluations Evaluation of South-south and Triangular Cooperation Evaluation of UNDP Strategic Plan Evaluation of Global Programme of UNDP

16. Country Programme Evaluation (Assessment of Development Results – ADRs) Evaluation Office has conducted 80 ADRs since 2002

17. Acting on evaluations Engaging with decision-makers – engagement for quality and credibility (management responses and update) Statistics on uptake – 92%

18. Global influence Support of professional networks, associations and events, to profile evaluation: Supportive of IDEAS, Barbados 2013 (key notes, panels courses) Supportive of IPDET – course presentation Supportive of EvalPartners – management board Support to continental and regional networks (AFREA, APEN, … Hosting of the National Evaluation Capacity workshops, next San Paulo, September 2013 www.NEC2013.org

19. Conclusions Evaluation is not an event, but a difficult journey requiring constant push, reappraisal, strategising to ensure relevance. It does not occur naturally and requires drivers An evaluation function must be independent, which can be supported when it is both inward and outward focused. The outward – through events like these brings in critical insights necessary for ongoing revision. An outward orientation is the only way that one get professionalisation that is necessary for evaluation.

20. Thank Youwww.undp.org/evaluation