IAOD Evaluation Section , the Development Agenda (DA) and Development Oriented Activities

1. IAOD Evaluation Section, the Development Agenda (DA) and Development Oriented Activities Julia Flores Marfetan, Senior Evaluator

2. 2 IAOD Evaluation Section Introduction to the Independent Evaluation Function Services provided to the Development Agenda Independent evaluation of the Development Agenda and development oriented activities Types of Evaluations Evaluation Approach for the Development Agenda and development oriented activities

3. Introduction to the Independent Evaluation Function

4. Evaluation Definition 4 IAOD Evaluation Section Evaluation is an objective, impartial and systematic assessment to provide useful feedback about an activity, initiative, project, program, strategy, policy, unit, organization or sector, whether financed from regular budget or extra budgetary resources

5. Evaluation in WIPO • What has IAOD and its Evaluation Section done since 2007? • The WIPO Evaluation Policy since 2007 • The Evaluation Section was set-up by the DG in 2008 within IAOD with a dedicated Evaluation Team • Focusing the evaluation function in their core business “Independent Evaluations” • Independent Review of the PPR process • Validation of the 2008 PPR • Having dedicated evaluation team within the Evaluation Section – 2 post were created • Training to improve existing monitoring and evaluation frameworks • Development of self-evaluation guidelines for program staff • Improving the previous PPR and making it more performance oriented • Where did all start? • Recently no real evaluation activity since 2004 • Evaluation was perceived as a program management function • What are the Evaluation Section future priorities? • Undertake high quality independent evaluations of development oriented activities in form of country, thematic, strategic or program evaluations • Develop WIPO Evaluation Strategy and Guidelines • Contribute to enhance accountability, learning and transparency by delivering relevant information to our stakeholders specially Member States 5 IAOD Evaluation Section

6. EVALUATION SECTION SERVICES 6 IAOD Evaluation Section

7. Services provided to the Development Agenda (DA)

8. EVALUATION SECTION SERVICES CORE SERVICES: INDEPENDENT EVALUATIONS Design, manage and undertake independent evaluations of the Development Agenda and development oriented activities Other Support Services Validation Support for capacity building Ensure that monitoring and performance management of the development oriented activities is of good quality May help Member States in building their capacities to undertake evaluation exercises Including impact evaluations depending on resources provided. This is important for The country focus of the DA and DOA. 8 IAOD Evaluation Section

9. Independent Evaluation of Development Agenda (DA) and development oriented activities (DOA)

10. Whz are independent evaluations necessary? • Independent evaluations contribute towards a UN system identifying ways to improve delivery of development oriented activities to the beneficiaries; overcoming weaknesses and building on strengths from a strong evidence base • Independent evaluations assist Organizations to take a deep and detailed look at internal systems and development oriented activities, and examine what practical steps need to be taken to improve them • According to the UNEG norms and standards the evaluation function and its products should be independent, credible, and useful for learning and accountability purposes • It is central to the credibility and reliability of evaluation • By increasing independence in evaluation, we can decrease certain types of bias including extreme conflict of interest • Evaluation quality without independence does not assure credibility and does not protect learning 10 IAOD Evaluation Section

11. Assuring the Independence ofEvaluations Organizational Independence The ES is part of IAOD and it performs its function independently from other WIPO Management functions to ensure impartial and independent reporting Behavioural Independence The evaluation reports are based on evidence and stakeholders are consulted at the various stages of the evaluation process. The budget for the evaluation is managed solely by IAOD Protection from outside interference IAOD Evaluation Section is be responsible for designing and executing the evaluation and evaluators results will not be subject to overruling or influence by any external authority Avoidance of conflict of interest IAOD will assure that the evaluators undertaking the evaluation will in no manner have an official, professional, personal or financial relation with any WIPO program and that he/she does not have any current or previous involvement with the development oriented activities or the entity being evaluated at a decision making level, or in a financial management or accounting role; or seeking employment with the Organization. 11 IAOD Evaluation Section

12. Why do we need to evaluate the DA and DOA? (1) 12 IAOD Evaluation Section Are we doing the right things? And are we doing them right? To know if we are achieving an effect (+/-) on the intended beneficiaries To contribute to learning To support accountability and transparency to stakeholders specially Member States. To inform the CDIP, other stakeholders and management To inform the planning, programming, budgeting, implementation and reporting cycle To contribute to the improvement of the performance of on-going and future activities and to the enhancement of institutional and development effectiveness.

13. Why do we need to evaluate the DA and DOA? (2) • To enhance the generation and use of value-added evaluative information for: • Decision-making process concerning the improvement of present and future activities • Policy formulation and review by Member States • Management oversight by the Director General 13 IAOD Evaluation Section

14. Requirements for evaluating the DA and DOA? • Establishing a program theory: how the intervention will benefit the beneficiaries and the potentials for side-effects • Setting up a monitoring framework which includes expected results targets, baselines, SMART KPIs including systems and data base for data collection • Allow comparison: before and after the intervention • Plan the evaluation at the start of the intervention • Define the evaluation approach, methodology and allocated a budget and time for it 14 IAOD Evaluation Section

15. What will guide the evaluation of DA and DOA? 15 IAOD Evaluation Section The WIPO Evaluation Policy approved by the DG in 2007 guide the undertaking of the independent evaluations The Evaluation Policy lays a framework for the evaluation function of WIPO The Evaluation Policy provides a comprehensive framework for planning and conducting the evaluation following international best practice, taking due account of the Norms and Standards for Evaluation in the UN System adopted by the United Nations Evaluation Group (UNEG), and other international good practices. The Policy is applied to all independent evaluations

16. What are the evaluation criteria applied to the evaluation of DA and DOA? Efficiency Is the Organization for the implementation of its development oriented activities using the least costly resources possible in order to achieve the desired results? Relevance Are the Organization’s development oriented activities suited to the priorities and policies of the target group? Effectiveness Are the development oriented achieving its objectives? Coverage which groups are included or excluded from the development oriented activities, and the differential impact on those included and excluded? DA and DOA Are we doing the right things? Are we doing them right? Where is the Evidence? Impact What positive and negative changes are being produce by a development oriented activities, directly or indirectly, intended or unintended? Co-ordination How are the development oriented activities harmonizing efforts with WIPO programs, other aid agencies and alignment with country priorities and systems? Sustainability Are the benefits of the Development Agenda likely to continue after the Organization’s funding has been withdraw? Coherence Are other policies and programs affecting the implementation of the Development Agenda? 16 IAOD Evaluation Section

17. What other information will theevaluation deliver? 17 IAOD Evaluation Section Evaluation will examine the processes and contextual factors to understand why, and to what extent, expected results have been achieved, and what impact they have had. Evaluation will recommend and suggest better ways of achieving expected results. Evaluation will identify good practices and lessons learned.

18. Types of Evaluations

19. Independent Evaluations vs Costs Country Level Evaluations Thematic Evaluations Program Evaluations Systemic Evaluations Organizational Assessments Project Level Evaluations only Above Sfr 1 mio 19 IAOD Evaluation Section

20. DA and DOA evaluation timing DA and DOA Stage of the Development Intervention Development Intervention Starts/ about To start Development Intervention Is in the middle Of implementation Development Intervention is completed Time Formative Evaluations Summative Evaluations Ex-ante evaluation Mid-term evaluation Ex-post or final evaluation Type of Evaluation 20 IAOD Evaluation Section

21. Evaluation approach for the Development Agenda and Development Oriented Activities

22. Utilization-focused evaluation approach 5. Follow-up day Assisting program managers with the Implementation of recommendations 1. ToRs drafting Participatory planning with Key stakeholders and partners 2. Evaluation process Supporting the evaluation team with a Resource or learning group 4. Evaluation report Dissemination to a wider public 3. Draft findings/ Draft report Consultation with key stakeholders And workshops 22 IAOD Evaluation Section