IAOD Evaluation Seminar

1. IAOD Evaluation Seminar “Demystifying Evaluation in WIPO- Best Practices from Initial Evaluations” Validation of Program Performance Report (PPR) Tuncay Efendioglu, Head, Internal Audit Section Geneva November, 8 2012 2012

2. Scope and Objectives Third validation exercise since 2008; All 29 WIPO programs included; Main objectives: Independent verification of information contained in the 2010/2011 PPR; Follow up of open recommendations of previous PPR Validation Reports; The level of ownership of the results-based framework (RF) by WIPO programs.

3. Methodology • One Randomly selected KPI per program; • Interviews with Program Managers/Alternates • Review of documentary evidence; • Discussion of findings/recommendations with all program managers during the fieldwork and reporting stage; • Draft Report Sent for comments by Program Managers; • Final report including management comments sent to Director General and submitted to General Assembly Member States.

4. Challenges • Time schedule set for PPR Validation - Undertaken simultaneously with the PPR- Validating a moving target • Availability of 29 Program managers- interview schedules • Interviewing managers for PIs they were not responsible for so Lack of sense of ownership coupled with lack of background knowledge about Performance measures.

5. Findings and Conclusions • Strengths found where • Timeliness of reporting on the individual PPRs; • Efficiently collected and easily accessible PD. • Limitations observed where • Partial relevance of PD; • Lack of sufficiency and comprehensiveness of PD; and • The RF was primarily used for reporting on performance rather than for management and learning.

6. Findings and Conclusions • Improved expected results (ERs), PIs and sensible baselines and targets in 2010/11 with regard to 2008/09; • Reporting on PIs is still perceived by some WIPO managers as a mandatory administrative exercise without clear linkages to the strategic and operational objectives of the Organization; • Although, ownership levels for Performance Indicators improved, information used for reporting not produced on a regular basis to track progress;

7. Findings and Conclusions • The use of the RF is confined to reporting on performance limiting its potential to enhance management and learning; • RF and monitoring tools need to be strengthened to add the expected value; • Customized training and coaching of responsible staff for designing, monitoring and reporting on the performance framework.

8. Recommendations • Quality assurance of PD as well as their use for the purpose of program management needs to be further strengthened; • Strike the right balance between the RF as a reporting and a management tool by better defining PIs, in future Program & Budget (P&B) documents; • Further increase Results-Based Management (RBM) and monitoring support to staff through more facilitated participative workshops; • Deadlines for submission of individual and consolidated PPR should be set well in advance enabling for timely validation of a final PPR.

9. Some Food for Thought • Is internal oversight offices validating the performance report a good practice to advocate for or not? • Is Member States’ reliance on Validation of PPR similar to reliance on External Auditors' report on financial Statements? • WIPO like others is confronted with KPIs measuring outputs rather than outcome and impact. What is the advice to give to improve the use of RBM? • Who should decide on the indicators of success, program managers or Member States?

10. THANK YOU