1 / 58

Future Trends in Evaluation

Future Trends in Evaluation. Live Webinar 1 st July 2010. DevInfo. This series of webinars are based on the book published by UNICEF in partnership with key international institutions. Authors: 40 global evaluation leaders Partnership:

gusa
Download Presentation

Future Trends in Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Future Trends in Evaluation Live Webinar 1st July 2010 DevInfo

  2. This series of webinars are based on the book published by UNICEF in partnership with key international institutions • Authors: 40 global evaluation leaders • Partnership: UNICEF, WB, UNDP, WFP, UNIFEM, IDEAS, IOCE, DevInfo

  3. Available for free download at www.mymande.org

  4. 2010 Monthly webinars:

  5. 2011 Monthly webinars:

  6. The recording of the previous webinars are available at www.mymande.org 1 2

  7. The recording of the previous webinars are available at www.mymande.org

  8. The recording of the previous webinars are available at www.mymande.org

  9. Agenda • 9h30 – 9h35 Welcome and introduction • 9h35 – 9h50 Michael Quinn Patton, Founder and Director, Utilization- focused evaluation, and former President of the American Evaluation Association • 9h50 – 10h05 Marco Segone, Systemic management, UNICEF Evaluation Office • 10h05 – 10h25 Questions and Answers • Moderator: Abigail Taylor, Knowledge Management Specialist, UNICEF Evaluation Office • 10h25 – 10h30 Wrap-up

  10. Questions and Answers 1 3 2

  11. Keynote speakers • QUINN PATTON, Michael former President of the American Evaluation Association and author of six evaluation books including Utilization-Focused Evaluation; Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use; and Qualitative Research and Evaluation Methods.

  12. Michael Q. Patton, UNICEF, July 1, 2010 Trendsin EVALUATION UNICEF Webinar Michael Quinn Patton

  13. Michael Q. Patton, UNICEF, July 1, 2010 Evolving Understandings I keep changing what I said. Any person who is intellectually alive changes his ideas. If anyone at a university is teaching the same thing they were teaching five years ago, either the field is dead, or they haven’t been thinking. Noam Chomsky “The Professor Provaocateur,” The New York Times Magazine, Nov. 2, 2003: 13.

  14. Michael Q. Patton, UNICEF, July 1, 2010 TRENDS • 1. Globalization of the Profession • IOCE • IDEAS • Challenge: Non-professionals doing evaluation • Diversity of evaluation

  15. Michael Q. Patton, UNICEF, July 1, 2010 Evaluation Standards and Culture • Utility • Feasibility • Propriety • Accuracy NDE: International Perspectives on Evaluation Standards, No. 104, 2004.

  16. Michael Q. Patton, UNICEF, July 1, 2010 • 1. Globalization of the Profession • 2. Evaluation as a Transdiscipline • Evaluative Thinking • IDRC

  17. Michael Q. Patton, UNICEF, July 1, 2010 • 1. Globalization of the Profession • 2. Evaluation as a Transdiscipline • 3. Increased political interest in accountability, performance indicators, and transparency

  18. SPEAKING TRUTH TO POWER

  19. Loading... Loading...

  20. Michael Q. Patton, UNICEF, July 1, 2010 Trends • 4. Growing emphasis on evaluation capacity-building and skill development. • Essential skills – beyond just methods • Opportunities: IPDET • The Evaluators’ Institute • Professional associations

  21. Michael Q. Patton, UNICEF, July 1, 2010 Trends • 5. Continuing debate about what constitutes methodological rigour

  22. GOLD STANDARD DEBATE:METHODOLOGICALAPPROPRIATENESS notMethodologicalorthodoxy or rigidity

  23. Michael Q. Patton, UNICEF, July 1, 2010 Mixed Methods • Matching methods to evaluation questions, purposes, and resources; • Contingency-based evaluation

  24. Michael Q. Patton, UNICEF, July 1, 2010 Trends • 6. Systems thinking and • complexity concepts • as frameworks for evaluation

  25. Michael Q. Patton, UNICEF, July 1, 2010 AEA monograph: “Using Systems Concepts in Evaluation” edited by Bob Williams and Iraq Iman

  26. Michael Q. Patton, UNICEF, July 1, 2010 Seeing Through A Complexity Lens

  27. DevelopmentalEvaluation:Applying Complexity Conceptsto Enhance Innovation and Use Michael Quinn Patton 2010, Guilford Press

  28. Michael Q. Patton, UNICEF, July 1, 2010 Complexity Concepts • Emergence • Openness • Dynamic • High uncertainty • Nonlinear: Small actions, large impacts • Co-evolution • Adaptation • Unanticipated consequences

  29. Michael Q. Patton, UNICEF, July 1, 2010 Unanticipateddevelopmentsas an expected aspect ofinnovation and a focus and challenge for evaluation.

  30. Michael Q. Patton, UNICEF, July 1, 2010 Beyond just formative and summative toDevelopmental Evaluation

  31. Michael Q. Patton, UNICEF, July 1, 2010 Development not just Improvement. Core questions: What is developing? What is being developed?

  32. Michael Q. Patton, UNICEF, July 1, 2010 Challenge Matching the evaluation process and design to the nature of the situation: Contingency-based Evaluation

  33. Michael Q. Patton, UNICEF, July 1, 2010 Contingency-basedEvaluation • Situational analysis & responsiveness • Context sensitivity • Clarify and focus on intended users: stakeholder analysis • Clarify and focus on intended uses • Methodological appropriateness • Criteria for evaluating the evaluation: credibility, meaningfulness, utilty

  34. Michael Q. Patton, UNICEF, July 1, 2010 Summary: Six Trends • 1. Globalization of the profession • 2. Evaluation as a transdiscipline. • 3. Increased political importance of M & E • 4. Growing emphasis on capacity-building and essential skills • 5. Debate about methods: Gold standard • debate • 6. Using systems thinking and • complexity concepts

  35. Michael Quinn Patton UNICEF WebinarJuly 1, 2010 References • Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. • Michael Quinn Patton, 2010, Guilford Press. • http://www.guilford.com/cgi-bin/cartscript.cgi?page=pr/patton.htm&dir=research/res_eval&cart_id=435936.20323 • Utilization-Focused Evaluation, 4th ed., • Michael Quinn Patton, Sage Publications, 2008.

  36. Michael Q. Patton, UNICEF, July 1, 2010 References • “Future Trends in Evaluation.” Pp. 44-57 in From Policies to Results: Developing capacities for country M & E systems, Marco Segone (ed.) UNICEF, 2010.

  37. Keynote speakers • SEGONE, Marco is responsible for the decentralized evaluation portfolio at the UNICEF Evaluation Office, serves as a Senior Advisor to the IOCE Board and was IOCE Vice-President.

  38. Key elements for a conceptual framework National evaluation capacity development Marco Segone, Systemic Strengthening, UNICEF Evaluation Office

  39. WHY

  40. Paris Declaration on Aid Effectiveness In line with new international development paradigm… National ownership and capacity development: the key ingredients to development Managing for results Mutual accounta bility Harmoni zation Alignment Ownership

  41. … the UN General Assembly … • national Governments have the primary responsibility for coordinating external assistance and evaluating its contribution to national priorities • mandated the United Nations system to promote national ownership and capacity development, and to make system-wide progress in harmonizing evaluation practices

  42. … and UN Agencies Boards request. • 2009 UNICEF ExBoard: • emphasizes that programme countries should have greater ownership and leadership in the evaluation of all forms of assistance • underlines the importance of increasing the participation of national counterparts and strengthening national capacity in evaluation • encourages UNICEF to use national evaluation systems, where available, to continue to incorporate capacity-building mechanisms into programme design and implementation and ensure that evaluations are responsive to national demand

  43. HOW

  44. A Systemic approach to Capacity Development. Individual Level (skills, knowledge, experience)

  45. Individual Level • Demand side • Capacity to strategically plan evaluations, and to identify the key evaluation questions • Capacity to manage evaluation for independence and credibility • Capacity to use evaluation • Supply side (capacity to conduct an evaluation): • Behavioural independence • Independence of mind & integrity • Knowledge and respect of evaluation standards • Agreed evaluation processes & products • Professional competences • Formal education (Masters) • Specialized training • Professional Conferences and meeting • On the job training (country-led evaluations) • Community of Practices and networking

  46. A Systemic approach to Capacity Development. Institutional Level (policies, procedures, frameworks) Individual Level (skills, knowledge, experience)

  47. Institutional Level • Evaluation culture • Set of values and attitudes supporting evaluative (critical) thinking within an organization • Institutional commitment to learning from evaluation, support evidence-based policy debate and demand for accountability. • Individual more self-directed learners and use information to act; take higher risks but also develop a greater sense of personal accountability and responsibility; consult, coach, and support each other more. • Protective culture (Remove repercussions on careers) • Understanding of the foundations and principles of Monitoring and Evaluation • Institutionalizing independence, credibility and utility • Evaluation policies • Work programme and budget • Independence & adequacy of budget • Conduct of evaluations • Institutional endorsement of standards • In-built Quality Assurance systems

More Related