Impact Pathways: An Approach for Understanding, Fostering and Evaluating Research-for-Development Outcomes

1. Impact Pathways: An Approach for Understanding, Fostering and Evaluating Research-for-Development Outcomes Boru Douthwaite, Technology Policy Analyst Sophie Alvarez, Consultant Simon Cook, Leader BFPs Rick Davies, M&E Specialist, Pamela George, CPWF Program Manager, John Howell, M&E Specialist, Ronald Mackay, Professor Emeritus, Jorge Rubiano, National University of Colombia CIAT Seminar, 1st November 2006

2. Impact Pathways Matter

3. How change happens • “Improvements in poverty alleviation, food security and the state of natural resources result from dynamic, interactive, non-linear, and generally uncertain processes of innovation.” EIARD, 2003 EIARD represents a group of European donors 15 EU Countries plus Norway and Switzerland

4. Impact Pathways Approach • People plan and implement projects on the basis of their change models - their implicit theories about how the world works • If you can improve these theories you can improve the practice, making impact more likely • Impact Pathways Approach – A participatory approach for: • Making practitioners’ theories explicit about how they will achieve adoption and impact (impact pathways, program theory); • Improving these theories • Using those models / frameworks for M&E and impact assessment • As a result, contributing to project and program “adaptive management” and thus likelihood of impact

5. History and Current Work • Past Work in Nigeria on Striga • Douthwaite, B., T. Kuby, E. van de Fliert and S. Schulz. 2003. Impact Pathway Evaluation: An approach for achieving and attributing impact in complex systems. Agricultural Systems 78 pp243-265 • Douthwaite, B., Schulz, S., Olanrewaju, A., Ellis-Jones, J. 2006. Impact pathway evaluation of an integrated Striga hermonthica control project in Northern Nigeria. Agricultural Systems. Published on-line • Current Work (since Oct. 2005) • CPWF-supported, CIAT-led impact assessment project in 9 river basins ($900,000) • Douthwaite, B., Alvarez, B.S., Cook, S., Davies, R., George, P., Howell, J and Mackay, R. 2006. The Impact Pathways Approach: A Practical Application of Program Theory in Research-for-Development. For submission to an Evaluation Journal • Future Work • EU-funded, Wageningen-led “eco-system approach for co-innovation of farm livelihoods” project (Euro 1.8 million with 6 PhDs and 4 PostDocs) • Phase II of Knowledge Sharing for Research Project (with Simone Staiger) • PRGA INIS Project (CIAT and CIP led) • See www.impactpathways.pbwiki.com

6. Impact pathways – two conceptualizations…. Logic model Network maps

7. Impact Pathways Two complementary conceptualizations of a project, a program or an organization’s impact pathways: • A visual description of the causal chain of events and outcomes that link outputs to the goal (logic model); and • Network maps that show the evolving relationships necessary to achieve the goal • Implementing organizations; stakeholders; ultimate beneficiaries • Shows the project rationale; its logic • Foundation of ex-ante (and ex-post) impact assessment

8. Foundations of the IP Approach • Synthesis of concepts and tools from: • Program Evaluation • Renger and Titcomb (2002) – problem trees • Chen (2005) – program theory • Mayne (2004) - performance stories • Innovation histories • Douthwaite and Ashby, 2005 • Appreciative Inquiry • Whitney and Trosten-Bloom, 2003 • Social network analysis • Cross and Parker, 2004

9. The Process of Developing Impact Pathways

10. Participatory Development of Impact Pathways

11. Example of a Problem Tree

12. Turning a problem tree into an objective tree

13. The Process of Developing Impact Pathways – The Workshop

15. Scaling Out and Scaling Up • Scaling up - an institutional expansion, from adopters and their grassroots organizations to policy makers, donors, development institutions • Scaling out - spread of a project outputs (i.e., a new technology, a new strategy, etc.) from farmer to farmer, community to community, within the same stakeholder groups

16. Develop a vision of project success two years after the project ends • Work in project groups • Take 5 minutes to individually answer the question • You wake up 2 years after your project has ended. Your project has been a success and is well on its way to achieving its goal. Describe what this success looks like to a journalist: • What is happening differently now? • Who is doing what differently? • What have been the changes in the lives of the people using the project outputs, and who they interact with? • How are project outputs disseminating (scaling-out)? • What political support is nurturing this spread (scaling-up)? How did that happen? • Discuss and develop a common vision Keep it realistic Workshop

17. Example of a Vision

18. The Process of Developing Impact Pathways – The Workshop

19. Develop a project timeline from when your project started until 2 years after it will end • Build a timeline of activities, outputs and outcomes that take you from the beginning of the project to achieving the vision • It is a story of adoption of project outputs (scaling-out) and the political support that helps it along (scaling-up) Workshop

20. Example of a Timeline

21. The Process of Developing Impact Pathways – The Workshop

22. Family ties Friendship ties Workplace ties

23. Today’s tasks….. • Identify relevant actors & relationships • Develop network diagrams for • Your project now • Residual network 2 years after project has finished • Identify key extension (scaling out) and political support (scaling up) linkages • Identify differences between the two networks and discuss implications Workshop

25. Differences between the MUS project’s maps

26. The Process of Developing Impact Pathways – After the Workshop

27. Impact Narrative • Text description of the project impact pathways • Achieves the integration between the logic and network models • Helps with colligation (tracing of logical steps, Roberts, 1996), making hidden assumptions explicit • Helps with the plausibility of ex-ante impact assessment

28. IP Logic Model (Draft 1)

29. IP Logic Model (Final)

30. Network Maps

31. Separate relationships

32. Final maps based on answers

33. The theory behind the IP approach “Stakeholders' implicit theories are not likely to be systematically and explicitly articulated, and so it is up to evaluators to help stakeholders elaborate their ideas.” (Chen, 2005, p. 14)

34. Impact of IPs 1 • From Workshops • “I will use Impact Pathways in future design of projects” • “The dynamics of the networks is useful to envision the future” • “It helps show gaps” • “It is good for planning” • “It helps explain impact of my project” • “Constructing impact pathways should not be one-shot” • “The impact pathways should be a living document”

35. Impact of IPs 2 • Significant Change Stories resulting the Volta IP Workshop • Locally-organized IP Workshop to build basin program • Exploiting an opportunity for political lobbying • Network concepts hybridized with influence mapping to become main PhD research methodology • Problem and objective trees used to explain project to primary stakeholders

36. Impact of IPs 3 • Science Council review of CPWF 2007 – 2009 Medium Term Plan • ‘The CPWF has introduced the use of “objective trees” at the MTP project and CP level, a useful and innovative complement to the MTP logframe. In addition to providing a useful overview, the process of preparing these flow charts has clearly helped the CP provide the necessary focus, clarity and cohesion that now exists in the research plans at all levels.’

37. Impact Pathways Evaluation • Monitoring and evaluating progress along impact pathways • Regularly updating objective tree, timeline and network maps • Most Significant Change to pick up unexpected consequences • Provides the information needed for “adaptive management” • Impact Pathways Evaluation = Action research • Is publishable; raises the status of M&E

38. IPs and ex-post Impact Assessment • Ex-post impact assessment should identify and describe (EIARD, 2003): • The concept or model of innovation; • The logic model underlying a project or program • In other words, the normative and causative program theory

39. Research Questions • What types of network should R4D programs attempt to foster to achieve impact? • Food security, poverty alleviation, improved health, environmental security • How does improving the congruence between implicit and explicit stakeholder theories improve research-for-development effectiveness? • Are impact pathways generalizable? • Can one project’s impact pathways help planning and implementation of others?

40. IPs – the elevator conversation • What do you do? • Work to mainstream an impact orientation in agricultural R4D projects and programs • What is impact orientation? • That the people implementing projects are clearer about how their research will make a difference, and take responsibility that it does • How do you do that? • Work with practitioners to make explicit their impact pathways • Carry out research to understand which impact pathways work, when

41. Summary • Change Models matter: People plan and implement projects on the basis of their change models - their implicit theories about how the world works • If you can improve these theories you can improve the practice, making impact more likely • The Impact Pathways Approach – A participatory approach for: • Making practitioners’ theories explicit about how they will achieve adoption and impact (impact pathways); • Improving these theories • Using these models / frameworks for M&E and impact assessment • IP Evaluation = Action research • As a result, contributing to project and program “adaptive management” and thus likelihood of impact • A work in progress, including research on network structures and impact pathways

42. Impact Pathways Matter