Outcome Mapping Planning, Reflecting & Learning

1. Outcome MappingPlanning, Reflecting & Learning Shalini Kala July 2011

2. Outline • OM: basic principles • Overview of the steps • An example of OM application • My experience of using OM • Q&A and short exercises in between

3. Acknowledgements This presentation makes use of materials from Simon Hearn and Terry Smutylo available at www.outcomemapping.ca

4. OM history • 1990s: post-Rio need to demonstrate ‘sustainable’ results • 1998: Barry Kibel and Outcome Engineering • 1999: Methodological collaboration with projects • 2000: Publication of manual in English • 2002: Training, facilitation & usage globally • 2006: OM Learning Community www.outcomemapping.ca • 2008: Latin America Centre for Outcome Mapping (CLAMA) • 2010: East Africa and beyond

5. Evaluation challenges that led to the development of OM • How do you measure what difference research is making • How do you establishing cause & effect in open & complex system we work in • Timing: when do you evaluate • Iterative learning

6. The research to impact connection is complex

7. Focus of Outcome Mapping Program influence decreases Community ownership increases Inputs Activities Outputs Outcomes Impacts Outcome Mapping

8. Key ideas characterizing OM • Looking at the bigger picture • Recognizing that change is: continuous, multi-dimensional, complex, non-linear • Learning as you go • Contribution not attribution

9. Brief definition of OM • A participatory method for planning, monitoring and evaluation • Focused on changes in behaviour of those with whom the project or program works • Oriented towards social & organizational learning

10. Three key concepts in OM • Sphere of influence • Boundary Partners • Outcomes understood as changes in behaviour

11. Sphere of interest Sphere of influence Sphere of control There is a limit to our influence Project Partners Beneficiaries

12. Sphere of interest Sphere of influence Sphere of control Changes in behavior as ‘outcomes’ Inputs, activities, outputs Outcomes: Changes in behavior Impact: Changes in state

13. Participating farmers learn how to use drip irrigation equipment Reduced numbers of new wells Increased knowledge of techniques Farmers participate in field trials Extension workers visit demonstration farms Farmers adopting drip irrigation methods Participatory research on demonstration farms to develop approaches to drip irrigation Training of extension workers Greater quantities of groundwater available Extension workers promoting drip irrigation Publication of performance of different set-ups Source: Terry Smutylo

14. Who are your boundary partners? Programme Beneficiaries Stakeholders Boundary Partners

15. To what extent do these ideas apply to the programs and context you work in?

17. 4 Key Planning Questions Vision Boundary Partners Outcomes Challenges, Progress Markers Mission, Strategy Map, Organizational Practices

18. Step 1: Vision improved human, social, & environmental wellbeing

19. A vision statement…. • guides • motivates • is an accountability-free zone

20. Step 2: Mission The mission is that “bite” of the vision statement on which the program is going to focus.

21. A mission statement describes… • How the program intends to apply its resources in support of the vision • The areas in which the program intends to work • How the program will support the achievements of outcomes by its direct partners

22. Swayamsiddha Project • Context: Women in India are disempowered • Project: • Started in 2000, closed in 2005 • Funded by CIDA and IDRC • Managed by national NGO and local NGOs • Aims: • Develop network of government, non-governmental and community based organisations • Increase gender responsiveness in local health care, families and community institutions • Decreasing drudgery in women’s and girl’s work • Increase access to and control of financial services

23. Swayamsiddha Vision Across rural India, women and girls utilize and benefit from appropriate health care, education, food and water security and freedom from violence. Women have access to the markets, credit, banking and municipal services they need to pursue their livelihood goals. They use drudgery-reducing technologies and agricultural inputs that contribute to personal well-being and to ecological sustainability. Villages are fully served by public transport, are well lit at night and police enforce all laws fully and equitably. Girls attend school full time and families have the information and resources to make informed decisions regarding their health, safety and social needs. Gender equity governs household labor and decision-making; and men in the community understand and support gender-responsive laws.

24. Swayamsiddha Mission The Swayamsiddha Project works with governments, NGOs & CBOs to improve women’s health and empowerment. It facilitates the development of women’s self help groups. It provides them with funding and training to help them influence community and government services to be more responsive to their health and livelihood needs. It fosters mutual respect and joint action between these self-help groups and: banks; police; health and social service providers; and government agencies. It researches and promotes the application of ecosystem approaches to human health in agriculture and in the provision of health and sanitation services. Swayamsiddha addresses equity issues in all its activities. It uses participatory methods to monitor progress, to learn how to become more effective in supporting its partners and to report on its results.

25. Step 3: Boundary Partners Those individuals, groups, & organizations with whom a program interacts directly to effect change & with whom the program can anticipate some opportunities for influence.

26. CIDA IDRC BAIF State NGO State NGO State NGO State NGO State NGO State NGO Families PHCs Banks Community Leaders Police SHG Swayamsiddha BPs

27. Questions

28. Group exercise 1: 15 minutes • Get together in small groups with your boundary partners present here • Come up with a vision & mission statement for your project • Identify two levels of boundary partners of your project You might want to start by thinking of key words or ideas that should feature in these statements

29. Step 4: Outcome Challenge • Describes behaviour of a single boundary partner • Sets out the ideal actions, relationships, activities • Describes the boundary partner’s contribution to the vision

30. Outcome Challenge for Swayamsiddha SHG Women’s self help groups are taking action to make community and government services more responsive to the health and livelihood needs of women and girls. They influence banks, police, health and social service providers, local officials and state and national government agencies in relationships of mutual respect and joint action to improve women’s well being. Women’s self help groups arrange bank loans for members and for life skills training for girls to be included in the school curriculum. They influence local, state and national government policies and expenditures on community improvement and transportation and support women candidates to run for election to local government office.

31. Step 5: Progress Markers Love to see (Deep transformation) Like to see (Active engagement) Expect to see (Early positive responses)

32. Progress Marker Checklist Each Progress Marker: • Describes a changed behaviour by the boundary partner • Can be monitored & observed As a set, Progress Markers: • Are graduated from preliminary to more profound changes in behaviour • Describe the change process of a single boundary partner

33. Why Graduated Progress Markers? • Articulate the complexity of the change process • Allow negotiation of expectations between the program and its partners • Permit on-going assessment of progress • Encourage the program to think about how it can intentionally contribute to the most profound transformation possible • Help identify mid-course corrections and improvements

34. X

35. Sample progress markers: Swayamsiddha SHG

36. Group exercise 2: 20 minutes • Get together with your boundary partners present here • Come up with an outcome challenge statement • Come up with a list of associated progress markers

37. Step 6: Strategy Map

38. 6 kinds of strategies

39. Step 7: Organisational Practices • Prospecting for new ideas, opportunities, and resources • Seeking feedback from key informants • Obtaining the support of your next highest power • Assessing and (re)designing products, services, systems, and procedures • Checking up on those already served to add value • Sharing your best wisdom with the world • Experimenting to remain innovative • Engaging in organizational reflection

40. Vision Mission BOUNDARY PARTNERS Outcome Challenges Progress Markers Strategies Organizational Practices Outcome Mapping: Main Elements

41. OM principles of use • Flexible: modular to be adapted to use and context • Complementary: can be combined with other methods • Evaluative: promotes culture of reflection, results oriented thinking, & social & organizational learning • Participatory: seeks dialogue and collaboration with partners

42. Quick Questions

43. OM in real life Experience of networking actors to reduce rural poverty in Asia

44. ENRAPKnowledge Networking for Rural Development in Asia-Pacific Region • Started in 1998, covering 5 countries-15 projects, expanded to 8 countries (40 projects), ended in March 2011 covering all of Asia-Pacific (over 65 projects) • IFAD-IDRC collaboration • Goal: Increased sharing of knowledge & information for rural poverty reduction in Asia. • Objective: IFAD partners are using networks to actively share knowledge and information. • Improved knowledge sharing was directed at supporting project performance

45. ENRAP promoted knowledge sharing through network buildingacross projects & among rural dev. players at the regional and national level. • IFAD poverty reduction projects & grant projects • National government agencies & NGOs • People’s organizations (farmer federations) • Research institutions • Worked through IFAD projects & later country offices • Depended on supportive environment at IFAD HQ for effectiveness

46. Why did we choose OM? Project aimed at behavior change, in this case: networking for knowledge An evolving KM environment at IFAD A relatively small player (attribution vs. contribution) 1-2m for portfolio of over 450m 3-year phases External As the learning framework

47. What did we do? Adapted OM pragmatically Used along with LFA Worked with small number of boundary partners over email to draft the initial outcome map Revised the outcome map based on M&E survey result after one year of programming Contracted an M&E agency for annual monitoring

48. How did we use OM? To monitor progress Tracking change in behavior, in gen. Tracking comfort of network members with new technology (social media tools, internet, etc.), in particular To learn along the way to inform project mgt. To validate our perceptions

49. Benefits It works for better project management Able to demonstrate progressive change As a concept, easier to understand & apply Greater effectiveness in learning &monitoring because of the participatory nature Adaptable – we used what we could Growing community of users, very ready to help

50. Challenges Partner’s lack of understanding of OM Time & resource intensive Relatively less known Training is expensive Trainers are few Not a whole lot of organizations have experience with it Convincing partners requires time & effort