1 / 75

Challenging Evaluation: An Introduction to Outcome Mapping

Challenging Evaluation: An Introduction to Outcome Mapping. Amy Etherington & Rebecca Lee Mini-training for IDRC Interns & PDAs December 7 & 8, 2005. Session overview:. Evaluation context – where does OM fit? Introduction to OM methodology – main concepts and tools OM book – it’s useful!

hop
Download Presentation

Challenging Evaluation: An Introduction to Outcome Mapping

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Challenging Evaluation: An Introduction to Outcome Mapping Amy Etherington & Rebecca Lee Mini-training for IDRC Interns & PDAs December 7 & 8, 2005

  2. Session overview: • Evaluation context – where does OM fit? • Introduction to OM methodology – main concepts and tools • OM book – it’s useful! • How is OM being used – when is it appropriate?

  3. Warm-up question: What words come to your mind when you think of “evaluation”?

  4. IDRC’s Evaluation System • Promotes ownership & use of findings at all levels • Decentralized • Focus on evaluation processes • Monitors quality • Accountability for results at program & project levels

  5. IDRC’s Evaluation Unit • Strategic evaluations • Cross-cutting issues • Corporate reporting • To BoG on performance and results • Information systems • Capacity building • With Southern partners • Promoting M&E with programs • Tools & methods • Organizational Assessment • Outcome Mapping

  6. Monitoring & evaluation challenges: • Establishing cause & effect in open systems • Sharing ownership & participation • Recognizing the contributions of others • Tracking progress • Encouraging iterative learning • Measuring development results of research • Timing

  7. Shifting influence over time Influence Time

  8. Are we having an impact yet ?

  9. A framework that allows researchers to plot human behaviour and actions and assess their contribution to the aims of research projects and programs • Outcomes as changes in behaviour, relationships, activities or actions of the people, groups and organizations with whom a program works directly IDRC’s Evaluation Unit (1998-2000)

  10. OM bumper-stickers: The map is not the territory! • Be prepared for surprises along the way Keep your eyes wide open! • Being attentive along the journey… • This is as important as the destination

  11. OM t-shirts CONTRIBUTION ATTRIBUTION

  12. OM t-shirts INFLUENCE IMPACT

  13. OM recognizes that in a development context, change is: • Complex (multiple actors and factors) • Continuous (not limited to the life of the project) • Non-linear (unexpected results occur) • Beyond the control of the project (but subject to its influence) • Two-way (program also changes)

  14. Principles of use • Flexible: modular - use adapted to circumstances • Participatory: seeks dialogue and collaboration with boundary partners in P,M,&E • Evaluative thinking; culture of reflection:promotes social and organizational learning

  15. Step 1: Vision

  16. I have a dream! Martin Luther King, Jr. August 28, 1963

  17. Vision is a guide: North Light house Distant star

  18. Vision statement: Reflects the broad human, social, and environmental betterment in which the program is engaged and to which it is contributing • Written in present tense - as if the vision were already a reality

  19. “Those who dream make a difference!” Ms. Kalpana Pant -Chaitanya

  20. Women and girls in rural India enjoy full access to appropriate health care, education, food and water security and freedom from violence. They earn their own livelihoods with free access to markets and to banking and municipal services. Women knowledgeably use drudgery reduction technologies and agricultural practices that support human and ecological well-being. Villages are served by public transport, are well lit at night and have police forces that enforce laws fully and equitably. Girls attend school full time and, with their families, have the information and resources to make informed decisions regarding their personal health, safety and social needs. Gender equity is evident in the allocation of household labor and household decision-making; and men in the community support gender-responsive initiatives. Schools reinforce the role of women as educated, valuable and empowered community members. Through community-based organizations, women participate in and influence decisions, policies and programs affecting their well-being and share their experiences with others.

  21. Vision: facilitation questions • What is the human, social and/or environmental condition that you hope to see in the region in which the program is being carried out? • Who is contributing to this situation - and how? Dream boldly !

  22. Step 2: Mission

  23. The mission is that “bite” of the vision statement on which the program is going to focus. • Written in future tense - as something the program will do

  24. Your mission is your “business” • What do you do? • Who do you do it with? • Why do you do it?

  25. Mission: facilitation questions • How can the program best support or contribute to the achievement of the vision? • What areas do you need to work in? • Where do you have credibility? • Who can you work with?

  26. Step 3: Boundary Partners

  27. Boundary Partners: definition • interacts directly to effect change • anticipates opportunities for influence • engages in mutual learning Those individuals, groups, and organizations with whom the program:

  28. = partners The rest of the world Program

  29. Sphere of influence The rest of the world Program = partners

  30. Boundary Partners have Boundary Partners program program’s bp bp’s bp

  31. Swayamsiddha State NGO State NGO State NGO State NGO State NGO State NGO BAIF SHG Community Leaders Police IDRC Banks Families PHCs CIDA

  32. Boundary Partners: Facilitation Questions • In which individuals, groups, or organizations is our program trying to encourage change so that they can better contribute to the vision?  • With whom will we work directly?  • Are we choosing X BP because we want to influence their behaviour and actions, or because they will influence others? Or both? • What behavioural changes do we (the project and BPs, collectively) want to see in the BP that will contribute to the vision?

  33. Step 4 & 5: Outcome Challenges and Progress Markers

  34. Outcome Challenge: definition • One OC is about a single boundary partner • Describes the ideal behavioural changes, relationships, actions and interactions in this partner • Describes how these changes will contribute to the vision.

  35. Outcome Challenge: characteristics • One for each boundary partner • Does NOT describe program strategies • Written like this: “The program intends to see [boundary partner] who [description of behaviours in the active present tense]”

  36. Outcome Challenges: facilitation questions • Ideally, how would your boundary partner be acting? • With whom would they be interacting? • What would they be doing?

  37. Progress Markers: description • A graduated set of statements describing a progression of changed behaviours in the boundary partner • Describe changes in actions, activities and relationships leading to the ideal outcome • Articulate the complexity of the change process • Can be monitored & observed • Permit on-going assessment of partners’ progress (including unintended results)

  38. 3 Levels of Progress Markers The program sets out what it would: • Expect to see the boundary partner doing? • Like to see the boundary partner doing? • Love to see the boundary partner doing?

  39. Progress markers = ladder of change Love to see Truly transformative Set quite high Like to see More active learning, engagement Expect to see Early response to program’s basic activities

  40. X

  41. Why graduated progress markers? Taken as a set, the progress markers: • are graduated from easier to more difficult to achieve changes in behaviour • describe the change process of a single boundary partner • are more complete than a single indicator • help the program think about how it can intentionally contribute to the most profound transformation possible • facilitate mid-course corrections and improvement

  42. BoundaryPartner Program Program relevance & viability Org. Practices Program Delivery Strategies Program Results Progress Markers Influencing Outcomes

  43. Step 6: Strategy Map

  44. Strategy Map • Outlines the program`s approach in working with the boundary partner • Indicates the relative influence the program is likely to have on boundary partner • Helps pinpoint strategic gaps in the approach or if the program is overextended

  45. 6 Types of Strategies Strategy Causal Persuasive Supportive I-1 • Direct Output I-2 • Arouse New Skills/ Thinking I-3 • Supporter who guides change over time Aimed at the Boundary Parnter E-1 • Alter physical or regulatory environment E-2 • Modify the information system E-3 • Create / Strengthen a Peer Network Aimed at the Boundary Partner`s Environment

  46. Facilitation Questions:

More Related