Theory of Change, Impact Monitoring, and Most Significant Change

1. Theory of Change, Impact Monitoring, and Most Significant Change EWB-UK Away Weekend – March 23, 2013

2. Theory of Change Theory of Change is a roadmap of change, offering pictures of destinations and guiding you on your journey so you know you’re on the right path. This can be done by mapping outcomes, which in the long-term are known as impact.

3. Impact • Long-term outcomes – the actual change you want to achieve • Intended and unintended consequences • Can be positive or negative

4. Outcome Map One way of laying out this pathway is to create a “so that” chain: You start with a powerful strategy, so that a short-term outcome will occur, so that a long-term outcome will happen, so that there will be an impact. I.e. Link strategies with outcomes, long-term goals, and vision

5. ToC Outcome Map Powerful Strategies lead to Short-Term Outcomes which lead to Long-Term Outcomes (Goals) which lead to the desired Impact (Vision)

6. EWB-UK Theory of Change Map Under the new strategy, everything connects to everything! Each Portfolio contributes in different ways to the same goals

7. Vision First, you want to clarify your goals by identifying the ultimate impact you aim to achieve – this is generally an ambitious visionary statement: “A world where everyone has access to the engineering they need for a life free from poverty”

8. Powerful Strategies Strategies that are already in place that address the ultimate impact: “Create a sense of belonging where one can identify with the cause, a sense of being part of a movement”

9. Short-Term Outcomes Example from EWB-UK ToC: “Platform to bring motivated students from across the UK together”

10. Long-Term Outcomes (Goals) • Awaken greater attention to global challenges and opportunities • Educate engineers about international development • Excite and inform people about the role and impact of engineering • Empower engineers to respond to global challenges • Enable new paths of development that are appropriate, sustainable, and inspirational • Transform the engineering profession into an enabling environment for positive change • Relieve poverty in the communities where our international partners work • Enhance the capabilities of people, communities, and partners • Discover and evolve technologies and approaches that address barriers to development • Unleash passionate, talented, and transformational leaders

11. Long-Term Outcomes (Goals) 4) Empower engineers to respond to global challenges

12. Vision “A world where everyone has access to the engineering they need for a life free from poverty”

13. Theory of Change ToC is a strategic picture of the multiple interventions required to produce the early and intermediate outcomes that are preconditions of reaching an ultimate goal (impact).

14. How ToC and M&E feed in to: Impact Evaluation

15. Monitoring and Evaluation • Monitoring is: • continuous collection of data throughout project cycle • Internal / for project management • Evaluation is: • Carried out at significant stages to assess progress towards goals • Can be a self-evaluation or external

16. Logic Model

17. Understanding M&E for Impact Monitoring: ToC vs. Logic Model • A logic model is a tactical explanation of the process of producing a given outcome. It outlines the program inputs and activities, the outputs they will produce, and the connections between those outputs and the desired outcomes. • The ToC summarizes work at a strategic level, while a logic model would be used to illustrate the program-level understanding of the change process.

18. ToCand Logic Models Once a precondition (or outcome) has been identified through the ToC process, a logic model can be used to explain how that outcome will be produced. Thus, one ToC could actually be linked to a number of logic models.

19. M&E Tools: Indicators SMART • Specific • Measurable • Achievable • Relevant • Time-bound

20. Most Significant Change Most Significant Change (MSC) technique is a means of “monitoring / evaluating without indicators”

21. MSC is most useful: • Where it is not possible to know what the outcome will be • Where outcomes will vary widely

22. What is MSC? MSC is  a form of participatory monitoring and evaluation. It is participatory because many project stakeholders are involved both in deciding the sorts of changes to be recorded and in analyzing the data collected.

23. How it links to M&E • It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program. • It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as a whole.

24. How it works • Essentially, the process involves the collection of significant change (SC) stories from the field level, and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff. The designated staff and stakeholders are initially involved by ‘searching’ for project impact. Once changes have been captured, selected groups of people sit down together, read the stories aloud and have regular and often in-depth discussions about the value of these reported changes, and which they think is most significant of all. In large programs there may multiple levels at which SC stories are pooled and then elected. When the technique is implemented successfully, whole teams of people begin to focus their attention on program impact.

25. Activity • Split up into groups of 5 • Take turns briefly describing to the other members of your group what the biggest impact of being a member of the national executive has had on your life – positive or negative • Look for common themes and pick one story to represent your group as a whole • Each group shares their story