Measuring Outcomes How do we understand the changes we are making …

1. Measuring OutcomesHow do we understand the changes we are making … Workshop for Funding Fair 2013

2. VAW’s Just For You project Using mainly the Charities Evaluation Services (CES) outcomes-focussed approach to planning, monitoring and evaluation to: • Make your work more effective. • Improve how you communicate the outcomes of your services (especially to funders).

3. What’s the story? • Clear about the changes (outcomes and impact) that we are seeking to achieve. • Clear about the relationship between these and what we do. • Identifying quantitative and qualitative indicators of changes we make. • Robust measurement methods. • Clear and concise reporting.

4. Outcomes Planning Triangle Impact: lasting change brought about by our work Need: a problem or issue, or situation where something needs to change to make things better, for a person, a group of people, an environment or an organisation. Overall Aim or Mission Outcomes: the changes, benefits, learning or other effects that happen as a result of our work. Specific Aims: This is why we do our work Inputs: the resources put into a project to carry out an activity. Outputs: the detailed activities, services and products we do or provide. Objectives: This is what we do

5. Task – 5-10 minutes • Think about the Outcome or Outcomes (change) that motivate you most – use the triangle if it helps. • Share with your neighbour • Not a test! – rather a way for us all to focus our thinking on the changes that we each individually are involved in bringing about.

6. Feedback

7. Need: children are falling behind at school because of problems at home THEORY OF CHANGE The beginning of making a difference Angela Kail, Tris Lumley NPC April 2012

8. Making sure that outcomes are realistic • Many charities have grand aims, such as ‘the alleviation of poverty’. But aims like these are too large for a charity to achieve on its own, so it is not sensible to think about how to measure them. • A plan for change helps charities focus on concrete, defined aims and outcomes, which are potentially measurable.

9. Understanding progress towards the final goal • Some final goals cannot easily be measured. They involve change that happens too gradually or change that happens in the lives of people who are difficult to track. • Planning intermediate steps that lead to the end goal can help organisations work out whether they are making a difference towards that end goal, by measuring the intermediate steps.

10. Measuring outcomes http://www.biglotteryfund.org.uk/funding/funding-guidance/applying-for-funding/

11. Outcome indicators • Outcome indicators are well defined pieces of information that can be assessed or measured to show whether outcomes have been achieved. • Outcomes indicators enable us to measure progress towards planned outcomes • They are signs or clues that you monitor and can be qualitative or quantitative. • Each outcome has at least one indicator - some have many indicators.

12. Outcomes indicators – outcome ‘people are ready for work’ • An example of a quantitative indicator might be the number and type of qualifications gained by people attending a training centre. • An example of a qualitative indicator might be the change in confidence of those attending the centre (whether it increased, decreased or stayed the same).

13. Robust measurement methods 1 • Knowing from the start what the need is, what change you seek to make (aim), the change you will make (outcome), the ways that change - and any others - will be demonstrated and measured (outcome indicators) …. you already know the what and why. • So what is left is HOW?

14. Robust measurement methods 2 • Collect only the information you need and can use • Use methods that are sensitive to the people you want to collect information from • Involve others in the choice and design of your information collection methods • Fit this info collection into other work processes as much as possible.

15. supporting voluntary and community action Interviews Self-completion tools (surveys, tests, forms, diaries) Observation Keeping records and notes Group activities – focus groups, group interviews, Visual/audio methods – photos, video, voice recordings Social media – blogs, podcasts, twitter

16. Task Identify with your neighbour one way each that you are familiar with or have used recently to collect information about your work. • Did the information demonstrate a change? • If so, what was that change? • Was it quantitative or qualitative info? • How robust was it – is it open to challenge, will it ‘fall down’ if questioned?

17. Feedback

18. Assessing Change: Developing & using outcomes monitoring tools * Planning your outcomes monitoring Selecting appropriate methods Developing monitoring tools Designing your own tools Finding and using off-the-shelf tools Preparing to collect outcomes information Collecting outcomes information *By Diana Parkingson and AvanWadia for CES Oct 2010 http://www.ces-vol.org.uk/Publications-Research/publications-free-downloads/publications-free-assessing-change http://www.biglotteryfund.org.uk/funding/funding-guidance/applying-for-funding/

19. Clear and concise reporting B) Monitoring A) Planning Overall aim Specific aims: This is why we do our work Objectives: This is what we do 1) Clear purpose Outcomes: 4) Demonstrate results 2) Defined aims Outputs: 3) Coherent activities 5) Evidence C) Evaluation 6) Lessons learned

20. Discussion and Questions

21. THANK YOU Emma Plouviez Organisational Development Officer Voluntary Action Westminster eplouviez@vawcvs.org