Natalie Nicholles nef (new economics foundation) 19 th October 2010

1. Workshop: Value for Money (VfM) in international development using Social Return on Investment (SROI) Natalie Nicholles nef (new economics foundation) 19th October 2010

2. Housekeeping • Ground rules • Fire exits • Toilets • Workshop materials

3. Agenda

4. Aims for the day • To present an overview of what Value for Money (VfM) means • To introduce ways of capturing Value for Money in international development through the Social Return on Investment approach • To share nef’s work on this subject to date • To facilitate discussion on how to further develop demonstrating value and impact of international development work

5. new economics foundation (nef) • Founded in 1986 • Economics Think Tank working to promote: • Innovative solutions that challenge mainstream thinking on economic, environmental and social issues • nef consulting is the consultancy arm. It exists to promote and disseminate nef solutions • History of working with public sector on implementing value for money solutions

6. Problems with measurement • Financial measurement: • limited measure of value 2. We allocate resources only to the things we can measure 3: Stakeholders are left out of decision making

7. The challenge • Measurement across the ‘triple bottom line’ People The environment The economy

8. Value for Money (VfM) • VfM is about making sure that spending achieves as much as possible • A way of making decisions about how to use limited resources • Sometimes mistaken for lowest unit cost • Without a measure of quality or effectiveness, risk false economies • E.g. Children in care

9. Treasury VfM Definition “VfM is defined as the optimum combination of whole-of-life costs and quality (or fitness for purpose) of the good or service to meet the user’s requirement. VfM is not the choice of goods and services based on the lowest cost bid.” HM Treasury (2006) Value for money guidance

10. Treasury VfM Definition “Wider social and environmental costs and benefits for which there is no market price also need to be brought into any assessment. They will often be more difficult to assess but are often important and should not be ignored simply because they cannot easily be costed.” HM Treasury Green Book • Effectiveness, not lowest cost • Wider costs and benefits • Whole-of-life costs

11. Value for Money Often VfM is understood by comparing unit costs Resources / Investment Service & Wider Outcomes Money Economic Outputs People Inputs Social Environment Environmental Real VfM is achieved by comparing outcomes with investment

12. How does SROI fit in? • Framework for telling us how effectively money is spent – make VfM decisions • Adjusted form of cost-benefit analysis. SROI = value of positive + negative outcomes investment (or cost) • Essentially a measure of the efficiency of achieving outcomes • It considers triple-bottom-line benefits and investments – economic, social & environmental

13. Early 2000s: new economics foundation History of SROI Mid 1990s: REDF & Jed Emerson Mainstreaming: nef consulting & others

14. SROI today • Third Sector: SROI Guide • Local Government: commissioning • Think tanks/public policy • Central Government • Department of Health • National Audit Office: VfM • DFID: VfM • Pilot: HIV Aids Alliance • International interest • Europe, Australia, Canada, Asia

15. Crash Course on SROI: Understanding change

16. SROI Principles 1. Involve Stakeholders 2. Understand what changes 3. Value the things that matter 4. Only include what is material 5. Do not over claim 6. Be transparent 7. Verify the result

17. SROI Process • Engage stakeholders to identify outcomes • Data collection • Outcomes • Deadweight, attribution, displacement • Benefit period and drop off • Model and calculate • Valuation of non-traded outcomes • Report

18. Features that enhance VfM measurement • Focus on outcomes • Places quality and effectiveness at heart of analysis • Measures what matters, not what is easiest • Stakeholder-informed • Grapples with outcomes measurement • Values traded and non-traded outcomes • Failing to monetise non-traded outcomes effectively they gives these a value of zero • Rigour and transparency • Concerned with impact: deadweight, displacement, attribution, drop off

19. Develop a theory of change Inputs Activities Outputs Outcomes Output: Tells you an activity has taken place and is usually quantitative (e.g. number of people trained) Outcome: The change that occurs as a result of an activity (e.g. improved well-being of training participants)

20. Case study: African Community Development Foundation (ACDF) Background • ACDF started of as a U.K community based organisation, supporting socially isolated Africans in various ways. • In 2009, ACDF sent 28 volunteers as a pilot in Kisumu, Kenya - known as Diaspora Development Associates (DDA). • ACDF specialise in team volunteering • Kenya has a preference for formal employment rather than self-employment but ormal employment hard to get • High level migration from Kisumu to Nairobi to secure jobs • Due to preference for formal employment, very limited public interest for business training in Kisumu so organisations do not provide it. • The Government of Kenya in 2008 launched an ‘enterprise fund’ to encourage people to start their own business but there was hardly any uptake of these funds in Kisumu.

21. Case study: African Community Development Foundation (ACDF) Activity • ACDF delivered Train the Trainer courses to infrastructure organisations and to direct beneficiaries. • This comprised of DDAs with skills covering- business set up, business management, book-keeping, organisational development, fundraising etc.

22. Exercise 2: Impact Mapping What changes do the stakeholders experience as a result of the project? I.e. what is the story of change?

23. Impact map Inputs / Activities Stakeholders Outputs Outcomes Train the trainer courses on: business set up, business management, book-keeping Kisumu’s unemployed resident X Kisumu’s infrastructure organisations ACDF’s volunteers (DDAs) State

24. Outputs vs. outcomes • Outputs do not always lead to the desired outcome • Key challenge in determining VfM is measuring outcomes • Outcomes measurement rarely takes place • Biggest obstacle in our work across sectors/organisations is the lack of outcomes data

25. Crash Course on SROI: Measuring & valuing change

26. Measuring outcomes • Some outcomes are ‘hard to measure’ • subjective, qualitative, diverse, far removed from the activity, long term • Systems and tools for capturing complex outcomes are improving • distance travelled • translating into quantitative format that enables aggregation • collecting complex information simply

27. Evidencing outcomes • Distance travelled

28. Example: National Accounts Model of well-being www.nationalaccountsofwellbeing.org

29. Evidencing outcomes • Select ‘ways of knowing’ that an outcome (a change) has taken place = indicators

30. Ways of measuring

31. Valuing • SROI analyses value to all material stakeholders, not just the one funding the activity. • Key question: “what does this (change) mean to you?” Exercise • Imagine your employer gave you a day off. What will you do on your day off? What does this free time mean to you? • If you had to put a financial value on this, what would it be? I.e. what is this day off worth to you?

32. Valuing and Pricing Value means different things to different people. • How much is your house/flat worth? • Who sets the price of fish? BUT • What does it mean to a Kisumu resident to have a job? • SROI uses financial proxies to estimate the social value of non-traded goods to different stakeholders. • Common currency

33. Subjectivity of value Value is adjusted...not by any accurate measure, but by the higgling and bargaining of the market, according to that sort of rough equality which, though not exact, is sufficient for carrying on the business of common life.

34. Subjectivity of value Can I put a value on anything? Can I find a value which, though not exact, is sufficient for helping me evaluate change?

35. Financial proxies • Sometimes this is straightforward • mainly with outcomes to the State (e.g. value of health) • More difficult with non-traded outcomes • These do not have a ‘price’ and so require a proxy, or stand-in (e.g. emotional well-being) • Standard economic valuation techniques • Contingent Valuation • Willingness to pay (e.g. new park) or willingness to accept compensation (e.g. noise pollution) • Revealed Preference • Hedonic pricing (e.g. high risk job); Travel cost method (e.g. local service); Observed spending on related goods (e.g. leisure)

36. Monetisation • Each outcome is then valued • Where no direct financial value is available, we use financial proxies to represent the social value created

37. Valuation exercise Exercise: • In pairs, select 2 outcomes from the ACDF impact map • Think about options for financial proxies for the outcome. Remember, we are putting a value on the outcome for that particular stakeholder!

38. Crash Course on SROI: Attribution & Partnership working

39. Understanding impact • Deadweight: what would have happened anyway? • Attribution: how much is down to this project, and how much down to other factors? • Displacement: have we just moved an outcome to / from somewhere else? • Benefit period: how long does the outcome last, and does the effect ‘drop off’ over time?

40. Impact: Attribution • Attribution – how much credit can your organisation take for the outcomes? • Expressed as a percentage • Not an exact science • Methods: • Based on your understanding of the journey of change • Involve stakeholders – interviews or surveys • Consult with other organisations

41. Attribution stages • No account of attribution levels • Effectively attribution = 100% 2. Informed estimate of attribution levels Qualitative research allows SROI practitioner to divide attribution between contributors 3. Quantitative research Quantitative stakeholder research gives average stakeholder’s view on contributors • Academic research • Academic research can help refine (although not replace) stakeholder research

42. Quantitative stakeholder research Attribution Time (years)

43. Partnership working

44. Projecting into the future • Benefit period – the length of time over which outcomes are expected to endure • e.g. benefits of an employment training programme may endure for some years after the course • Drop off – the rate at which benefits decrease over time • e.g. it is likely that benefit for training participants wears off as time goes on • Time value of money : discount rate • NB. Comparing value in different countries

45. Inputs • Size of investment • If just looking at one project within organisation, need to work out its ‘share’ of inputs (Full Cost Recovery) • Where multiple funding streams, need to have identified the share of benefits when scopingif looking at return on just one of these investments • Include financial and non-financial inputs e.g. volunteer time

46. Example & Organisational decision-making

47. Hypothetical Example Outcome: capture the value of the influence on national policy-making i.e. changes for stakeholders that result from the national policy. INGO in Nigeria • Part of National Task Group on Sanitation • Successfully completed community-led total sanitation pilot (CLTS) • Advocated within the Task Group for drafting of National Strategy for Scaling-up Rural Sanitation and Hygiene • Key lesson: INGO’s advocacy on the National Strategy is not the outcome – it is a means to an end.

48. INGO Impact Map Outcomes Value of improved health to the individual Value of increased economic activity to the State

49. Value to the individual • No monitoring in place to track impact of the National Strategy • Forecasted using secondary research and stakeholder engagement Outcome Incidence Assumption 1: An additional 2% of the rural population is provided with improved access to sanitation.

50. Value to the individual Impact • Deadweight (population reached) -0.25% • baseline trend improvements in sanitation for rural communities (UNICEF/WHO) • Attribution to National Strategy 75% • significant change but concurrent factors included: International Year of Sanitation 2008 which saw increases in funding and other new initiatives • Attribution to INGO 5% • number of other actors involved in Task Group e.g. UNICEF, DFID. Credit for role in policy formation is 35% but implementation involved other actors including local government and this reduced attribution to 5%.