Landscape Mapping for Impact Measurement and Management in South Africa

1. Landscape Mapping for Impact Measurement and Management in South Africa

2. How to navigate this guide Click the icons to explore the differences by stakeholder group: Click on the Next Slide and Dashboard buttons to navigate through the infographic Overall Results Intermediaries Next Slide Dashboard Investors Investees Click on icons throughout the infographic to explore relevant resources and hear more about what study participants had to say about each of the themes! Terminology: • We use ‘survey respondents’ to refer to data collected through the online survey. • ‘Interview participants’ refers to data collected through in-person or telephonic interviews. • ‘Study participants’ includes all individuals and organisations engaged as part of the study. Next Slide Dashboard

3. Dashboard Are organisations measuring and managing impact? Challenges with practicing IMM Why are organisations practicing IMM? Using data and reporting results Why are organisations not practicing IMM? Future of IMM What IMM techniques are organisations using? Resources Benefits of practicing IMM

4. Are organisations measuring and managing impact? 84%of interview participants and 66% of survey respondents reported having an impact measurement and management (IMM) practice Interview participants practicing IMM Survey respondents practicing IMM Overall Results Investees Intermediaries All Respondents All Respondents n = 31 n = 58 Investors Dashboard

5. Are organisations measuring and managing impact? 78% of investee interview participants and 50% of investee survey respondents reported having an IMM practice Interview participants practicing IMM Survey respondents practicing IMM Overall Results Investees Intermediaries Investees Investees n = 9 n = 8 Investors Dashboard

6. Are organisations measuring and managing impact? 88% of intermediary interview participants and 93%of intermediary survey respondents reported having an IMM practice   Interview participants practicing IMM Survey respondents practicing IMM Overall Results Investees Intermediaries Intermediaries Intermediaries n = 8 n = 8 Investors Dashboard

7. Are organisations measuring and managing impact? 83% of investor interview participants and 73% of investor survey respondents reported having an IMM practice   Interview participants practicing IMM Survey respondents practicing IMM Overall Results Investees Intermediaries Investors Investors n = 12 n = 8 Next Slide Investors Dashboard

8. Why are organisations practicing IMM? • Organisations recognise IMM as central to achieving company growth and organisational objectives Overall Results Investees Intermediaries Investors Dashboard

9. Why are organisations practicing IMM? • Investors identified IMM as central to achieving company growth and organisational objectives Overall Results Investees Intermediaries Investors Dashboard

10. Why are organisations practicing IMM? • While intermediaries emphasised the importance of IMM for increased buy-in from stakeholders and investors Overall Results Investees Intermediaries Investors Dashboard

11. Why are organisations practicing IMM? • Similar to investors, investees identified achievement of company growth and organisational objectives as the most important reason for practicing IMM Overall Results Investees Intermediaries Next Slide Investors Dashboard

12. Why are organisations not practicing IMM? • We asked survey respondents to indicate why they do not practice IMM. • These are the top six-ranked reasons for why organisations are not measuring and managing their impact: 1 Insufficient financial resources 2 We don’t know what IMM is or how to measure impact 3 Insufficient human resources 4 We haven’t set impact goals 5 We haven’t found the right tool 6 Impact is not a priority for our organisation Overall Results Investees Intermediaries Investors Dashboard

13. Why are organisations not practicing IMM? Training on tools to use to measure impact. We also require budget for implementing and maintenance of data. We might also need to get someone skilled to help us with the process. - Investee • 1 • Insufficient financial resources • 2 • We don’t know what IMM is or how to measure impact • 3 • Insufficient human resources No, we are not currently measuring impact because we don’t know how. This is an area that I am interested in but not familiar with so I don’t know how to get started. - Investee [We need] more human resources. - Investee • 4 • We haven’t set impact goals • 5 • We haven’t found the right tool • 6 • Impact is not a priority for our organisation Overall Results Investees Intermediaries Investors Dashboard

14. Why are organisations not practicing IMM? • 1 • Insufficient financial resources • 2 • We don’t know what IMM is or how to measure impact • 3 • Insufficient human resources • 4 • We haven’t set impact goals • 5 • We haven’t found the right tool Overall Results • 6 • Impact is not a priority for our organisation • As an organisation, we are still defining what impact means for us and what steps are needed in order for us to achieve impact. • - Intermediary We are new to the space and no, we do not have a specific technique at the moment. - Intermediary Investees Intermediaries Investors Dashboard

15. Why are organisations not practicing IMM? • 1 • Insufficient financial resources • 2 • We don’t know what IMM is or how to measure impact • 3 • Insufficient human resources Starting IMM is a distinct challenge for an emerging [investor]. We don’t have the resources. - Investor • 4 • We haven’t set impact goals • 5 • We haven’t found the right tool Overall Results • 6 • Impact is not a priority for our organisation This would need to become the industry standard and a requirement of external investors (funders/LPs). - Investor Investees Intermediaries Next Slide Investors Dashboard

16. What IMM techniques are organisations using? We started by understanding whether study participants’ use of IMM techniques varies according to the investment cycle. This is because different techniques, whether standardised or custom, can be used for different objectives based on the stage of the investment cycle. However, only22% of survey respondents report that their approaches vary according to the investment cycle. Evaluating Impact To prove social impact, exit investment and adjust investment thesis Forecasting Impact For investment selection and due diligence “The intention is to go right through the [IMM] cycle. For instance, pre-investment, we screen companies. During forecast and planning, we look at specific data and goals to get a better understanding of where a company is and what percent of impact we can look to improve over time…” Establish impact mandate and investment thesis Develop impact metrics Evaluate performance against impact mandate & investment thesis Effective communication of results “No, our approach is consistent across the cycle… We’re mainly just tracking the businesses that have been created.” Track and manage investment performance Develop measurement framework and tools Planning Impact For investment management Monitoring Impact To improve performance Overall Results Click on the coloured arrows to see examples of different approaches that can be used in each stage of the investment cycle! Dashboard

17. What IMM techniques are organisations using? We started by understanding whether study participants’ use of IMM techniques varies according to the investment cycle. This is because different techniques, whether standardised or custom, can be used for different objectives based on the stage of the investment cycle. However, only22% of survey respondents report that their approaches vary according to the investment cycle. • EVALUATING IMPACT • Expected Return: • Social return on investment (SROI) • Theory of Change: • Logic model • Quasi-experimental and Experimental Methods: • RCT • Historical baseline • Pre/post test • Regression discontinuity design • Difference in difference (DID) Evaluating Impact To prove social impact, exit investment and adjust investment thesis Forecasting Impact For investment selection and due diligence Establish impact mandate and investment thesis Develop impact metrics Evaluate performance against impact mandate & investment thesis Effective communication of results Track and manage investment performance Develop measurement framework and tools Planning Impact For investment management Monitoring Impact To improve performance Overall Results • Click on the coloured arrows to see examples of different approaches that can be used in each stage of the investment cycle! Dashboard

18. What IMM techniques are organisations using? We started by understanding whether study participants’ use of IMM techniques varies according to the investment cycle. This is because different techniques, whether standardised or custom, can be used for different objectives based on the stage of the investment cycle. However, only22% of survey respondents report that their approaches vary according to the investment cycle. • FORECASTING IMPACT • Expected Return: • SROI • Theory of Change: • Logic models • Quasi-experimental and Experimental Methods: • RCT • Historical baseline • Pre/post test • Regression discontinuity design • Difference in difference (DID) Evaluating Impact To prove social impact, exit investment and adjust investment thesis Forecasting Impact For investment selection and due diligence Establish impact mandate and investment thesis Develop impact metrics Evaluate performance against impact mandate & investment thesis Effective communication of results Track and manage investment performance Develop measurement framework and tools Planning Impact For investment management Monitoring Impact To improve performance Overall Results • Click on the coloured arrows to see examples of different approaches that can be used in each stage of the investment cycle! Dashboard

19. What IMM techniques are organisations using? We started by understanding whether study participants’ use of IMM techniques varies according to the investment cycle. This is because different techniques, whether standardised or custom, can be used for different objectives based on the stage of the investment cycle. However, only22% of survey respondents report that their approaches vary according to the investment cycle. • PLANNING IMPACT • Theory of Change: • Logic model • Mission Alignment Methods: • Social value criteria • Scorecards Evaluating Impact To prove social impact, exit investment and adjust investment thesis Forecasting Impact For investment selection and due diligence Establish impact mandate and investment thesis Develop impact metrics Evaluate performance against impact mandate & investment thesis Effective communication of results Track and manage investment performance Develop measurement framework and tools Planning Impact For investment management Monitoring Impact To improve performance Overall Results • Click on the coloured arrows to see examples of different approaches that can be used in each stage of the investment cycle! Dashboard

20. What IMM techniques are organisations using? We started by understanding whether study participants’ use of IMM techniques varies according to the investment cycle. This is because different techniques, whether standardised or custom, can be used for different objectives based on the stage of the investment cycle. However, only22% of survey respondents report that their approaches vary according to the investment cycle. • MONITORING IMPACT • Expected Return: • SROI • Mission Alignment Methods: • Social value criteria • Scorecards Evaluating impact To prove social impact, exit investment and adjust investment thesis Forecasting Impact For investment selection and due diligence Establish impact mandate and investment thesis Develop impact metrics Evaluate performance against impact mandate & investment thesis Effective communication of results Track and manage investment performance Develop measurement framework and tools Planning Impact For investment management Monitoring Impact To improve performance Overall Results Next Slide • Click on the coloured arrows to see examples of different approaches that can be used in each stage of the investment cycle! Dashboard

21. What IMM techniques are organisations using? Only 32% of survey respondents report that they align to formal impact measurement techniques These were the most commonly cited techniques from the survey and interviews: Overall Results Investees However, few of the study participants who identified these formal techniques were applying formal techniques in full. Instead, these participants noted that their organisation adopts standardised techniques where relevant, and then builds on these formal techniques with custom or in-house developed techniques. For example, 42% of survey respondents who reported to use standardised metrics indicated that they use a “combination of some already established metrics and internal metrics to account for unique product / services offered.” Interview participants explained that the benefit of combining standardised and custom techniques is that they are able to create a more structured IMM practice while accounting for adaption to context. Intermediaries n = 38 Investors Dashboard

22. What IMM techniques are organisations using? Only 31% of investee survey respondents report that they align to formal impact measurement techniques These were the most commonly cited techniques from the survey and interviews: Overall Results Investees While these techniques were most commonly cited across stakeholder types, there was limited convergence in techniques used by investees. Most commonly, investees were familiar with their funders’ reporting requirements, but did not identify or attribute those requirements to a specific technique. Intermediaries n = 16 Investors Dashboard

23. What IMM techniques are organisations using? Only 31% of intermediary survey respondents report that they align to formal impact measurement techniques These were the most commonly cited techniques from the survey and interviews: 1 Overall Results 1 Investees We are drawing on the DCED standard – it is so pragmatic. It provides a useful frame to think about our results chain and what we are doing… However, this is selective adoption of DCED where relevant. - Intermediary Intermediaries n = 8 Investors Dashboard

24. What IMM techniques are organisations using? Only 38% of investor survey respondents report that they align to formal impact measurement techniques These were the most commonly cited techniques from the survey and interviews: 2 3 4 Overall Results 2 3 4 Investees No investment will be considered without an ESG and impact assessment. - Investor IRIS is built into our database where there was natural alignment. But I find IRIS quite unwieldy and difficult to work with. - Investor The SDGs provide a good platform to promote impact and sustainability. - Investor Intermediaries n = 8 Next Slide Investors Dashboard

25. What IMM techniques are organisations using? However, 68% of survey respondents were not aligning to standardised techniques, but using in-house, custom IMM techniques. Similarly, the majority of interview participants were also using in-house techniques. While some interview participants indicated that their organisation has a clear theory of change and aligned metrics… …Many expressed that they do not have a clearly articulated theory of change, but that they are tracking internally developed metrics. Click on the bubbles to explore relevant resources! Dashboard

26. What IMM techniques are organisations using? However, 68% of survey respondents were not aligning to standardised techniques, but using in-house, custom IMM techniques. Similarly, the majority of interview participants were also using in-house techniques. While some interview participants indicated that their organisation has a clear theory of change and aligned metrics… …Many expressed that they do not have a clearly articulated theory of change, but that they are tracking internally developed metrics. Having an explicit theory of change is advantageous in that it encourages investees, intermediaries and investors to probe their impact thesis and to identify and understand potential challenges and underlying assumptions. Additionally, theories of change also enable organisations to select key metrics based on an understanding of the intended impact pathway, or interconnected chain of results. Ultimately, theories of change encourage organisations to be judicious in pursuing business activities and/or investments to ensure that these are in line with the impact that they aim to achieve. Click on the bubbles to explore relevant resources! Dashboard

27. What IMM techniques are organisations using? However, 68% of survey respondents were not aligning to standardised techniques, but using in-house, custom IMM techniques. Similarly, the majority of interview participants were also using in-house techniques While some interview participants indicated that their organisation has a clear theory of change and aligned metrics… …Many expressed that they do not have a clearly articulated theory of change, but that they are tracking internally developed metrics We observed through discussions that the levels of the theory of change are often used interchangeably, indicating that many study participants appeared unclear about the “impact chain,” cascading from outputs, to outcomes and impact. Lack of clarity and consistent understanding of these terms can be a challenge to the extent that it may inhibit meaningful engagement between stakeholders and interrogation of theories of change. In general, the metrics that organisations are tracking are at the output and outcome levels. Few organisations were tracking longer-term impact metrics. Next Slide Click on the bubbles to explore relevant resources! Dashboard

28. Benefits of practicing IMM Most survey respondents indicated that the top benefit of IMM is being able to demonstrate how their organisation makes a difference Overall Results Investees Intermediaries Investors Dashboard

29. Benefits of practicing IMM There was less variation among investees, with the top benefits of IMM related to demonstrating the difference made, achieving the organisational purpose, and improving policies and practices in the organisation Overall Results Investees Intermediaries Investors Dashboard

30. Benefits of practicing IMM Among intermediaries, the top benefits of IMM are related to demonstrating the difference made and achieving organisational purpose Overall Results Investees Intermediaries Investors Dashboard

31. Benefits of practicing IMM Among investors, the top benefit of IMM is related to demonstrating how their organisation makes a difference Overall Results Investees Intermediaries Next Slide Investors Dashboard

32. Challenges with practicing IMM Balancing measurement and reporting requirements when there are two or more funders/supporters Determining a clear impact definition and how to measure impact Challenges in data collection and reliability of data Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

33. Challenges with practicing IMM Balancing measurement and reporting requirements when there are two or more funders/supporters Determining a clear impact definition and how to measure impact Challenges in data collection and reliability of data As an organisation, we are still defining what impact means for us and what steps are needed in order for us to achieve impact. - Intermediary Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

34. Challenges with practicing IMM Balancing measurement and reporting requirements when there are two or more funders/supporters Determining a clear impact definition and how to measure impact Challenges in data collection and reliability of data Some data are very clean, and I assume people know how many people they employ, for instance, and what percentage of those people are female, so I think those data are fairly straightforward and clean…Some [clients] are really good at capturing data, some are really poor, so where does their information really come from? Is somebody just going with a calculator and spitting out some nice numbers? So, with a bit of caution on the reliability of the numbers. - Investor Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

35. Challenges with practicing IMM Balancing measurement and reporting requirements when there are two or more funders/supporters Determining a clear impact definition and how to measure impact Challenges in data collection and reliability of data It is challenging for start ups and educating funders on the potential impact of this type of support to informal businesses. Also, [it’s challenging] when we need to work on aligning to what the each of the funders are looking for. - Investee Next Slide Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

36. Using data and reporting results Overall, investors are the most important audience for IMM data and results Most Important 1 Investors 2 Customers Internal Team 3 4 Beneficiaries Board of directors 5 6 Executive Team and Senior Management General Public 7 Least Important 8 Individual Next Slide Dashboard

37. Using data and reporting results While investors are overall the most important consumer of data collected through IMM practices, results varied by stakeholder group Next Slide Dashboard

38. Future of IMM Partnerships and collaboration for capacity building Data-driven investment and decision-making 4 1 How to develop the landscape further? Better IMM resources Evolved, yet pragmatic approaches 3 2 Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

39. Future of IMM Partnerships and collaboration for capacity building Data-driven investment and decision-making Many large corporations consider strong IMM to be critical to unlocking funds and growing the sector. - Intermediary 4 1 They (grant makers) want to understand the impact that you are having on your community. It comes into every grant making applications, impact is probably 25% of decision-making for grant giving. - Investee How to develop the landscape further? Better IMM resources Evolved, yet pragmatic approaches 3 2 Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

40. Future of IMM Partnerships and collaboration for capacity building Data-driven investment and decision-making 4 1 How to develop the landscape further? My feedback is trying to make this as practical as possible. So, I think just getting more people who can learn how to do it themselves, even if it’s at a basic level, and then improving the standard of M&E. People who are part of the industry need to be thinking differently and not making it bigger, more complicated, and more expensive. - Investor Better IMM resources Evolved, yet pragmatic approaches 3 2 Focus on using IMM to do things better. Be pragmatic and relevant. - Intermediary Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

41. Future of IMM Partnerships and collaboration for capacity building Data-driven investment and decision-making 4 1 How to develop the landscape further? There is a lot of information. [We] need to make data readily available and understandable to the end receiver. It would also be helpful knowing what is out there as a base to start from. - Investee Better IMM resources Evolved, yet pragmatic approaches 3 2 It needs to be completely demystified. I don’t know what metrics we’re supposed to be using, what are we supposed to be doing? We need some guidelines. - Investor Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

42. Future of IMM Partnerships and collaboration for capacity building Data-driven investment and decision-making The last issue is how to have this conversation and move our investees forward on this topic. There is not a meaningful conversation behind why. Showing that to our investees and saying, “This is how we do it,” would help that conversation. Just like board level, “This is what we do. What do you do?” - Investor 4 1 How to develop the landscape further? Learning is important for us as well, to look at what other people have achieved, then emulate and learn from their lessons to try and involve this in other programmes as well. - Intermediary Better IMM resources Evolved, yet pragmatic approaches 3 2 Next Slide Click on the bubbles to see what study participants had to say about each of the themes! Dashboard

43. Resources Bertha Centre for Social Innovation and Entrepreneurship. Innovative Finance In Africa Review. Cape Town: Graduate School of Business UCT, 2016. IRIS Metrics, Impact Reporting and Investment Standards (IRIS), 2018, Available: https://iris.thegiin.org/metrics?search=employees (accessed April 01, 2018). BetterEvaluation, Randomised Controlled Trials, Available: https://www.betterevaluation.org/ar/node/1125 (accessed June 04, 2018). Jackson, Edward T, “Interrogating the theory of change: evaluating impact investing where it matters most,” Journal of Sustainable Finance & Investment, 3:2, (2018) 95-110. Available: https://www.tandfonline.com/doi/abs/10.1080/20430795.2013.776257. BetterEvaluation, Social Return on Investment, Available: https://www.betterevaluation.org/en/approach/SROI (accessed June 03, 2018). Khandker, Shahidur R., Koolwal, Gayatri B., and Samad, Hussain A., Handbook on Impact Evaluation: Quantitative Methods and Practices, The World Bank: Washington DC, 2010 Available: https://openknowledge.worldbank.org/handle/10986/2693 (accessed July 11, 2018). BetterEvaluation, Theory of Change, Available: https://www.betterevaluation.org/en/resources/guide/theory_of_change (accessed April 01, 2018). Reisman, Jane, and Olazabal, Veronica. Situating the Next Generation of Impact Measurement and Evaluation for Impact Investing. New York: The Rockefeller Foundation, 2016. Donor Committee for Enterprise Development (DCED), Measuring Results & the DCED Standard, Available: https://www.enterprise-development.org/measuring-results-the-dced-standard/ (accessed April 01, 2018). Skopos Impact Fund and Bridges Impact+. More than Measurement, A Practioner's Journey to Impact Management. Amsterdam: Skopos Impact Fund and Bridges Impact+, 2016. Global Impact Investing Network (GIIN). 2016. The Landscape for Impact Investing in Southern Africa. Global Impact Investing Network (GIIN). TONIIC. Insights from the Frontier of Impact Investing. San Francisco: JP Graphics, 2016. United Nations Department of Economic and Social Affairs, Sustainable Development Goals (SDGs) Indicators, 2019, Available: https://unstats.un.org/sdgs/indicators/indicators-list/ (accessed April 08, 2018). The GIIN. The State of Impact Measurement and Management Practice. First Edition. (2017). Available: https://thegiin.org/assets/2017_GIIN_IMM%20Survey_Web_Final.pdf (accessed November 22, 2018). Verrinder, Noel Blais, Zwane, Kagiso., Nixon, Debby. & Vaca, Sara, “Evaluative tools in impact investing: Three case studies on the use of theories of change,” African Evaluation Journal 6(2), (2018). Available: https://doi.org/10.4102/aej.v6i2.340 International Finance Corporation (IFC), Performance Standards, 2019, Available: https://iris.thegiin.org/metrics (accessed April 22, 2018). Links can only be accessed with a live internet connection Dashboard

44. Landscape Mapping for Impact Measurement and Management in South Africa