Using Outcome Mapping as a Research Methodology for Information Communication Technology Projects

1. Using Outcome Mapping as a Research Methodology for Information Communication Technology Projects Chris Morris African Advanced Institute for Information & Communications Technology (AAIICT)

2. Comparative study of “first mile” and “first inch” technology in different low density contexts. Project funded by the IDRC

3. Angola UCM (Pemba) Angonet (Huambo) Mozambique ISPU (Quelimane) Wireless (Harare) Zimbabwe SchoolNet Mozambique (Inhambane) Power Line Comms (Rooiwal) MICTI (Maputo) CSIR HLT (Openphone) Wireless Mesh (Peebles Valley) South Africa CSIR Telehealth (Eastern Cape) UWC MuTi (Eastern Cape) Translate.org (HTML Pub) FMFI Projects & Partners

4. Objectives The objectives of this project are: • To develop innovative ICT technologies and to implement “first mile” solutions • To develop business models for first mile solutions • To research behaviour in the use of ICTs – how the use of ICTs has changed community life • Scalability and replicability of technologies – the viability of rolling out the solution • Influence on policy and regulation – demonstration of project benefits to the regulator • To publish a reference book for “first mile” and “first inch” implementation in rural Africa.

5. Research Question? • What is the sector you want to contribute to through your project? How does it link to the overall vision of the First Mile/First Inch Network? • What is it that you want to make known through your project? (What is the research question?) Why? • Who are you going to influence through your project? • What kinds of influence are you intending to have on your boundary partners’ behaviours? • What are the main methods by which you are going to answer your research question? Why did you select these methods? (relates to issue of validity) • What are the main strategies you are going to use to influence your boundary partners? Why did you select these strategies? (relates to issue of relevance) • How would you propose to monitor and report on the project process as it unfolds?

6. Choice? • research methodology that was simple to apply • robust in being able to provide an implementation framework but flexible enough to adapt • we needed to be participatory with our project partners • to understand the development process and learn more about human behaviour and how that influences technology choice • we are a learning organisation, we wanted to look at the change taking place, not simply make a claim to it. • a tool that helped us document the journey & understand the processes - what worked and what did not work and why • one that would develop a space for social learning and creative dialogue

7. Choice contd. ? • we needed to be able to reflect and review our plans and make changes when necessary. • at the end of the project we needed to know where we have been and what were the lessons learned. • we needed to know how to improve next time. • to be accountable to our stakeholders and donors. • we needed to undertake comparative research across projects, across countries. • we needed a dissemination strategy - identify Primary Intended Users and Primary Intended Uses of research results.

8. Why Outcome Mapping? Outcome Mapping methodology for planning, monitoring, and evaluation in order to document, learn, and report results • Development is accomplished by, and for, people • Focuses on outcomes as behavioural change • Not impact but contributions to outcomes • Boundary partners control change

9. Outcomes Mapping • OM provides a method for development programmes to plan for and assess the capacities that they are helping to build in the people, groups, and organizations that will ultimately be responsible for improving the well-being of their communities. • Outcomes are defined as changes in the behaviour, relationships, activities, or actions of the people, groups, and organizations with whom a program works directly • Boundary partners are those individuals, groups, and organizations with whom the program interacts directly and with whom the program anticipates opportunities for influence

10. Outcome Mapping…Cont. Outcome Mapping asks the following questions : • Why? (What is the vision to which the program wants to contribute?); • Who? (Who are the program's boundary partners?); • What? (What are the changes that are being sought?); and • How? (How will the program contribute to the change process?).

11. Why? • VISION • This inspired ambitious action and commitment, - but was not an yardstick by which the project would be measured

12. Who? - Boundary Partner Map

13. Outcome Challenges

14. What? - Progress markers Outcomes have been assigned to each boundary partner & progress markers are used to monitor behavioural changes of boundary partners in achieving the desired outcome. The Progress Markers for the Outcome Challenges are categorised according to the current status where: • Expect to see = Behaviours the project expects to see • Like to see = Behaviours the project would like to see • Love to see = The behaviours associated with achieving the best outcome

15. Progress Marker: Clinic Sister

16. How? - Strategy Map

17. OM 3 STAGES • Intentional Design • Why? Who? What? How? • Outcome & Performance Monitoring • Outcome journals • Strategy journals • Performance journal • Evaluation Planning

18. Organisational Practices • Prospecting for new ideas • Opportunities and resources • Seeking feedback from key informants • Obtaining support from your next highest power • Assessing and redesigning products, services, systems and procedures • Checking on those already served to add value • Sharing your best wisdom in the world • Experimenting to remain innovative • Engaging in organisational reflection

19. Evaluation • Assess contribution to development outcomes • Understand linkages between behavioural changes and strategies • User uptake, benefits of ICTs • Influence on regulatory policy • Documentation of lessons learned • Understand intended uses and intended users • By using OM as a framework it allows comparison across cases, across countries

20. Dissemination Strategy PI Users PI Uses Influence on regulatory policy : Department of Communication • Create license exemption for use of wifi in research • Create license exmption for use of wifi for non-commercial use e.g. health & education • Dissemination strategy – inner circle, LINK, DoC, NGOs, University Influence on health policy : Department of Health Eastern Cape • Support for the use of wifi for telehealth in rural clinics • Demonstrate value of eHealth training module • Focus group key DoH decision makers,Provincial Head, Supply Chain Manager

21. Conclusion • OM as a research tool enabled us to answer the research questions – what is it that you want to make known that was not known before? The research produced primary data in discovering something new. OM provided the route map and data gathering tools to produce evidence based results. The evaluation of these research results enabled us to answer the research question user uptake, influence on policy • OM provided a methodology for the comparative analysis between projects, between countries • OM provided the framework for planning, documenting the journey, and capturing the lessons learned during the life of the project. It helped the project team document “where it had been”, understand the development process, social learning & creative dialogue • OM enabled the project team to review progress and adjust strategies when necessary recognising development does not have fixed goals but is a moving target. • OM provides a framework for understanding who are the boundary partners with whom the project seeks to influence and can influence, what are the uses and who are the intended users of the research results.

22. Thank You Chris Morris ICTs for Development Tel: +27 (12) 841-2509 Cell: 082 892 9349 E-mail: cmorris@csir.co.za URL: http://www.csir.co.za URL: http://www.fmfi.org.za