Moving from pilot to scale in education: What does it take?

1. Moving from pilot to scale in education: What does it take? 2017 ICT4D Conference, Hyderabad, India Sarah Pouezevara - @spouez

2. About this presentation • Using prior analysis of scale up in basic education programs, how do we apply the lessons and frameworks to ICT-based scale up? • What steps or pre-conditions are similar? What needs to be approached differently? • What have been your experiences? ? Question: Do the terms “scale up” and “sustainability” mean no change is possible? RTI International - @spouez

3. Ingredients for national reading programs • Assuring basic inputs (curriculum, materials) • Improving instruction (pedagogy, training, support) • Monitoring and assessment (monitoring instruction, outcomes and routinized accountability) • Communication (mobilize enthusiasm and create demand) ? Are there common—or added—technical ingredients that need to be aligned for ICT innovations? RTI International - @spouez

4. Ways to think about scale (and sustainability) • Expansion: Originator of an intervention implements it in an increasing number of schools/places • Replication: Originator of an intervention partners and networks with other organizations/entities (including Ministry of Education) to implement it in an increasing number of schools/places. • Collaboration: Originator of an intervention uses the pilot experience to assist others to implement, including to inform government policy and practice so that elements of the innovation are taken up by the education system and implemented at scale ? How can programs employ a combination of these approaches to get to achieve scale and sustainability? RTI International - @spouez

5. “Ingredients” for successful scale-up (Brookings and MSI) Brookings and MSI Scale Up Frameworks

6. Tusome National Tablet Program for Instructional Coaches 2014 Pilot 150 Users and cost effectiveness study 2015 Nat’l scale up 1200+ users 2016 Expansion 2000 users 2017 Enhancements User-centered design Outputs Data at county and national level

7. Keys to Tusome’s success in scaling up • Used a combination of the three approaches (starting small, encouraging others to adopt the model, and working through government). • Uses the pilot experience to inform government policy and practice so that elements of the innovation are taken up by the education system and implemented at scale. • Worked alongside government at all stages. RTI International - @spouez

8. MobiLiteracy Uganda – Urban Planet and RTI 2013 Pilot 150 Users and cost effectiveness study Baseline and endline skills measurement EGRA methodology Plan for scale Telecom partnership Results Effective method but not scaled

9. Lessons from MobiLiteracy’s pilot • Measured comparative approaches: technology vs. paper vs. oral only. • Scale was planned through partnership with local telecoms, but there is no evidence that this happened. • Did not work alongside government at any stage: example of innovation and scale operating outside of formal system • Lack of ‘champion’ to help create demand? Insufficient investment in partnership strategy? A = mobile B = paper C = oral RTI International - @spouez

10. Emerging conclusions • Scale up needs to be based on data about what works (defining the ‘ingredients for success’) • Innovations must be designed with scale in mind. • The interplay between scale and sustainability is not just in financial terms—it is also about in-country capacities needed to support large scale change. • Long-term sustainability requires demand for the innovation, including mobilizing champions, leaders, and constituencies and alliances to support the innovation. • Most innovations will be altered as they go to scale and they will also alter existing arrangements in the education system; flexibility to respond to changing contexts is a good thing provided the minimum ingredients are still present. • Pilots need to be designed with local institutions—including public sector—to build necessary capacity during the process and address alignment/coherence issues (sending the right “signals”) RTI International - @spouez

11. Some ways that technology can be applied • Filling gaps by unburdening teachers (e.g., home access to learning) • Expanding opportunity (e.g., personalized learning, more feedback, anytime/anywhere learning) • Keep costs low without sacrificing quality (economies of scale, automating processes, generating efficiencies like mobile money) • Accountability and data for monitoring and evaluation • Data gathering for impact evaluation • Mobilizing demand and communicating successes, elevating the role of teachers • Overcome a context-specific barrier (e.g., long distances, difficult transportation) RTI International - @spouez

12. Checklist for evaluating scalability From: Cooley and Kohl (2006) • Credibility - is there sound evidence demonstrating impact, and is that evidence espoused by recognized leaders • Observability - teachers, principals, officials, and parents must be able to see tangible results • Relevance - does the innovation respond to a recognized need (e.g., as a response to poor EGRA results) • Comparative/Relative Advantage - does it show a cost-effective alternative to existing practice • Ease of transfer and adoption - it cannot be too far of a leap for the average teacher • Compatibility - it cannot be completely incompatible with existing norms, values, and arrangements (e.g., it cannot require a complete reimagining and reorganization of the education system) • Testability - can potential users try it out safely without risking too much RTI International - @spouez

13. Discussion • Question: Do the terms “scale up” and “sustainability” mean no change is possible? • Question: How to we employ a combination of approaches to planning for scale up? • Question: Are there common—or added—technical ingredients that need to be aligned for ICT innovations? • Question: What else do you consider a critical point on the ‘checklist’ for ICT4E innovations? “Scalability is demonstrated by (a) the ability to replicate the key elements of a project that were deemed critical to its effectiveness at scale (regional, national) and (b) the project's affordability over the mid and long-term, given a country’s total projected resource envelope.”- USAID, 2011 RTI International - @spouez

14. References Cooley, L., and Kohl, R. (2006). Scaling Up- From vision to large-scale change: A management framework for practitioners. Washington, DC: Management Systems International. Perlman Robinson, J., Winthrop, R., & McGiveny, E. (2016). Millions learning: Scaling up quality education in developing countries. Washington, DC: Brookings Institute. Piper, B., Zuilkowski, S., Kwayumba, N. Strigel, C.(2016). Does technology improve reading outcomes? Comparing the effectiveness and cost-effectiveness of ICT interventions for early grade reading in Kenya. International Journal of Educational Development (49): 204-214 DeStefano, J. and Healey, H. (2016). Scale up of early grade reading programs. Prepared for USAID's Education Data for Decision Making Project II (Ed Data II), Task Order 15. Washington, DC: RTI International. US Agency for International Development (USAID). (2011). Education Strategy Implementation Guidance. Washington, DC: USAID, p. 6. RTI International - @spouez

15. More Information Name Sarah Pouezevara @spouez spouez@rti.org For copies of presentation and other RTI-authored documentation on ICT in education, scale and sustainability, see: http://shared.rti.org http://tinyurl.com/m3eolte