GeSCI – MINEDUC, Rwanda Partnership Internal Review

GeSCI – MINEDUC, Rwanda Partnership Internal Review Team: Mary Hooker, Richard Niyonkuru, Esther Wachira, Evode Mukama and Denise Clarke February 2010 transforming education, empowering communities, promoting development

Terms of Reference - Internal Evaluation • The purpose – review GeSCI-MINEDUC partnership to understand: • Draft report responding to the critical questions • Extent of objectives achievement - capacity building, products and services • Extent in which MINEDUC priorities matched with the activities on the ground • Successes ,challenges and shortfalls • Key lessons emerging from the engagement • Partnership continuation? In what capacity ? • an extension of the existing country programme • a new programme, • a transition • provide support in specified thematic areas aligned to its focus areas and expertise • Engagement parameters including GeSCI’s role, strategy, personnel and other resource commitments and expected outcomes as well as the MINEDUC’s role and deliverables.

Context / Background • Strategic Documents – • Rwanda Vision 2020/ GeSCI Strategic Plan 2009/11 • GeSCI-MINEDUC, Rwanda Partnership MoU – • March 2007 – October 2008 • GeSCI-MINEDUC, Rwanda Partnership MoU – • November 2008 – March 2010 • Detailed Programme Plans 2008/ 2009 • Situational analysis report, Inception report, Trip reports, Activity reports - capacity building workshop s, research reports, programme visit reports

Internal Review Process / Methodology • An internal process of reviewing the partnership • serve to further inform a more formal evaluation (external) • focusing on evaluating learning and progress, and possible way forward • rather than a termination point

Internal Review Process / Methodology • Mixed method – quantiative & qualitative components • Consolidation and review documents through mail & PM tool • Participative approach to develop tools and questionnaire • Joint review team (GeSCI-MINEDUC) • In depth interview with GeSCI-MINEDUC-DP- P partnership constituencies • Methodologies • Content Analysis, Activity System Analysis, Most Significant Change • Technique • Limitations • Demanding schedule • Paucity of time • Data overload • Competing priorities

Deliverables • Initial findings to GeSCI Team Meeting • Focus group validation of Draft Report - internal review circulation of draft of report with MINEDUC/GeSCI officials and stakeholders for comments and inputs • Final report incorporating comments, extrapolation of lessons learnt, critical success factors, etc. to inform future GeSCI / MINEDUC partnership engagements and to be shared more widely through publication • Tool kit collating review methodologies and approaches for use in future programme reviews

Interviews with…

Interviews with… 8

Achievements / Emerging Success • Findings • Achievements: • Draft ICT in Education Policy , Costed Strategic Implementation Plan • Establishment of ICT in Education Department and OdEL Unit • Capacity building for systemic ICT integration in key components Teacher Professional Development /e-Content/ Infrastructure • E-Content evaluation framework / Training for Curriculum Developers in e-Content Evaluation/ Contextualization of digital content in Mathematics and Science/ • Streamlining multi-stakeholder partnership dialogue on ICT-Education • Strengthening coordination and capacity development (e.g. PME for OLPC) • Development of new variants of TCO for saturation models • Visits to other regions for knowledge sharing and partnership building • Raised awareness and understanding on critical components of infrastrucutre/ e-content / capacity development for ICT Integration in Education • Literature, knowledge sharing, mutual learning from different perspectives on I CT integration • Senior Managers sensitized and willing to advance ICT-Education agenda • ICT prioritization in strategic planning

Achievements / Emerging Success • Findings • Emerging Successes: • Adaptation of UNESCO ICT Competency Framework to ICT-TPD matrix • Adaptation and use of partnership training materials and methodologies by stakeholders in own work practices • South-south twinning for distance learning (e.g. KIE – East Africa-India Unviersity Network) • Collaboration and knowledge sharing through AKE • Development of EMIS package 10

Challenges • Disparities in understanding & knowledge about strategic parameters of partnership purpose, goals, strategy and objectives • Ambition of partnership programme too broad and comprehensive for the timeframe (21 months) of intensive engagement • Human resource limitations (GeSCI/MINEDUC) to address in-depth requirements of programme • Institutionalization of capacity development hindered by frequent turnover of staff • Research discourse disjointed

Shortfalls • Little output around maintenance and technical support • Adaptation of ICT-TPD matrix to local conditions and skills ladder and assessment components • Absence of vertical level in programme design for in-depth analysis of policy implementation as it translates into programmes • Lack of system framework & tools for monitoring & evaluation of horizontal and vertical impact of partnership activities • Research discourse disjointed and still in embryonic stage

Partnership Relevance “This Partnership was a perfect match” “The Partnership was built on a sensitive and critical domain to the country”

Partnership Relevance • To what extent did the partnership intervention purpose and overall • objectives respond to the needs and priorities of the different stake • holders at different levels (GeSCI/MINEDUC)? • Parallelism between Rwanda and GeSCI Vision and Mission for building Knowledge and Information-based Economies and Societies • Historical mandate of GESCI’s role as neutral facilitator to governments in development on ICT Education • Partnership relevance in Rwanda enhanced by political environment and top level championship of ICT in Education & Development • Second Partnership MoU developed from Inception Report focused on priorities and objectives more aligned to stakeholders needs for skills development & education relevance • Partnership responsiveness to dynamic environment for ICT deployment – strategic advice on partner coordination , capacity building and quality assurance • Common stakeholder understanding of parntership mandate through engagement with partnership programme – workshops – multi-stakeholder consultations – joint planning

Partnership Relevance “When we see GeSCI , we see Senthil” “We are lucky to have Senthil – he is available at any hour and at any time – the Ministry has benefited a lot” “We wish we could get more Senthils – and have him in many places ”

Partnership Relevance • GeSCI only institutional advisor bringing dedicated ICT expert physical presence into the ministry • CFP a critical factor to partnership success: • affinity with the Rwanda ICT-Education needs • close relationship with stakeholders & institutions • building understanding and trust • attributes of sensitivity and diplomacy / social-awareness/ cultural understanding/ hard-working/ passionate/ fully integrated “

Partnership Relevance • And when you see the partnership... • ...who do you see? • ...what do you see?

Partnership Relevance • Tensions: • Disparities in understanding & knowledge about strategic parameters of partnership mission & goals due to inadequate communication/ sensitization strategy • Partnership dynamic & literature limited to national sphere of influence and stakeholders working closely with partnership in MINEDUC • Partnership acceptance by stakeholders embedded horizontally in national institutions and agencies while fragmented vertically in outer layers of stakeholder & beneficiary levels • Partners unclear on strategic advise role of CPF and seeming deviation to tasks not directly related to advisory function (supervision EMIS) • Lack of clarity on how stakeholders should work with partnership • Attribution of ICT in Education awareness to work of partnership problematic in environment of multiple contributory factors (national vision & commitment, other national & regional ICT partner networks & initiatives)

Partnership Relevance • Tensions: • First MoU understood but too broad and lacking in clarity • Misconception of GeSCI role - perceived as implementer - expectations for funding support as well as strategic advice • Initial phase underutilization of partnership expertize • Ministry officials not conversant with role of ICT in Education or need for sector specific policy • Inadequate baseline for understanding of Rwandese educational context • Partnership products at times at variance with local conditions and context • Partnership limitations to support ‘on the move’ dynamic agenda in a timely fashion

Partnership Effectiveness “It is evident that something positive has happened as a result of the partnership” “What has made things work is collaboration” “Everybody had his or her voice heard”

Partnership Effectiveness • To what extent has the GeSCI-MINEDUC partnership achieved its intended objectives and how did the deliverables contribute to enhance the quality of education, empower communities and accelerate socio-economic development? • Completed drafts of ICT Policy & costed Strategic Plan • New structure of ICT Education Department & OdEL expanding technology use profile beyond conception of IT support • Raised awareness on system wide requirements for ICT integration based on policy / strategy / planning for critical components of teacher training, e-content development and infrastructure • Innovative digital content contextualization building understanding on concepts - evaluation, contextualization, adaptation, integration and utilization – a phenomenal achievement in view of capacity gap in this area • First African country to utilize UNESCO ICT Competency Framework for Teachers to develop ICT-TPD matrix • Multi-stakeholder consultation and integration in iterative cycles for ICT policy, strategy & project planning development • Strengthened capacity in MINEDUCto articulate on policy & strategy and to influence ICT-Education uptake in Departments and Agencies • Concretization of theoretical vision for ICT in Education use

Partnership Effectiveness • Tensions: • Elements of partnership programme too ambitious and broad • Frequent change over of leadership and key directors interrupted momentum with resultant lack of institutionalization of capacity development • Bureaucratic bottlenecks & delayed approval hindered operationalization & institutionalization • Competing education priorities put on hold ICT agenda in spite of its key position as 1st pillar in Government flagship for development • Plans and strategy effects not yet realized at the beneficiary level (vertical gap in partnership engagement) • Absence of guidelines and training for pedagogical integration of e-content • Teacher resistance to e-content use in practice due to perceived curriculum incompatibility • Prohibitive costs of e-content contextualization limiting module production • Models (saturation v lab) for ICT not tested/ school e-readiness not enabling for deployment roll-out / insufficient time and scope for school based experimentation “”

Partnership Efficiency “The partnership was cost effective … …given it’s short period of actual operation and limited resources ”

Partnership Efficiency • Have the available means been optimally exploited? Could the partnership have produced the same products and the proposals (outputs) with fewer means including time spent? Could the partnership have produced more with the same means? • Key elements of partnership MOU outputs positioned • Good logistical support, team effort, strategic setting in MinEDUC, enabling political environment, timely access to research & information accelerated achievement of ambitious programme objectives • Given constraints of staff limitations and under-financing , investment has been effective in terms of emerging policy, planning and understanding

Partnership: Efficiency • Tensions: • First MOU succeeded by period of inertia with activities commencing only half way through • Potential partnership expertise under-utilized • MINEDUC human resource capacity for timely decision-making and delivery • Stakeholder latency • Human and financial resource limitations for programme ambition of second MoU • Support by specialists good but not sufficient for planned activities • Expertize model requires adjustment for longer in-country technical support • Insufficient time for adequate analysis of product utilization “”

Partnership Impact “GeSCI has done well in what has been a forest of partner activities embracing ICT without proper planning & costing” “The change has been the systematic approach... access to models that are out there... access to expertize and knowledge”

Partnership Impact • What wider effects have been caused by the partnership • intervention in the education sector / society? • Partnership put in place frameworks, systems and structures to respond to national objectives for technology integration in Education & Development • Shift in thinking process from ad-hoc and component based approach to comprehensive / system wide approach for ICT in Education • Systemic and systematic ICT planning approaches being adopted by MINEDUC - with improved responsiveness in terms of timeliness & efficiency • ICT in Education policy, Strategic Implementation Plan, ESSP developed in integrated manner for seamless linkage to National Vision & Planning for Development (Vision 2020, NICI Plan, EDPRS, LTSFF) • Increased funding allocation for ICT in Education from sector plan and development partners • Environment of trust creating strong multi-stakeholder partnerships and collaboration • Emerging dialogue and discourse on educational reform and transformation • Emerging literature from Rwanda practice disseminated internationally • Enhancement of Rwanda’s image for educational progression and technological innovation

Partnership Impact Significant Change “Current understanding of ICT in Education has moved beyond perception of putting technology in schools to a more holistic understanding of a system wide approach to make ICT integration a development path towards educational transformation”

Partnership Impact • Tensions • Quality assurance: • Lack of system framework and tools for monitoring & evaluation of horizontal and vertical impact of partnership activities • Institutional capacity deficit for assessment of partnership product (e-content) utilization in practice • Lack of joint programme accountability for tracking partnership programme influence

Partnership Sustainability “The Partnership has been a good experience … …a lot of learning for the Ministry”

Partnership Sustainability • Is it likely the partnership positive effects can continue after dis-engagement at system level? • Bits and piecesof ICT in Education initiatives synchronized • Significant project management capacity building carried out with MINEDUC team • EMIS package developed for effective planning and management • ICT in Education department established to coordinate programmes • Increased capacity among the MINEDUC and its implementing agencies • Increased resource allocation both from sector plan and development partners • Partner enablement to complement ICT in Education initiatives • M&E work is progress • GeSCI to continue providing strategic advice and capacity building on a needs based basis • Trust established with MINEDUC to approach us for quality strategic advice.

Partnership Sustainability “The production of a document has a finality” “If I need more support , where will I go?”

Partnership Sustainability • Tensions : • Bits and pieces are on paper – haven’t been developed – awaiting operationalization • Partnership activity at embryonic stage • Operationalization and staffing of ICT Unit not yet realized • ICT Task Force and Steering Committee organized on ad hoc basis • Understaffing, contract staffing and high staff turnover detrimental to institutionalization of capacity built • Technical and financial capacity of institutions to use partnership products limited • Partnership reliance on training with few measures for follow-up support • E-content project momentum weakened by gaps in understanding and ownership • Lack of a counterpart can undermine benefits of partnership work in a new phase • Multi-stakeholder collaboration and complementarity to partnership programme agenda requires framework for institutionalization • Research discourse disjointed

Lessons Learned “Partnership should focus on 10 objectives …not 21 …and should prioritize 5”

Lessons Learned • Partnership focus was too broad. Resource and time frame limitations for intensive country engagement require a more realistic objective set definition. • Programme comes down to individuals and inter-personnel skills . The CPF and Specialists are key success factors in any programme. There is at the same time a risk factor in dependency on CPF skills. • More stakeholder involvement in programme conceptualization, analysing ICT Education trends, planning and bringing resources to the table will help to maintain sustainability • Working in an integrated manner, being flexible, continuous learning and responsiveness to partners priorities is key to a partnership approach for mutual learning and development. • Without commitment from both partners to joint accountability and systematic programme review - learning, ownership and continuity of partnership activity can be compromised .

Recommendations / Way forward • Development of Jointly Agreed Workplans (LFA/OM) as appendix to MoU • to include components for joint programme design, accountability and rolling review (inception, quarterly, mid-term, final) • to identify partnership counterpart (individual?/team?) who is going to be engaging on a daily basis with the Ministry and who is accountable to the Minister for tracking progress indicators in partnership action plan

Thank you… Mary, Esther, Richard, Evode and Denise

GeSCI – MINEDUC, Rwanda Partnership Internal Review