Monitoring and Evaluation Experience of Centralised Framework at Sector Level in Namibia's Education Ministry

1. Monitoring and EvaluationExperience of Centralised framework at sector level Mr B Khama Ministry of Education, Namibia CABRI Conference 2006 28-30 November 2006 ADDIS ABABA

2. Structure of presentation • Introduction • Description of Namibia’s PEMP and its application in education • Analysis of role of PEMP in education • Design, process and implementation • Development partners, sector strategic planning and PEMP • Way forward and lessons from Namibia

3. Introduction • Namibia case illustrates key issues regarding Monitoring and Evaluationframeworks between central oversight and control agencies and spending agencies • Where should objectives, actions, desired outputs and targets be set? • Balance between central oversight and sector knowledge • Desirability of policy coherence in government points to centralising efforts • Need to keep frameworks simple and comprehensible to limit overload at centre • Need to make framework comprehensive and useful for sector management • Important system objectives • Process • Data availability and integrity • Efficiency • Multiple frameworks not desirable • Donor requirements complicates

4. PEMP in Namibia • Centrally developed government-wide monitoring system • Aim is to link government’s operations to to National Development Plans • Sets out objectives and desired outcomes for ministries • Part of budget management system • MTEF system: Macro-fiscal framework and Ministerial MTPs • Critical analysis of past performance and MTPs informs thinking on resource allocation • Queries on PEMP data integral part of Budget Hearings and assessment • Forward analysis look at stretch/achievability of future targets; potentially redundant and overlapping spending; realism. • In-year monitoring • Future reforms include accountability reports, quarterly in-year monitoring, top level review and expenditure audits

5. Education sector and M&E • Sector overall in period of transition and significant reform • Drive to reform sector policies and management vs legacy thinking – “line items”, “needs”, “funding gap” ‘tyranny of targets” Producer capture and unwillingness to be accountable for results • Hierarchy and decision making within Education • Effects of decentralization • Initiative fatigue • Issues affecting M&E • EMIS late • Disjunct financial year/academic year • PEMP: Age + turnover = not ours

6. PEMP in Education: substance • 8 Objectives • All children receive quality education • All Namibians are functionally literate • Enhance learners’ potential for beneficial participation in the economy and in society • Increased employment • A workforce with the competencies to enable the labour market to operate efficiently • Well-educated citizens • Increased level of value-added research • Advancement of scientific knowledge • Plus indicators for administration • Total of 96 indicators…just for one ministry

7. Examples of indicators All Children receive quality education Problematic: Enrolment numbers, number of schools, quality of teaching Appropriate: enrolment rates, progression rates, survival rates, textbook ratio’s, teaching hours per learner, pass rates, drop out rates, PTRs All Namibians are functionally literate Problematic: Number of people passing literacy and numeracy tests Appropriate: Adult literacy rate, numeracy level, Enhance learners’ potential for beneficial participation in the economy and society Problematic: Employment rate for school leavers after X months; attendance levels; %lessons rated satisfactory by inspectors; Employment levels after graduation by subjects offered

8. Examples of indicators Increased employment Problematic: Relative employment levels of qualified and unqualified 6 months after completion; Unit cost of processing applications Appropriate: Youth unemployment (MoE target 2006/7, 80 000); Formal and informal employment levels (MoE target for 2006/7, 43 000) A work force with the competencies to enable the labour market to operate efficiently Problematic: Levels of skills shortage; 25% enrolling in VET; Levels of qualifications by field, length and year completed Appropriate: % of unemployed youth trained in SME promotion Well educated citizens Problematic: Unit cost of all qualifications; number of Namibians educated to first degree etc; internet users within the last month Appropriate: Attendance levels at tertiary institutions

9. Examples of indicators Increased level of value added research Leveraged funding ratio (GRN:Foreign); Total R&D spend in private sector; BUT ALSO: Number of research projects by sector Advancement of scientific knowledge Subject breakdown of citations (MoE target 2006/7 90 000) Unit cost of citations Administration Number of rapes by hostel BUT ALSO: Utilisation rate of school building; average age of outstanding payments; unit cost of catering and laundry per boarder

10. Does system work for Ministry? • Some positives • Existence of M&E framework used by both MoF and MoE positive • Some objectives, indicators are SMART • Specific, measurable, attributable, relevant and time-bound • GoN framework linked to NDPs • But other issues problematic for Ministry to use PEMP as core of performance framework • Issues of design, process and implementation • Multiplicity of stakeholders, processes and monitoring frameworks • Ministry own processes under-developed

11. Issues of design DESIGN OF OBJECTIVES • Overlap between objectives; at same time not all Ministry functions captured; at same time relationship between objective and MoE not straightforward or singular • Objectives and indicators not aligned clearly with key sector concerns, eg access, quality, equity, efficiency. DESIGN INDICATORS • Too many indicators: overlap between indicators • Relationship between objectives and indicators not always clear • Not clear what direction indicator should move • Where indicator come with set target, not clear on what basis target was chosen • Data problematic and expensive to source for many indicators • Indicators often use absolute numbers instead of relative numbers – indicator on own won’t indicate progress/regress • Many indicators can be manipulated

12. Process and implementation issues DEVELOPMENT PROCESS • Framework developed centrally without sufficient consultation with MOE • Efforts to adjust framework to be more in line with MoE strategic plans met with resistance • Although annual request for adjustments IMPLEMENTATION PROCESS • While on paper framework given prominence in budget process, in practice role not as prominent • Limited analytical capacity in MoF to do justice to information and perception of window dressing • System still bedding-in, yet to see full benefits. Targets and IFMS only from March 06 • Can Ministry’s budget be allocated against indicators outside of its control? • Framework not suitable for use in ministry to manage performance of sub-sectors: not all sub-sectors included • Data not available for many of indicators • Multiplicity of stakeholders

13. Multiplicity of stakeholders • DP attempts at imposition/override • Creation of separate processes and frameworks • JAR, ESPAG, ISCBF SC, ETSIP • Pareto 80:20 rule and DPs • ETSIP monitoring framework some sector ownership • Education and Training Sector Improvement Plan • Developed with intense consultations within the sector and with stakeholders over 2 year period • Based on thorough research on key challenges for sector • Comprehensive, focused, specific, resource-bound • Integrates operational and development spending against available internal and external resources • Approved by Cabinet and tabled in Parliament • Plan has Monitoring and Evaluation framework

14. ETSIP M&E framework • Main objective clear and attributable • To increase the immediate supply of middle-high level skilled labour to meet labour market demands and support overall national development goals • Indicators against main objective: • Objective and indicators SMART: Increased net enrolment ratios in Senior Secondary ; Increased proportion of students entering Grades 5,8 and 11; Significantly reduced drop-out rates; National average SACMEQ scores improved; Employment rates of VET students increased; Survey of employers verifies improved skill levels of graduates; Improved throughput to tertiary education with increasing emphasis on priority subjects • Clear, non-overlapping sub themes with detailed SMART indicators • 8 programmatic sub-themes with sub-objectives aligned to each • Limited number of indicators with how indicators will be used specified (process, output and outcome indicators) • Key performance issues of access, quality, equity, efficiency and HIV/AIDS addressed at objective level

15. Role of ETSIP framework • ETSIP framework driving management in sector • ETSIP framework used between Ministry and Development Partners and other funders • Negotiated framework with high MoE ownership • Relevant, specific, attributable objectives and indicators • Indicators aligned with existing data, but limited additional data collection being set up • M&E framework will be used to manage donor funds

16. Issues and way forward • Key issues for MoE: • MOE indicators to be agreed upon with MOE; • ETSIP framework linked to key additional resources, captures all Ministry activities at high level of output and outcome, linked to approved strategic plan • Urgent to reconcile two frameworks to make M&E activities streamlined, comprehensive, strategic, effective and efficient • New NDP development • Harmonize ETSIP and NDP 3 • Agree on PEMP indicators; • Resolving of these issues urgent

17. Conclusion • Lessons from a Namibian line ministry • Mutually agreed performance frameworks very important for principal agent contract between MoF and MoE • There is a need to design a M + E framework • Specific, measurable, attributable, relevant and time-bound • Criteria for design performance is whether it can be used by ministry itself • If that possible ties policy, budgeting and implementation together • Line-ministries understand sector policy issues and performance measurement better • Superior knowledge should be used in design process • But participation (effectively oversight) by centre important to ensure integrity of framework (stretch and achievement) • Frameworks and their functionality are symptomatic of institutional set-up of policy making and financing • Single framework needs to be agreed between centre, line ministry and external funders • First requirement for integration of activities in sector towards single set of strategic goals