1 / 27

Improving Education Management in Madagascar: Results and Implications of an Impact Evaluation

Improving Education Management in Madagascar: Results and Implications of an Impact Evaluation. Gérard Lassibille, Jee-Peng Tan, Cornelia Jesse and Team AGEMAD Africa Program for Education Impact Evaluation (APEIE) Second Workshop, Dakar, Senegal Dec 17-19, 2008. Outlline. Country context

tkristin
Download Presentation

Improving Education Management in Madagascar: Results and Implications of an Impact Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Education Management in Madagascar: Results and Implications of an Impact Evaluation Gérard Lassibille, Jee-Peng Tan, Cornelia Jesse and Team AGEMAD Africa Program for Education Impact Evaluation (APEIE) Second Workshop, Dakar, Senegal Dec 17-19, 2008

  2. Outlline • Country context • Current management processes • AGEMAD tools and procedures • Design of the experiment • Implementing the impact evaluation • Key results and implications • Scaling up and institutionalization

  3. Country Context • Education viewed as central to government reforms to accelerate growth and reduce poverty • Key recent measures in primary education include: • Restructuring of the primary and secondary cycles of schooling • Introduction of new pedagogical approaches • Reduction of grade repetition • Distribution of school kits to all primary school children • Increased hiring of contractual teachers • Use of school grants to improve service delivery and performance • AGEMAD set in the context of these reforms: • To improve understanding of management issues in education • To support the ongoing and future reforms

  4. Evidence of progress in primary education: • Increase in primary education’s share of total spending on education: 59% in 2006, up from 38% in 2002; • 4 million in school in 2006, up from 1.7 million in 1996 • 59,000 teachers in 2006-7, about twice the number 10 years ago

  5. Yet, many problems and challenges remain … • Evidence of weak performance: • ½ of each cohort of 1st graders don’t finish the primary cycle; • Repetition rate still high at 18% in 2005 (30% in 2000) • Low test scores: in 2004-5 PASEC, average score of 50% in Maths and Malagasy and 32% in French; deteriorated since 1997-98 • More than ½ of teachers are contractual teachers; poorly trained and poorly paid • Multiple causes of weak sector performance: • Some beyond control of the sector • Some depend on decisions within the sector, including acting to reduce: • Inconsistencies in teacher allocation across schools; • Poor utillization of instructional time; • Ineffective management of the pedagogical processes at the school and classroom levels.

  6. Current Administrative Structure

  7. Evidence of Weak Management …. • Many aspects of the pedagogical processes are poorly managed and tasks essential for student learning are neglected • Poor monitoring of pupil absenteeism: • Only 13 days a month, on average, monitored by teachers • Absences not supervised by school directors: • 10% never monitor them • 1/3 of attendance records not signed off by directors • Not a topic of discussion with teachers in 80% of cases

  8. Evidence of weak management (contd.)… • Neglect of basic pedagogical tasks: • Among teachers: • 20% don’t prepare daily lesson plans • Only 15% consistently prepare daily and bi-weekly lesson plans • Among school heads: • 1/3 never discuss with teachers their daily lesson plans • ½ fail to sign off on their teachers’ lesson plans • 70% never sign off on teachers’ daily lesson plans

  9. Evidence of weak management (contd)….. • Poor communication on student learning: • Results of tests and quizes are poorly recorded, if at all; • 25% of teachers don’t prepare individual student report cards; • Communication from teachers to parents is often perfunctory; • Pupil absenses is poorly communicated to parents. • School directors hardly involved in following-up on student performance: • 3/4 don’t discuss issues with learning outcomes with their teachers; • only 20% sign-off on test results and student report cards)

  10. Evidence of weak management (contd.)….. • Inadequate supervision of teacher absences: • Absenteeism rate of nearly 10% • Only 8 % of school directors follow teacher absences closely • taking daily attendance; • monitoring and posting a monthly summary of absences • More than 80% of directors fail to report teacher absences to administrators at the sub-district and district levels

  11. AGEMAD tools and procedures • Objective • Strengthen management of service delivery in primary education • How? • Modifying behavior of key actors by inserting supervision and follow-up at key points in the adminstrative hierarchy • Making explicit to the actors their responsiblities and supporting them with the tools to accomplish their tasks

  12. AGEMAD tools and procedures • More than 30 tools developed for tasks considered essential for a well-functioning system: • 7 for teachers • 8 for school directors • 8 for sub-district administrators • 9 for district administrators • Tools targeted following areas: • Pedagogy • Student learning and follow-up • Management of instructional time • Administration • School statistics • Partnership with the local community

  13. AGEMAD tools and procedures • AGEMAD tools and procedures: • Most tools are not new • Should be used by each actor at specific times during the year • School report cards for school directors, sub-district and district levels officers: • Complement the AGEMAD tools and procedures • Draw attention to schooling outcomes • Include comparative data, allowing a school to compare its outcomes with those of other schools • Serve as basis for dialogue and accountability

  14. Setting up the experiment • Objective: Evaluate impact of the AGEMAD interventions on service delivery and schooling outcomes • Experiment seeks to answer three key questions: • Do AGEMAD interventions work? • Which particular AGEMAD package worked? • Were there spillover effects? • Experiment will run for two school years, 2005-6 to 2006-7 • Support from MIT Poverty Lab for experimental design

  15. Implementing the experiment • Report cards of schools, sub-districts and districts: • Developed by a team in the EMIS section of the Ministry of Education. A total of 10,000 report cards were produced. • Tools, operational guides and training: • 200,000 tools et 11,000 guides produced and distributed • Training provided to nearly 4,000 participants • 1st year: 4-day course for the sub-distrcit and district officiers; 2-day course for teachers and school directors; • 2nd year: 2-day course for the sub-district and district officiers; 1-day course for the school directors and their teachers; • Training for teachers and school directors provided by sub-district officers • School meetings: • Over the two years, 1,500 meetings tool place with support for organization and facilitation provided by specially recruited, to discuss elaboration of school improvement programs

  16. Implementing the experiment • Implementation team: • Stable team of about 15 MOE staff • AIDE et ACTION • AFD (2 technical assistants on site) • World Bank • Support from more than 50 persons • Financing from the WB, AFD, MOE, Ireland, and the EFA-FTI EPDF

  17. Collecting the data • School surveys: • Impromptu school surveys (1,200 schools and 4,000 teachers) • Survey of the sub-district and district administrators • Artifacts of service providers’ work: • Administrative and pedagogical tools used by teachers and school heads (at the end of the experiment, in 40 schools and 850 artifacts). • Achievement tests (incl. pupil surveys): • Year 1: tested about 25,000 pupils in Grade 3; • Tear 2: tested about 22,000 pupils in same cohort (i.e., Grade 4) • Maximum of 25 pupils per school • Administrative data: • Pass rate on the end-of-cycle examination.

  18. Analytical approach • Two types of AGEMAD impact expected: • Direct impact on the behavior of the actors • Indirect impact on student achievement • Evaluating the indirect impact is easy: simply compare across the treatment and control groups • Evaluating the indirect impacts presents more difficulty: • The simplest (or simplistic): compare task by task across across the treatment and control groups • Complication 1: multiple actors; focused on school personnel (director and teachers) • Complication 2: multiple tasks per actor; focused on tasks considered essential by Malagasy educators

  19. Analytical approach • Seven essential tasks: • Teacher: • Takes daily roll call • Prepares daily lesson plan • Makes two-monthly plans of lessons • Monitors student learning • Tested pupils during the past two months • Helps lagging students • Discusses student learning issues with the director • School director: • Keeps a register of enrollments • Signs off on daily roll call • Analyzes student absences on a monthly or bi-monthly basis • Reviews pupils’ test results • Takes stock of teacher absences • Informs sub-district or district officer about teacher absences • Follows up with teachers on lesson planning

  20. Analytical approach AGEMAD interventions seeks to influence key actors to discharge their key responsibilities: Definitions adopted for the analysis: • A good teacher performs all 7 essential tasks in TORs • A good director executes all 7 essential tasks in TORs • In a well managed school all the staff execute all their essential responsibilities

  21. Results—Impact on Provider Behavior

  22. Results: Impact on schooling outcomes

  23. Implications of the results • Who to target? • Prioritize school-level actors • “Cascade” model alone, as currently defined, doesn’t work • What’s in the intervention package? • Materials in the form of tools and guides; • Training actors in using the tools and guides; • On-site support from facilitators; and • Community-school meetings around school report cards, as basis for school improvement plans • What are the costs? • Cost of intervention per school high during impact evaluation • Costs during impact evaulation NOT necessarily cost to scale up

  24. Implications for scaling up and instutionalization • Mainstream IE results into MoE activities • Need a champion from the start • Need early involvement of a national team, with good technical support • Necessary to sustain change in actors’ attitudes & behaviors • Use existing structures and mechanisms for scale up: • Tools, guides and training modules integrated into teacher training • Tool distribution, training and facilitated school meetings funded through the local catalytic funds based on regional, district and school performance plans and needs • Develop leaders to drive change in management practices • Discussion underway on collaboration in leadership training between Madagascar MoE and partner organization in another country

More Related