Southern and Eastern Africa Consortium for Monitoring Educational Quality

1. SACMEQ Southern and Eastern Africa Consortium for Monitoring Educational Quality

2. Goal 6 Dakar Final Framework “Improving all aspects of the quality of education and ensuring excellence for all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy, and life skills.”

3. Mission Undertake integrated research and training activities that will: • Expand opportunities for educational planners to gain the technical skills required to monitor the quality of basic education • Generate information that can be used by decision-makers to plan the quality of education

4. Botswana Kenya Lesotho Malawi Mauritius Mozambique Namibia Seychelles South Africa Swaziland Tanzania (Mainland) Tanzania (Zanzibar) Uganda Zambia Zimbabwe Membership of SACMEQ

5. Working Style • Highest priority: • "learning by doing" approach • Institutional capacity building • African collaboration • Self-help approach • South-south cooperation • Not a research project -- but educational planners involved in research • Involvement of Ministers of Education

6.   IIEP IIEP IIEP SCC Phase 1 (1991) Research (IIEP = initiator) Phase 2 (1992-1994) Training (IIEP = facilitator) Phase 3 (since 1995) Capacity Building (IIEP = external friend) Evolution of SACMEQ Working Style

7. Working Style – Cost Sharing • International Cost • By SCC (through UNESCO-IIEP budget and/or external budget) • For operation of SCC, sub-regional training, provision of software, standardization of materials • National Cost • By each Ministry (through internal budget or bi-lateral arrangements) • For within-country training, instrument reproduction, data collection, report dissemination, purchase of materials, etc.

8. Management & Finances • SACMEQ Assembly of Ministers • responsible for general policy and training/research directions • SACMEQ Co-ordinating Centre • responsible for (a) working with partners to obtain funding for cooperative sub-regional activities, and (b) providing technical support • SACMEQ Management Committee • responsible for providing administrative/financial guidance to the Sub-regional Coordinating Centre

9. Pilot 1994 Main 1995-1998 Seven Ministries of Education Around 20,000 pupils, about 2,000 teachers, 1,000 primary schools in the Southern Africa Sub-region Reading Pilot 1999 Main 2000-2002 Fourteen Ministries of Education Over 40,000 pupils, about 5,300 teachers, 2,300 primary schools in the Southern and Eastern Africa Sub-region Reading and Mathematics SACMEQ I SACMEQ II

10. Impact of SACMEQ (national) • To inform presidential commissions of inquiry into education (Namibia, Zimbabwe) • As inputs in education sector analyses (Kenya, Uganda) • To develop national assessment systems (Lesotho, Uganda, Zambia) • For the development of education policy investment frameworks or programmes (Malawi, Zambia, and Uganda) • For the development of education sector master plans or strategic plans (Mauritius, Namibia, Mozambique) • In educational reforms and school improvement initiatives (Seychelles, Namibia, Lesotho, Zambia) • Strengthening of national EMIS

11. Impact of SACMEQ (international) • To inform the formulation of regional and global strategies for the development of education systems in Africa and elsewhere • Regional EFA Plans, • SADC Human Resource Development Sector’s programmes, • NEPAD's strategic framework, • EFA Global Monitoring Report) • Use of SACMEQ Data Archive by university professors, independent researchers, doctoral and other post-graduate students. • Pool of talented planners and researchers in the sub-region

12. Contents and Structure of SACMEQ Data Archive • Data files • Country files and mega file • Compatible with different software tools • Data from different units of analysis (school heads, teachers, and G6 pupils) have been merged at pupil level. • Technical documents • Policy concern documents • Sampling frames and parameters • Curriculum and test blue prints • Instruments • Training materials for research coordination, data collection and data cleaning • Code books • Dummy tables • Software • Reports and articles, etc.

13. Planned, Achieved, and Effective sample size (SACMEQ II)

14. Describe domains (content) Describe skills (behaviours) Construct test blueprint (domains by skills grid) Write trial-test items (twice as many as required) Implement trial testing and complete Rasch and classical item analysis Select test items to achieve (a) balance across test blueprint, and (b) “overlaps” with other studies Cross-check the descriptions of competency levels with the test blueprint to ensure face validity and construct validity Implement main data collection Conduct Rasch item calibration and test equating Complete skills audit to identify the specific skills required for success on each item Cluster items with similar difficulties and requiring similar skills into competency levels Develop summary descriptions for each competency level Analyze official curricula, school syllabi, textbooks, and examinations

15. Test Blueprints

16. Structure of SACMEQ Tests SACMEQ-II Pupil (83 items) SACMEQ-II Teacher (49 items) SACMEQ-II Pupil (63 items) SACMEQ-II Teacher (41 items) SACMEQ-I Pupil (59 items) Mathematics Test Reading Test

17. Essential items • Established by subject experts • Prior to the data collection

18. Levels for Reading Literacy

19. Levels for Mathematics Literacy

22. Gender Differences in AchievementReading Mathematics Girls better Girls better Boys better Boys better