Re-evaluating School-based Moderation: Enhancing Accountability in Education

1. Towards a critical re-evaluation of school-based moderation processes in the era of accountability 13th SAAEA Conference, Botswana 19-22, May 2019 Mapaleng Lekgeu

2. Presentation outline • Contextualisation of the study • Research purpose and question • Literature Review • Theoretical framing • Methodology • Discussion of findings • Conclusions

3. Contextualisation of the study • School-based Assessment (SBA) was introduced in 2001 in South Africa (SA) by minister of Education – Kader Asmal, after having been piloted in three provinces (WC,NC & GP), in 2000. • Objectives: (1) to fragment the high-stakes, once-off examination system; (2) to use assessment for formative purposes and to provide feedback timeously; (3) to modify instructions; to diagnose and remediate areas of learners’ weaknesses. • Named Continuous Assessment (CASS) under Curriculum 2005 (OBE, 1998); but became SBA under National Curriculum Statement (NCS) in 2003 and Curriculum and Assessment Policy Statement (CAPS) in 2012.

4. Contextualisation Continues…… • Moderation of SBA tasks became cumbersome: • site-based, district and provincial, quarterly; • done according to exam norms to meet standards; • thus highly summative,defeating the ends • Amid a plethora of policy directives, the problems of implementation of SBA moderation continued unabated. • Some of the problems are: • disregard of policy imperatives; with no uniform interpretation and application of policy directives • variations in the way schools implement moderation; • Lack of due diligence in moderation • quality of tasks affected; • marks inflated, performance obscured as a result, Poliah (2010) • Moderation criteria: CAPS-aligned content, cognitive levels, variety of questioning techniques, marking guideline and weightings

5. Contextualisation Continues…….. • As reported by Umalusi evaluators, • poorly constructed tasks, • unreliable memoranda and • inconsistent mark allocations; could still be found in some of the schools • This study therefore aims to • re-evaluateSBA moderationprocesses and • to suggest ways of improving moderation and to hold moderators liable • The re-evaluation of SBA moderation relates to independent schooling sector in Gauteng province, South Africa • The definition of moderation, in its broadest terms, also receives the attention of the study.

6. Research Purpose and Question • Purpose: to re-evaluate the SBA moderation process in such a way as to make moderators accountable and to suggest new ways of doing things . • Research Question: How can SBA moderation processes be used for moderator accountability?

7. Literature Review • There are differences in the definition of moderation across the literature: • comparability, reliability and validity of teacher judgements (Grant, 2013) • communicative and collaborative process of judging learner portfolios (Van der Schaaf, Baartman & Prins, 2012) • quality control process involving monitoring, approval and judgements ensuring consistency in interpretation and application of standards (Maxwell, 2006) • process which ensures that assessment of outcomes are fair, valid and reliable (Umalusi, 2018) • This study adopts the definition as put forward by Maxwell (2006), i.e. that of moderation • as quality control involving monitoring of teacher judgements; • thereby ensuring conformity in interpretation and application of standards

8. Literature Review continues…… • But the study further views moderation • as not only quality control but also quality assurance process; • a process in which teachers collaborate and corroborate in community of practice • In fact the study argues that moderation should be a five-stage process: • pre-assessment moderation • process moderation • verification of marking • verification of marksheets • feedback to teachers (developmental) and learners (diagnostic)

9. Methodology • Quantitative study involving content analysis of reports archived at Umalusi • The reports included data collected from forty (40) independent schools in Gauteng Province visited between 2016 and 2018 • The schools were sampled purposively. • Schools were then grouped according to regions, namely Ekurhuleni, JHB and Tshwane for analysis purposes

10. Analysis of the five-stage moderation • Pre-assessment moderation: the first stage of moderation which looks at CAPS content; cognitive levels; questions variety; mark allocation; intended skills and question analysis • Process moderation: this is a stage where portfolios are moderated; irregularities managed; and actual writing of assessment tasks monitored. • Verification of marking: this includes script sampling procedures (10%); checking of compliance to marking guideline; and learner moderation • Verification of marksheets: moderators verify compliance to correct design; correct recording and weighting of marks; and mark analysis • Feedback to teachers and learners: developmental feedback to teachers; diagnostic and enrichment feedback to learners

11. Findings

12. Findings continues…

13. Discussion of the findings • Compliance to the five-stage moderation process appears to be erratic in all schools across the identified regions • Pre-assessment moderation is popular in Tshwane region with 100% of schools complying • As much as 83% and 75% of schools in Johannesburg (JHB) and Ekurhuleni regions respectively comply with pre-assessment moderation • Less than 70% of schools across the regions comply with the criterion of moderation feedback to teachers and subsequently to learners. • The situation of compliance to moderation feedback is not good in Ekurhuleni where 58% of schools are non-compliant. • In fact, Ekurhuleni comes third in compliance to all criteria across the regions

14. Discussion continues……. • It appears that between 80 and 100% of schools in all regions comply with CAPS-aligned content, questions variety and marking guideline as criteria for pre-moderation • Less than 40% of schools conduct question analysis, thereby determining level of difficulty of questions before they are written. • A satisfactorily number of schools (over 65%), comply with the criterion of cognitive levels • Although over 50% of schools in JHB and Tshwane regions comply with the criterion of weighting of marks, merely 25% of them do so in Ekurhuleni.

15. Conclusions • In conclusion, the following points are worth noting: • a broad-based countrywide study in the implementation of the five-stage moderation model is imperative • the five-stage model proved to be implementable in this study, thus productive and rewarding. • the actualisation of the suggested model of moderation varies between schools in all the identified regions • insufficient literature regarding moderation liability by moderators • schools, rather than moderators, should carry the responsibility of accounting for moderation • management of moderation processes should involve quality assurance, quality control and quality review

16. Recommendations • the district offices should establish an accountability office made up of district officials to adjudicate on laissez-faire moderation; proposing and reporting on plan of action • moderators should be mobilised into networking systems with colleagues in cluster of schools as community of practice – the idea of social moderation • create platforms where school-based moderators would interrogate moderation feedback from external moderators • introduce consequence management for underperforming moderators: assistance first and then be subjected to corrective action and ultimately sanctions • build teachers’ assessment literacy and capacity • incentivise good practices

17. Thank You