Investigating SBA and Examination Scores in Adult Education in South Africa

1. A Quantitative Investigation of SBA and Examination Scores in Adult Education in South Africa 13th SAAEA Conference Gaborone, Botswana Dr Nkoloyakhe Mpanza 19-22 May 2019

2. Presentation Outline • Introduction and Background • Purpose and Research questions • Formulation of hypotheses • Methodology • Findings • Conclusion

3. Introduction and Background • Umalusi is responsible for the qualifications on level 1-4 of the National Qualification Framework (NQF). • The NQF Level 1 qualification for adults: • provides second chance to adults and • out of school youth who cannot be accommodated in the mainstream schooling sector.

4. Introduction and Background

5. Introduction and Background • Assessment in this qualification comprises Site-Based Assessment (SBA) and examination. • The SBA and examination contribute 50% each towards the final mark. • SBA scores are included in some of the high stake qualifications around the world (Williamson, 2016).

6. Introduction and Background • Literature revealed inaccuracies regarding SBA in some countries Tailor (2005, in Williamson, 2016), • statistical moderation generally adjusted SBA marks downward in the UK. • van der Berg & Shepherd (2008) identified two types of assessment inaccuracies • Assessment leniency, where SBA mark is far higher than an examination mark; and • Low assessment reliability, where SBA marks and examination marks are weakly correlated.

7. Introduction and Background • For Umalusi: • SBA challenges also observed by Umalusi • Statistical moderation used to validate credibility • Difference above 10% between the means of SBA and that of examination indicate that the SBA is a poor signal of performance. • Hence is considered misalignment • Rejection of SBA scores

8. Purpose of the study & Research question Purpose • The study was conducted to investigate whether the means of SBA and examination scores per learning areas that are assessed by the Department of Higher Education and Training (DHET)are comparable. Research Question • Is the mean of SBA scores comparable to that of examination scores in each of the 5 learning areas assessed the DHET for the NQF Level 1 qualification for adults?

9. Formulated Hypothesis • There is no significant difference between the mean of SBA scores and that of examination scores in the five learning areas assessed by the DHET.

10. Methodology • This quantitative study was conducted using five purposefully sampled learning areas assessed by DHET. • These learning areas are Small Medium and Micro Enterprises (SMME);Mathematical Literacy (MLMS); Mathematics and Mathematical Sciences (MMSC); Economic and Management Sciences (EMSC) and Natural Sciences (NATS). • Source of data: November 2018 examinations and SBA submitted DHET to Umalusi.

11. Methodology • Data was analysed using Statistical Analysis System (SAS). • Two-sample t-tests were conducted per learning area to test the formulated hypothesis. • The selected level of significance (alpha) is p= 0.05

12. Findings

13. Findings • Hypothesis • “There is no significant difference between the mean of SBA and that of examination scores in the five learning areas assessed by the DHET.” • Rejection rule: • p-value < alpha then we reject. • The p-value is less than alpha(0.05) in all 5 learning areas, therefore, hypothesis was rejected. • There is a significant differencebetween the SBA and examination scores in all 5 learning areas

14. Conclusion • The study concludes that: • student performance in SBA and examination of selected learning areas is not comparable. • This motivates for a continuation of statistical moderation process.

15. Thank You