Introduction

1. Constructing an integrated model for the quality assurance of school based assessment in preparation for the National Senior Certificate

2. Introduction • International trend in assessment is towards a combination of both external and internal assessment. • Internal assessment is often put in place to collect evidence on learning outcomes that do not easily lend themselves to pen and paper tests. • Can also include learning outcomes that are traditionally assessed by external examinations. • Reliability of internal or school based assessment is always in question. • Statistical approaches to quality are restrictive, but are adopted because they are easy to implement.

3. Challenges Associated with School Based Assessment • The level of teacher preparation. • Variation in the scoring of assessment tasks among teachers. • Increase in both teacher and learner work load. • Variance in SBA can result from learners being assisted by their teachers, peers or parents or due to the difference in the difficulty of classroom based tasks. • Different approaches to SBA among different schools.

4. Current Status of SBA in South Africa • Quality education is a national priority in South Africa. • Emerging from an apartheid education, Government has embarked on overhauling the entire curriculum. • As from 2008, a new certificate, the National Senior Certificate (NSC) will be issued at the end of Grade 12. • SBA constitutes 25% and external examinations constitute 75% of the final assessment. • National system of education with the national Department of Education responsible for policy development and policy compliance and the nine provincial education departments, responsiblefor the implementation of policy.

5. Current Status of SBA in South Africa • Umalusi, the Quality Assurance Council, is responsible for the final standard and quality of assessment, leading to the issue of the certificate. • Compulsory inclusion of SBA as part of the final exit certificate was mandated in 2001. • Given the concerns regarding the reliability of SBA, a model of statistical moderation was adopted. • SBA marks are adjusted per school, per subject within a certain range of the adjusted examination marks. • Provincial Education Departments (PEDs), are responsible for internal moderation. • Effectiveness of internal moderation systems varies from one province to the other.

6. Model for Quality Assurance of SBA

7. Conceptual Framework for Quality Assurance of SBA Basic Principles: • Quality Assurance must focus on the quality of each component of the assessment and not just the end-product. • Quality assurance must focus on what happens before, during and after the assessment takes place • Before: Establishing appropriate circumstances for assessment to take place • During: Verifying that assessment procedures and judgments are appropriate. • After: Retrospective analysis to see whether assessment procedures or assessor judgments need to be improved. • Model based on the Input, Process , Output (IPO) model, which states there is a causal link between the inputs, processes and outputs.

8. Conceptual Framework for Quality Assurance of SBA Basic Principles: • Point of departure is the establishment of clearly defined standards that are accepted and understood by all participants in the assessment process. • Definition of standards must be followed by appropriate support, training and guidance. • Implementation must be accompanied by monitoring and evaluation, leading to feedback.

9. Conceptual Framework for Quality Assurance of SBA FEEDBACK • INPUT • Policies • Resources • Structures • Procedures • PROCESS • Designing • Administration • Evaluating • OUTPUT • Final Assessment Result STANDARDS

10. Conceptual Framework for Quality Assurance of SBA Fundamental Pillars • Setting of Standards • Curriculum Standards • Performance Standards • Assessment System Standards • Moderation Standards • Support and Guidance • To Managers • To Subject Advisors • To Educators • Monitoring and Evaluation • Evaluation of the Assessment System • Monitoring the implementation of the assessment process • Moderation of the Assessment Outcome • Retrospective Analysis

11. Setting of Standards • The DoE and Umalusi have the joint responsibility to provide standards for good assessment. • Standards for assessment must relate to the input, process and output of assessment. (a) Curriculum Standards • Contained in the National Curriculum Statements (NCS) and are articulated as learning outcomes, assessment standards and the prescribed content. • Proposed that the curriculum standards differentiate between knowledge and skills that are best measured through written examinations and those are best measured through SBA. • Performance Standards • Assists educators in judging how well learners have performed with reference to a specific curriculum standard. • Exemplars of learner performance assist in establishing a common standard among assessors.

12. Setting of Standards • Assessment System Standards • Comprises a description of the policies, regulations, structures and procedures that make up assessment. • This includes: design and production of assessment instruments; conduct of assessment; awarding of marks and evaluation, feedback and review. • Moderation Standards • Minimum requirements for internal and external moderation must be established and adhered to by all providers of assessment.

13. Support and Guidance • Support and Guidance to Managers • Managers include managers at national, provincial, district and school levels. • In most cases these managers are educators who have little or no management training or experience. • Training should relate to contents of the policy and management of the policy. • Support and Guidance to Subject Advisors • Subject advisors are the arbitrators and mediators of policy. • Support to subject advisors should include: • Policy and procedures relating to SBA. • Content areas of the subject. • Forms of assessment relating to SBA • Subject Advisory role. • Appropriate support material. • Support and Guidance to Educators • Success of SBA implementation is determined by the competency of educators and their commitment to this mode of assessment. • Pre-service and In-service training. • Development of item banks • Exemplars of learner evidence

14. Monitoring and Evaluation • Final leg in the quality assurance process. • PEDs must take responsibility for the schools and districts in the province. • The DoE will provide support,guidance and exercise national oversight over all PEDs. • Umalusi will take responsibility for the final quality assurance . Umalusi DoE 9 PEDs Districts Schools

15. Monitoring and Evaluation (a) Evaluation of the Assessment System (Input) • Includes an evaluation of all policies, regulations, structures, processes and tools used in SBA. • District to evaluate the schools under their control. • Provincial Head office to evaluate districts, by conducting sample school evaluations. • DoE will evaluate the PEDs, by conducting sample district evaluations • Umalusi will finally evaluate the assessment bodies. • Umalusi report will serve as the final report on the status of the assessment systems of the nine PEDs.

16. Monitoring and Evaluation (b) Evaluation of the Implementation of the Assessment Process • Focuses on the actual implementation of the assessment process. • Covers the following aspects: • Ensuring SBA is implemented at schools. • Ensuring compliance with policy. • Ensuring that assessment tasks are designed in accordance with subject assessment guidelines. • Ensuring that assessment tasks are administered in accordance with the principles of good assessment. • Ensuring that the assessment evidence is marked according to assessment guidelines. • Ensuring that marks are computed according to policy.

17. Monitoring and Evaluation (c) Moderation of the Assessment Outcome • IPO model pre-supposes that if the input and the process have been thoroughly quality assured, the outcome should demonstrate a high reliability. • Assessment tasks that mirror the examinations must be separated from the other SBA tasks (tasks that measure skills). • SBA assessment tasks that mirror the examinations can be statistically moderated to the examinations, allowing for a 5 – 10% tolerance range. • The other skills based tasks must be moderated through qualitative processes, which should occur at the four tiers of organisation: • School Level. • Cluster level. • Provincial Level • National level • Face moderation limited to skills based tasks. • Feedback provided at each level of the moderation process

18. Monitoring and Evaluation (d) Retrospective Analysis • Review of the • assessment procedures • assessment judgments • assessment outcomes. • Improvement of areas of weakness. • Feedback

19. Challenges • Making quality an integral part of each process and phase. • Provision of appropriate human resources at each level. • Effective training of educators. • Common understanding of clearly articulated standards. • Disparate resource and competency levels across schools based on their previous history

20. Conclusion • Statistical moderation does not have to be the sole determinant in the quality assurance of SBA. • Statistical moderation has a role to play in the correlation and moderation of assessment constructs that are similar. • Qualitative data from the inputs, process and outputs of assessment will provide useful information in the monitoring and evaluation of the final assessment outcome. • Clearly defined standards should be the starting point in the quality assurance process.