Competency-based Assessment in TQA Accredited Senior Secondary Courses

1. Competency-based Assessment in TQA Accredited Senior Secondary Courses

2. Issues to be discussed in this forum…Criterion- & competency-based assessment: how different are they?Why use competency-based assessment in TQA courses?Assessment: methods & issuesQuality Assurance: what, why, who and how?

3. Outcomes-referenced assessment: “Occurs where students’ performances are measured against explicit criteria…its focus [is] on achievement against learning objectives or outcomes.” (UTAS www.edu.au/tl/supporting/assessment/judgement.html)

4. Types of outcomes assessment: • Criterion-based approach: Students’ • achievement is measured against a range of • standards (eg from a ‘C’ to an ‘A’ rating) for • each learning outcome • Competency-based approach (or ‘Mastery learning approach’): Students’ achievement is measured against a predetermined standard (eg ‘pass’/‘fail’) for each learning outcome.

5. The criterion- and competency-based approaches share the same fundamental assessment principle. They are both concerned with assessing learning outcomes against predetermined standards.

6. In TQA accredited courses criterion-based assessment has traditionally had three gradations (ratings): ‘C’, ‘B’ and ‘A’ standards. The recently introduced competency-based assessment courses have a two gradations (ratings): ‘competent’; and ‘not yet competent’.

7. In theory there is no reason to limit the number of gradations in competency-based assessment – both criterion- and competency-based approaches to assessment are fundamentally the same. In practice the use of competency-based assessment with two gradations is suited to some kinds of TQA accredited courses.

8. What kinds of TQA courses use competency-based assessment? (I) • Those with learning outcomes where the focus of assessment is on: • the learner’s ability to perform an activity (usually • in a defined context) • the application of knowledge and skills in ‘real • world’ contexts.

9. What kinds of TQA courses use competency-based assessment? (II) Typically these are courses with learning outcomes where it is more important to determine mastery of a discrete learning outcome (ie. the learner ‘can do it’) rather than measuring the degree to which a learner can do something. Underpinning knowledge is implicitly assessed within the context of completion of a task or activity. Eg. Learning outcome: Use a household fire extinguisher safely

10. Assessment • Assessment instruments should make clear to the learner: • a) the nature of the specific assessment task (including the • size/format/due date etc) • b) which learning outcomes are being assessed • c) what the learner must demonstrate • d) how the assessment of the task relates to the final • assessment.

11. Assessment Examples of common assessment methodologies include: direct observation verbal/oral testing practical testing written testing written tasks.

12. Assessment • The method chosen should ‘make sense’ within the context of the course and the learning outcomes being assessed. • Direct observation of a practical competency/criterion • might be more valid than using a written task. • Analysis of a product might be more valid when • the process is not as significant as the outcome or • cannot be easily observed.

13. Assessment • A marking guide should be prepared for each summative assessment instrument. A marking guide articulates ‘what must be demonstrated in the task’. For example: • a ‘correct answer’ list for a test and the number of • questions a learner must get right to ‘pass’ • a list of the specific features of a product or • process that must be demonstrated to ‘pass’.

14. Assessment A marking guide should have a direct relationship to the performance indicators described in the course. The performance indicators may need to be elaborated in order that they ‘make sense’ within a specific assessment task.

15. Assessment Final assessment decisions should be on-balance, holistic ones based on the evidence of learning outcomes measured against the standards stated in the course. Considerations might include the: relative weighting given to individual assessment tasks consistency of demonstrated achievement 'currency' (evidence from the end of a course may have greater relevance and reliability than that collected early in the delivery).

16. Ratings (I) Summative assessment of learning outcomes result in a final rating for each competency. The ‘competent’ rating is a final, overall rating and should not be used as a grade in formative assessment/progress reports. Learners who are assessed as ‘not yet competent’ can be provided with further opportunities to demonstrate competency within the scope of the course’s design-time, provider resources, and TQA reporting requirements.

17. Ratings (II) Some providers use 'not yet assessed' in reports on learner's progress (such as term reports). While this is perfectly acceptable in this context, 'not yet assessed' is NOT a rating that is reportable to the TQA.

18. Ratings (III) Providers may use any form of grading they wish in assessment so long as the final rating is reported as per the course document (eg. ‘competent’, ‘not yet competent’, ‘highly competent’). Examples: % marks marks out of 10 A, B or C pass/fail gold, silver, bronze

19. Awards • Each TQA accredited course contains a description of award requirements. These are algorithms (or rules) defining the relationship between ratings and a final award. • Eg. To gain an award (‘Pass’) in the course the learner must be assessed as ‘competent’ in all five competencies.

20. Qualification • The qualification issued by the TQA on the learner’s Qualifications Certificate comprises the course name and the award gained. • Eg. Personal Pathway Planning: Pass

21. Quality Assurance & the TQA’s Role? The TQA is responsible for the quality assurance for qualifications it awards in senior secondary courses it has accredited. This means that it has to stand behind the statements made on certificates as to the knowledge and skills of the holder of the certificate.

22. Audit Model • How are providers selected for an audit? Risk based approach – ‘flags’ might include: complaints; number of learners/size of provider; major changes in staff; trends in data reported to TQA; time since last audit. • How do we know if we are/are not being audited this year? TQA notification process.

23. Audit Model • Who is audited? Provider not individual teachers. • Who does the auditing? TQA staff and subject experts (when appropriate).

24. Audit Model • Is an audit a ‘witch-hunt’? Opportunities for • confirmation of best practice, identification of • issues, continuous improvement. • What happens if we ‘fail’ an audit? Audit • findings and opportunity to ‘clear’ them, • follow-up visits, report to Authority.

25. What will Auditors look for? • Evidence that the course has been delivered • Evidence that learners ‘have learnt’ • Evidence that assessment is based on the • course’s learning outcomes and standards.

26. Course Delivery • Planning: did the course provider have an articulated plan for the delivery of the course and its assessment? • Was the course actually delivered?

27. Planning (I) • A structured course delivery plan would articulate: • the sequence of course delivery (what parts of the course were delivered to learners and in what order) • when formative and summative assessment occurs.

28. Planning (II) A course delivery plan is sometimes called a ‘scope and sequence’. Course delivery plans could be a ‘whole of course’ plan or could be arranged as a sequence of lesson, weekly, monthly or term plans depending on individual teachers’/providers’ preferred methodology.

29. Was the Course Actually Delivered? • Evidences could include: • records of attendance • assessment tasks and student work (especially • examples of compulsory tasks such as keeping • a journal or log book) • interviews with past and present students • records of assessment.

30. Evidence that Assessment is Based on the Course’s Learning Outcomes & Standards Evidences could include: • summative assessment tasks & marking guides • some examples of student work showing the • application of marking guides • assessment records and information about how they • were used to make final assessment judgements.

31. Assessment Records Should Contain: • the name of each learner • the competencies assessed and/or the assessment • instrument/s used to make the assessment • the date of assessment • the name of the assessor/s.

32. Assessment Records • The class assessment record could be a single document for a whole class (such as a spreadsheet or paper-based grid table) or comprise a set of records, one for each learner.

33. The ‘Enhanced’ Audit Model • Used for courses with special risk factor (eg Essential Skills courses with TCE ‘ticks’). • Same as other audit model but auditors will also wish to view examples of the work of ‘borderline’ learners. ‘Borderline’ means the gaining or not the ‘Pass’ award.

34. Examples of the Work of ‘Borderline’ Learners (I) • The nature of the work - major summative assessment task/s that allows a judgement to be made on all competencies/the award. • How many? - use those that arise (do not ‘invent’ them). Number of examples will depend on the number of borderline learners & the period that course had been delivered.

35. Examples of the Work of ‘Borderline’ Learners (II) • How kept? - archive copies (photocopies, CD-ROM or other storage mechanisms). • Where kept? - provider responsibility. System needs to be robust enough to handle staff changes.