Assessment Validation – Why and How Tony Auciello

1. Assessment Validation – Why and How Tony Auciello

2. Session objective • Gain a common understanding with regard to validation requirements for the VET sector and Sydney Institute

3. What is assessment validation? Process of comparison and evaluation of assessment tools/processes against competency standard/s and bench marks to ensure that assessment tools meet the Principles of Assessment and the Rules of Evidence.

4. NVR Standards for RTOs • 15.5 Assessment including Recognition of Prior Learning (RPL): (a)meets the requirements of the relevant Training Package or VET accredited course; and (b)is conducted in accordance with the principles of assessment and the rules of evidence; and (c)meets workplace and, where relevant, regulatory requirements; and (d)is systematically validated.

5. NVR Standards for RTOs • 15.3     Staff, facilities, equipment and training and assessment materials used by the NVR registered training organisation are consistent with the requirements of the Training Package or VET accredited course and the NVR registered training organisation’s own training and assessment strategies and are developed through effective consultation with industry.

6. What are the advantages of validation? • Consistency of interpretation of competency standards and benchmarks • Continuous improvement • Builds confidence in VET products • Customer service • Compliance

7. What do we validate? • assessment methods • assessment tools • assessment procedures • assessment decisions High Risk Units first

8. Underpinning Knowledge When Validating • Principles of Assessment • Rules of Evidence • Employability skills • AQF standards • Training Package/units competence

9. Principles of Assessment • Validity - The process assesses what it claims to assess. Validity requires that: • assessment against the units of competency • assessment of knowledge and skills must be integrated with their practical application • judgement of competence must be based on sufficient evidence. • Flexibility – To be flexible, assessment should: reflect the candidate’s needs, provide for recognition of competencies no matter how, where or when they have been acquired, draw on a range of methods appropriate to the context, competency and the candidate. • Fairness - Fairness in assessment requires: consideration of the individual candidate’s needs and characteristics and any reasonable adjustments that should be applied. • Reliability - Refers to the consistency of the interpretation of evidence and the consistency of assessment outcomes. Reliability requires a standard benchmark of assessor competence and relevant vocational competence.

10. Rules of Evidence • Validity -The process assesses what it claims to assess. Validity requires that: • assessment against the units of competency • assessment of knowledge and skills must be integrated with their practical application • judgement of competence must be based on sufficient evidence. • Currency - Relates to the age of collected evidence. Competency requires demonstration of current performance. • Sufficiency - Relates to the amount of evidence collected. • Authenticity -Relates to ensuring the evidence is from or of the candidate and not another person.

11. Employability skills • Employability skills are the non-technical skills and knowledge necessary for effective participation in the workforce, and can include skills such as communication, self-management, problem solving and teamwork. They are also sometimes referred to as generic skills, capabilities, enabling skills or key competencies.

12. AQF Qualifications • The Australian Qualifications Framework (AQF) is the national policy for regulated qualifications in Australian education and training. It incorporates the qualifications from each education and training sector into a single comprehensive national qualifications framework.

13. AQF Qualifications

14. The future of assessment • all new training packages assessments work-based or simulated environment • COAG directions external validation

15. Clustering • Clustering units that combine to form an integrated work activity/task. • Clustering generic elements across units • Clustering common themes across units

16. Assessment Design

17. Tips - designing quality assessments Visualise a competent person in the workplace

18. Dimensions of competency • Task skillsPerforming the task/job to the required standard. • Task management skills (variables)Able to do more than one thing at a time and managing the tasks correctly. • Contingency management skillsResponding appropriately to irregularities and breakdowns in routine within a job or workplace. • Job/role environment skills (outcomes)Able to deal with the responsibilities and expectations of the work environment.

19. Examples Assessment methods • Observation • Questioning • Portfolios • Work samples • Third party feedback • Simulations, role plays • Presentations • Self assessment

20. Mapping What is mapping? Why do you need to do this? The Steps in Mapping. • Analyse the Unit of competency • Select the assessment tool • Map the components of the UOC to the assessment tools chosen

21. Analyse the Unit of CompetenceThe following diagram shows the range of information taken into account at this stage of the development process:

22. How to identify assessment methods or evidence gathering techniques

23. Sample Mapping Document

26. Streamlining • Streamlining / Refining Assessment Validation • less forms X X

27. Under Review Under review Under review

28. For more information follow the links • Competency Based Assessment Publications • Training Packages @ Work • Training.gov.au