VET Practitioners’ Network Conference The Changing VET Landscape Workshop 2 Assessments designed for evidence gathering

1. VET Practitioners’ Network Conference The Changing VET Landscape Workshop 2 Assessments designed for evidence gathering – how much is enough? Friday 31 July 2009 John Molenaar Contract quality auditor Manufacturing Learning Victoria

2. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Overview of presentation: • Exploration of the purpose of assessment • Assessment – AQTF 2007 Essential Standards for Registration requirements • What does the auditor look for 4. Application to TAA40104

3. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? How much is enough? ???

4. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? It is all TOO much! !!

5. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? What does assessment mean to me? Why do I assess? Questions that I have about assessment!

6. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? What is the purpose of assessment?

7. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Assessment The process of collecting evidence and making judgements on whether competency has been achieved, to confirm that an individual can perform to the standard expected in the workplace, as expressed by the relevant endorsed industry/enterprise competency standards of a Training Package or by the learning outcomes of an accredited course AQTF 2007 User Guide to the Essential Standards for Registration

8. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? But assessment is not just about confirming that an individual can perform to the standard expected in the workplace. It is also about getting to know our students and the quality of their learning and to use this knowledge and understanding to their benefit.

9. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Assessment is without doubt one of the major ''drivers'' of the teaching-learning process. It is therefore important for us to be familiar not only with the technical aspects of the many different forms of assessment currently in use but also with their advantages and limitations and about assessment issues and concerns.

10. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? We need to ensure that learning is not simply assessment-driven. It can be argued that presently we have far too much assessment, (how much is enough!) but that neither the quality nor the diversity of this assessment is right. Students are highly intelligent people; if we confront them with a game where learning is linked to a rigid and monotonous diet of assessment, they will learn according to the rules of that game. To improve their learning, we need to improve our game.

11. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? All too often, assessment is associated with the end-product of the learning process with emphasis given to measurement, evaluation, grading and social acceptance. How much is enough?

12. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Assessment What for? Who for? to reward learners/teachers/outsiders to motivate learners to know what they know learners/teachers/outsiders to know what they don't know learners/teachers/outsiders to punish learners to certificate learners/outsiders to classify learners/outsiders to compare learners/outsiders to evaluate learners/outsiders to diagnose learners/teachers to appraise learners/teachers/outsiders to empower learners to improve the quality of learning learners/teachers/outsiders

13. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Formative and summative There are two main types of assessment: Summative assessment - Summative assessment is generally carried out at the end of a course or project. In an educational setting, summative assessments are typically used to assign students a course grade. Formative assessment - Formative assessment is generally carried out throughout a course or project. Formative assessment, also referred to as educative assessment, is used to aid learning. In an educational setting, formative assessment might be a teacher (or peer) or the learner, providing feedback on a student's work, and would not necessarily be used for grading purposes. Summative and formative assessment are referred to in a learning context as "assessment of learning" and "assessment for learning" respectively.

14. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Lets get back to the AQTF 2007 and compliance with the Essential Standards of Registration Assessment The process of collecting evidence and making judgements on whether competency has been achieved, to confirm that an individual can perform to the standard expected in the workplace, as expressed by the relevant endorsed industry/enterprise competency standards of a Training Package or by the learning outcomes of an accredited course AQTF 2007 User Guide to the Essential Standards for Registration

15. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? AQTF 2007 and compliance with the Essential Standards of Registration Standard 1 Element 1.5 Assessment, including (RPL): a) meets the requirements of the relevant Training Package or accredited course b) is conducted in accordance with the principles of assessment and the rules of evidence c) meets workplace and, where relevant, regulatory requirements.

16. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Assessment (which includes RPL): • meets the requirements of the Training Package or accredited course • is consistent with the training and assessment strategy • is valid, reliable, flexible and fair • focuses on the application of knowledge and skill to the standard of performance required in the workplace • involves the collection of sufficient, valid, authentic and current evidence to enable a judgement to be made about whether competency has been attained • confirms that workplace and regulatory requirements are met. Judgementsmade by assessors against the same competency standards are consistent. Systems, processes, tools and practices are improved.

17. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Assessment (which includes RPL): is valid, reliable, flexible and fair To be reliable, the assessment methods and procedures must ensure that competency standards are applied consistently. (Validation (assessors), moderations (students) To be flexible, assessment should be able to take place on-the-job, off-the-job, or in combinations of both or in accordance with the ‘Assessment Guide’ incorporated within each unit of competency. To be fair, the assessment must not advantage or disadvantage any particular group of candidates. To be valid, assessment must assess what it claims to assess, so sufficient evidence must be collected from authentic sources and locations.

18. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Assessment (which includes RPL): involves the collection of sufficient, valid, authentic and current evidence to enable a judgement to be made about whether competency has been attained

19. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Sufficiency relates to the quality and quantity of evidence assessed. It requires collection of enough appropriate evidence to ensure that all aspects of competency have been satisfied and that competency can be demonstrated repeatedly. Supplementary sources of evidence may be necessary. The specific evidence requirements of each unit of competency provide advice on sufficiency.

20. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Assessment is valid when the process is sound and assesses what it claims to assess. Validity requires that: • assessment against the units of competency must cover the broad range of skills and knowledge that are essential to competent performance (b) assessment of knowledge and skills must be integrated with their practical application (c) judgement of competence must be based on sufficient evidence (that is, evidence gathered on a number of occasions and in a range of contexts using different assessment methods). The specific evidence requirements of each unit of competency provide advice on sufficiency.

21. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? To accept evidence as authentic, an assessor must be assured that the evidence presented for assessment is the candidate’s own work.

22. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? What does the auditor look for? Intent Assessment ensures that only learners who hold the requisite skills and knowledge are certified as competent.

23. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? The auditor reviewing assessment tools and assessments conducted look at: For the unit, that the organisation’s assessment materials sufficiently address: • performance criteria • critical aspects of evidence • required knowledge • required skill • context and consistency of assessment requirements • principles of assessment and rules of evidence • workplace requirements • assessment methods align to the training and assessment strategy. • evidence guide • employability skills

24. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Information included in the audit report includes: - Required assessment methods include: - Assessment tools consist of: (list assessment tools) - The organisation has: • developed suitable benchmarking information for all assessment tools • provided information to students about the context and purpose of assessment. - For this unit, the organisation has gathered sufficient evidence to make valid judgement of competence (including RPL outcomes) for the following students: - Student names: (review student files)

25. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? • Elements • Performance Criteria • Assessment methods • Critical Aspects of evidence • Frequency of (e.g. observations) • Tools to reflect the assessment methods.

26. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Tools to reflect the assessment methods. How much is enough? One tool to reflect each assessment method and to show frequency of assessment.

27. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? TAA40104 Assessment methods and tools may include: (examples) • reports and documents outlining processes etc • Two delivery plans, individual learning relationships, planning and organising • documented applications, paticipation • conducting assessments on a number of occassions • developing and trialing • example of practice • teaching and learning materials • surveys

28. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Evidence may include: • reports • examples of practice • teaching and learning materials • Surveys • etc

29. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Unit - Work effectively in vocational education and training Critical aspects for assessment and evidence required to demonstrate competency in this unit • Address scope of the unit • Use a range of appropriate assessment methods and evidence gather techniques • Evidence gathered in the workplace where possible • Relate to a number of performances • Meet the rules of evidence • Judgments made - when consistency of performance has been demonstrated.

30. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Unit - Communicate with others in familiar and predictable contexts Appropriate assessment strategies include: • development of the minimum of two delivery plans • documentation of a minimum of two individual learning relationships • documentation of a minimum of two examples of developing a work-based learning pathway Assessment tools need to reflect these How many assessment methods should be identified for each unit? (The assessor looks for at least two and rationale why these are selected)

31. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Learning rather than assessment should be the end product of education. Students should learn through assessment not learn to be assessed.

32. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? A 10-point manifesto taken from 500 Tips on Assessment by Sally Brown, Phil Race and Brenda Smith, Kogan Page, 1996.

33. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? • Assessment should be based on an understanding of how students learn. Assessment should play a positive role in the learning experiences of students. • Assessment should accommodate individual differences in students. A diverse range of assessment instruments and processes should be employed, so as not to disadvantage any particular individual or group of learners. Assessment processes and instruments should accommodate and encourage creativity and originality shown by students. • The purposes of assessment need to be clearly explained. Staff, students, and the outside world need to be able to see why assessment is being used, and the rationale for choosing each individual form of assessment in its particular context. • Assessment needs to be valid. By this, we mean that assessment methods should be chosen which directly measure that which it is intended to measure, and not just a reflection in a different medium of the knowledge, skills or competences being assessed.

34. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? 5. Assessment instruments and processes need to be reliable and consistent. As far as is possible, subjectivity should be eliminated, and assessment should be carried out in ways where the grades or scores that students are awarded are independent of the assessor who happens to mark their work. External moderation should be active contributors to assessment, rather than observers. 6. All assessment forms should allow students to receive feedback on their learning and their performance. Assessment should be a developmental activity. There should be no hidden agendas in assessment, and we should be prepared to justify to students the outcomes we award them, and help students to work out how to improve. Even when summative forms of assessment are employed, students should be provided with feedback on their performance, and information to help them identify where their strengths and weaknesses are. 7. Assessment should provide staff and students with opportunities to reflect on their practice and their learning. Assessment instruments and processes should be the subject of continuous evaluation and adjustment. Monitoring and adjustment of the quality of assessment should be built in to quality control processes in universities and professional bodies.

35. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? 8. Assessment should be an integral component of course design, and not something bolted on afterwards. Teaching and learning elements of each course should be designed in the full knowledge of the sorts of assessment students will encounter, and be designed to help them show the outcomes of their learning under favourable conditions. 9. The amount of assessment should be appropriate. Students' learning should not be impeded by being driven by an overload of assessment requirements, nor should the quality of the teaching conducted by staff be impaired by excessive burdens of assessment tasks. 10. Assessment criteria need to be understandable, explicit and public. Students need to be able to tell what is expected of them in each form of assessment they encounter. Assessment criteria also need to be understandable to employers, and others in the outside world.

36. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? How much is enough?

37. VET Practitioners’ Network Conference - The Changing VET Landscape July 2009Workshop 2: Assessments designed for evidence gathering – how much is enough? Useful resources: • A Guide to developing Training Package Assessment Materials (DETYA July 2001) • On Track – moving towards Assessment validation (Vocational Education and Assessment Centre TAFE NSW 2002) • VETASSESS – Assessit.net (http://www1.assessit.net/)