University of Surrey Learning and Teaching in HE programme Introduction to assessment

1. University of Surrey Learning and Teaching in HE programme Introduction to assessment Norman Jackson

2. My teaching model Didactics - the art or science of teaching Didactic ‘directed model of learning’ Autodidactic ‘self-directed model of learning’ My teaching model - Collaborodidactic

3. Our professional learning system existing codified resources knowledgeable course participants teacher as facilitator our collective tacit knowledge gained through experience

4. Assessment what do you associate with the word assessment? 2 mins jot down

5. Assessment what is it? In the context of a course: assessment is the means by which teachers judge what students know and can do. It involves sampling a students knowledge, understanding and capabilities, making inferences about her responses and estimating the extent to which learning is demonstrated. What criteria? What is valued in grades/marks? About what? achievements, competency, potential, aptitudes, attitudes, motivations What forms of evidence? How? How much? Who does it? self, peers, tutors or employers What types of knowledge/skill? What levels of understanding?

6. What do you perceive are the weaknesses in the way we assess students’ learning in HE? * Overload of students/staff * Too many assignments with same deadline * Insufficient time for completion of assignments * Insufficient time for staff to mark * Inadequate feedback to students * Wide variation in assessment demands of different units * Wide variations in marking and use of criteria * Variations in marking within a module * Fuzzy or non-existent criteria * Undue precision and specificity of marking schemes * Students do not know what is expected of them * Students do not know what counts as good or bad George Brown with Joanna Bull and Malcolm Pendlebury Assessing student learning in higher education

7. Why do we do it? Some reasons for assessing students *to provide feedback to students to improve their learning * to motivate students * to diagnose a student’s strengths and weaknesses * to help students develop their capacity for self-assessment * to provide a profile of what a student has learnt * to pass or fail a student * to grade or rank a student * to show a student is capable of proceeding * to provide information for future selection * a licence to practice * to provide feedback to lecturers on students’ progress * to improve teaching * to evaluate the strengths and weaknesses of a course

8. Typology of assessment functions Diagnostic - assessment at the start or during a course to provide a basis for a programme of development usually at the start of a course. Formative - in course assessment to provide feedback on progress to aid further development Summative - assessment about what has been learnt normally at the end of a programme.

9. What has been your best/ worst personal experience of being assessed? What are the main difficulties and problems you have encountered when assessing students?

10. Assessment Any procedure used to estimate a student’s learning for whatever purpose.

11. Some trends in assessment over last 20y From To end year exams end- and in-module assessment written unseen exams to coursework and exams entirely tutor led mixed tutor/student-led implicit criteria explicit criteria competition competition & collaboration product assessment product & process assessment objectives intended outcomes subject know/under other types of know/und valued A level entry diverse entry cannot assume common background knowledge What impact have these changes had on teaching, learning and standards? Trends in degree classifications

12. Proposition Generally speaking students’ learning is driven by assessment.Students put most effort into those aspects of the course that are assessed. What you assess and how you assess it will have a major influence on what students learn and how they learn it So before you assess you need to decide what you want them to learn and why you want them to learn it. What types of learning do you expect students to gain and demonstrate in your modules?

13. Example of useful shared experiential knowledge about assessment BEng/MEng COURSE * weekly lectures + problem sheets & classes + exam * as classes got bigger teachers stopped marking problem sheets, students stopped doing problems and ‘hid’ in classes. * exam marks dropped from 55-45% PROBLEMTeaching response to doing more with less resulted in poor learning outcomes WHAT WOULD YOU DO?

14. WHAT THE COURSE TEAM DID BOLD EXPERIMENT * regulation - do 50 out of 80 problem sheets or don’t sit exam * 6 ‘problem solving’ sessions run by administrators * peer assessment of problem sheets using marking scheme * no quality control! * sheets handed back immediately with peer comments * same lectures, same problems, same marking criteria, same exam, only exam marks counted RESULT * average mark increased from 45% to 80%! what does this tell us about the way students learn?

15. Giving feedback to students to help them learn * set target for return of marked work * try to give feedback as close to the assessment as is possible * connect any grades to the commentary * if there are many comments - summarise the key points * balance positive and negative * indicate what the student needs to do to improve * make further suggestions for follow-up reading * encourage students to evaluate themselves using the same criteria * relate feedback to the assessment criteria

16. Feedback to students on their performance in formative assessment is an issue why? * they may not get any * when they get it it might be too late to help them* it may not help them understand how they can improve * it may not reflect the criteria by which they are judged Some ways of providing written feedback * Comments on assignments / exam scripts * Model answers in handout/bulletin board Feedback forms * strengths / suggestions for improvement * checklist against criteria * checklist against structure/format of assignment * self-assessment questions with answers Use examples in Sue and Trevor Habeshaw & Graham Gibbs Interesting ways to assess your students

17. What principles do you base your assessment on? Examples of principles for assessment 1 The purposes and focus will be clear 2 It is integral to course design 3 What is being assessed, how judgements are made and what is expected of students will be clear 4 Assessment tasks will be valid and manageable 5 Assessment workloads will be realistic 6 It will be free from bias and fair to all students 7 Marking and grading will be consistent 8 It will provide feedback to students to support learning 9 It will provide feedback to teachers to inform them about the effectiveness of their teaching. 10 The assessment process will be quality assured and open to independent peer scrutiny

18. Perspectives on assessment RELIABILITY Concerned with consistency / reproducability of measurement/marking Potential sources of inconsistency * consistency of marking by individual teachers * consistency of marking by different teachers * test-retest reliability * student inconsistency’s ‘good/bad days’ * changes in context e.g. increasing number of students Illustrate using research studies. How might we improve consistency of marking / grading?

19. Perspectives on assessment VALIDITY - a form of truth seeking is assessment fit for purpose ‘appropriateness of assessment tasks’ What assessment methods can be used to assess what types of learning? Paired or small group discussion then build a collective map.

20. Place of assessment in a rational planning model for course design and delivery 3 Intended Learning Outcomes the learning that is likely to result from these steps 2 Objectives the steps to achieve these goals assessment in design 1 Aims purposes/ goals for learning 5 Judgements on whether and how well ILOs have been achieved 4 Teaching, learning and assessment strategies to promote learning

21. An outcomes approach to course design EXPLICIT INTENDED LEARNING OUTCOMES expressed in terms of knowledge, understanding, skills and perhaps values and behaviors. Nature of understanding defined by verbs intended learning learning process actual learning ASSESSMENT ALIGNED TO LEARNING OUTCOMES 1) assessment methods that are valid to enable required learning to be demonstrated 2) criteria that enable learning to be evaluated and show students what is expected THAT ENCOURAGES STUDENTS TO BEHAVE IN WAYS THAT WILL ENABLE THEM TO ACHIEVE THE INTENDED OUTCOMES

22. Constructive alignment Biggs (1999) encourages teachers to think about teaching system what the teacher does to promote and evaluate learning learning system what the student does to learn and demonstrate learning

23. the key to describing learning is in the use of verbs that relate to understanding in a particular context. Biggs SOLO taxonomy - four layers of verbs 1) identify, memorise, do simple routine procedures 2) list, describe, combine, perform skills in sequence 3) analyse, relate, apply, argue/justify, compare/ contrast 4) generalise, synthesise, theorise/ hypothesise, reflect What verbs are you using in your assessment criteria? Are they consistent with stated learning outcomes?

24. Marking/grading models Royce Sadler 1 Grade awarded according to mark ranges grade mark range HD 85-100 D 70-85 C 60-75 P 50-60 PC 40-50 F <40 2 Grading according to qualitative descriptors Grade HD exceptional performance complete and comprehensive understanding, mastery of skills,…. D excellent performance, high level of understanding of subject, development of relevant skills C good performance and level of understanding P satisfactory performance and an adequate understanding of basic subject matter HD - pass with high distinction, D=pass with distinction, C=pass with credit P=pass

25. 3 Grading according to pre-determined distributions grade proportion of students HD 4-6% D 8-12% C 20-30% P 45-55% F 5-15% 4 Grading according to objectives grade main minor objectives objectives achieved achieved HD all all D all most C most most P most some F few/none some Use the engineering or other benchmark statement to illuminate this model

26. Assessment models! Current undgrad. model Normative rather than criterion referenced. Grading scale (100pt) culturally aligned to 5 bands of honours system Standards represent a configuration of learning characteristics in which compensation and trade-offs are a normal occurrence (Sadler 1987) We want to view learning and performance holistically QAA Benchmarking model Criterion referenced Two or three grading bands. Relationship to grading scales and honours classification open to interpretation Standards represent achievement evidenced against specified criteria. Outcomes must be satisfied in full. Compensation not permitted? Requires two stage decision making and judgements about satisfying all minimum criteria

27. What marking frameworks / grading models do you use? What is the marking practice/ culture in your discipline and how does it affect the use of the grading scale? Illustrate different grading profiles in different subjects

28. How does assessment in your module relate to the programme as a whole? Connect to programme specification and curriculum maps 1 2 3 4 5 6 7 8 etc curriculum building blocks Bench marks programme outcomes A B C D E F G H 1 2 3 4 5 6 7 8 etc TPA TPA P TPA P TPA TPA TPA TPA TPA P TPA TPA P TPA TPA TPA TPA PA TPA PA TPA TPA TPA TPA P P T=taught P= developed through practise A=assessed

29. Practical Assessment Design Learning Activity Step 1 30mins gps of 3 using example provided 1 Design and justify an assessment strategy that you feel would be valid 2 Develop examples of criteria that you would use to evaluate the achievement of learning outcomes 3 Identify the principles that you have used in this exercise 4 Identify any issues that the process raises Produce a flip chart poster Step 2 5 develop criteria for self evaluation of poster and evaluate/grade your own 6 evaluate the other posters using the criteria and put grade on yellow post-it 7 group discussion - what did we learn?

30. Useful resources for assessment George Brown with Joanna Bull and Malcolm Pendlebury Assessing student learning in higher education, Routledge. Sue and Trevor Habeshaw & Graham Gibbs Interesting ways to assess your students Dai Hounsell, Mary McColloch and mary Scott The ASSHE Inventory Changing Assessment Practices in Scottish Higher Education on-line at www.ltsn.ac.uk/genericcentre Many examples organised by subject/theme. Learning and Teaching Support Network Subject Centre web sites http://www.ltsn.ac/subject Learning and Teaching Support Network Generic Centre web site http://www.ltsn.ac/genericcentre (Project - Assessment)