Assessment Matters for Diversity

1. Assessment Matters for Diversity Ginny Saich DAICE

2. Overview • The H.E. Context for Assessment • The Stirling Context for Assessment • What is assessment? - What are its characteristics? • Why assess? - What is its purpose? • What is the effect of assessment? • How do we assess? • What are common problems with assessment? • What trends are apparent in assessment? • How do we grade? • What are the characteristics of good feedback? • Good practice in assessment

3. H.E. Context of Assessment Recent legislation including: • the disability discrimination act (DDA Part IV) • the race relations amendment act places an onus on university’s to embrace inclusive practices (including for learning and teaching) Implications? Concerns about plagiarism with increasing prevalence of online resources Assessment is one of the Quality Assurance Agency’s (QAA) enhancement themes for 2003/4.

4. Stirling Context of Assessment • The University of Stirling’s Learning, Teaching and Quality Enhancement Strategy: 2003-2005 has the following action items relating to assessment for implementation during 2003/4: • Instigate a critical review of assessment practices with reference to the relevant section of the QAA Code of Practice and in light of the Quality Enhancement engagement of 2003-4 • Investigate and report on the implications for Learning and Teaching of Disability, Special Educational Needs, and Racial Equality legislation, with recommendations for action • Monitor progress on embedding the accessible curriculum • Report on use of diagnostic testing to identify the needs of students • Report on effectiveness of the Special Academic Arrangements Panel after its first year in operation • Report on use of JISC Plagiarism Detection Service pilot

5. Preventing and Detecting Plagiarism Good Practice Guide from Oxford Brookes contains recommendations on preventing plagiarism (see http://www.northumbria.ac.uk/jiscpas - web site of the JISC Plagiarism Advisory Service) JISC Plagiarism Detection Service - detects plagiarism of online resources (and collusion) - to be piloted at Stirling in Spring 2004 Prevention is more effective than detection!

6. Licence to proceed • Ranking order • Improving learning Why Assess? • to grade or mark • to help learning • to indicateperformance The Student • to pass or fail • to allow to proceed • to licence • to predict success • to motivate • to give feedback • to detect strengths & weaknesses • to select for further training • for public relations • to meet expectations

7. Purpose of Assessment

8. What is Valued? If we wish to discover the truth about an educational system, we must look into its assessment procedure. What student qualities and achievements are actively valued and rewarded by the system? Rowntree (1987) Assessing Students London: Kogan Page, 2nd edition, p.1

9. The Effects of Assessment on Learning Assessment influences student learning eg. Surface learning is induced by:- Heavy workloads, assessment by unseen exam alone, multiple choice questions that test only recall. Deep learning is induced by:- Reasonable workloads, some choice, a variety of assessment tasks, project work, multiple choice questions that test understanding (Marton and Säljö, 1976: surface and deep learning) The kinds of assessments set are important!

10. If you want to change the mode of learning then change the assessment procedure. Brown, G., Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in Higher Education London: Routledge

11. Assessment is ….. a “systematic basis for making inferences about the learning and development of students … the process of defining, selecting, designing, collecting, analysing, interpreting and using information to increase student’s learning and development” (Erwin, 1991: 15)

12. What is Assessment? Sample Draw inference Estimate worth ? Feedback Formative assessment Summative assessment

13. “Continuous assessment meant that I knew how I was getting on, rather then waiting until the end of the session, then failing because I didn’t know what was required.” “I failed the first one but I don’t really know why. All I got were a lot of crosses and a few ticks and some exclamation marks.” Student Experiences of Assessment

14. Functions of Assessment

15. In practice....... So-called formative assessment is often multiple summative assessment - and little feedback is given.

16. Characteristics of Assessment Efficiency - saves you time Validity - effectively tests what you intended ie. fitting your learning outcomes etc. Reliability - provides consistency Enablement - develops understanding and expertise The difficulty lies in achieving all these characteristics together.

17. What is effective assessment? • It assesses what is taught and learnt • It emphasises deep learning • It measures skills and transfer • It is efficient for lecturers • It is a reasonable workload for students Do you have an assessment strategy? Does your department?

18. Reliability of Assessment Reliable assessment strategy  same results if repeated or if marked by a different marker Even apparently objective assessment may have a highly subjective component.

19. Objectivity 1 A candidate was required, for 10 marks, to multiply 4567 by 23. Below is the candidate’s answer compared with the correct answer. Candidate 4567 Correct: 4567 x 23x 23 13801 13701 91330 91340 105131 105041 What mark would you award?

20. Objectivity 2 Four markers comments: 1. This candidate cannot do simple long multiplication. Mark = 0 2. Layout and general procedure correct. Accuracy missing. Mark = 4 3. Errors in multiplication, but most correct. Addition correct. Mark = 5 4. 12 out of 14 arithmetical operations carried out correctly. Mark = 8

21. Validity of Assessment A valid assessment is trustworthy  tests what it is supposed to test. A test may be technically highly reliable but not valid eg. a test for critical thinking that asks questions requiring only memorisation

22. Inclusive Assessment Planning for inclusiveness in assessment means: • Ensuring that one group of students are not unduly advantaged or disadvantaged by the selected mode of assessment. • Ensuring that all students have the means to acquire the assumed knowledge and skill to successfully complete the course. • Being clear about the essential learning outcomes to be attained and willing to consider alternative means for student to demonstrate the attainment of them Stirling has a developing set of policies and guidelines for supporting students with disabilities. (Further information is available from departmental disability officers or the University’s disability officer, Catriona Mowatt in SISS.)

23. Levels of Objectives (Bloom’s Taxonomy) 1. Knowledge 2. Comprehension 3. Application 4. Analysis 5. Synthesis 6. Evaluation

24. Assessment Terms Methods Essays Problem sheets Unseen prose Lab reports Presentations Projects Group projects Posters Work-based learning Written exams Vivas MCQ's MRI's • Instruments • Implicit criteria • Global • Explicit criteria • Criteria reference • grading • Specific criteria • Marking scheme • Dimensions • Rating schedules • Checklists Sources Lecturer Other tutors Postgraduate tutors Demonstrators Student self Student peers Employers Mentors External examiner

25. Criterion referenced Assessment

26. Norm referenced Assessment

27. Assessment methods: what are they good for? Essays - for analysis, synthesis and creative perspectives MCQs - to sample knowledge and understanding of principles Problems - to sample problem solving skills! Practicals - to sample practical skills! Lab sheets - to sample report writing, understanding of procedures, data analysis and interpretation • Projects - as for lab sheets plus deeper understanding, • literature review (?), reflective learning (?), • time management • Learning - reflective and experiential learning • Logs

28. Common Weaknesses • Overload of staff and students • Too many assignments set with the same deadline • Lack of alignment between assessment and teaching • methods and/or learning outcomes and course aims • Poor assessment design  plagiarism/surface learning • Wide variations • in unit demands • in marking across units • in marking criteria/guidance Inadequate feedback (quality/quantity/timeliness) to students

29. AligningAssessment Aims Intended Learning Outcomes Methods of Learning Assessment methods and tasks Criteria Marking Feedback

30. An emerging model • Aims • Outcomes • Learning Methods External Influences on Assessment • Quality Management: • Standards • Benchmarking • Code of Practice • Prog. specification. • University: • Policy • Strategy • Professional Bodies Assessment Criteria Feedback Marking

31. Trends in assessment Tutor led Tests, Examinations Peer marking, Peer assessment Presentations Group work Projects, Group projects Student Centred Choice of assignments, dissertations Student led

32. Marking • Who does it? • What points of reference are used? • Types of assessment criteria • Marking schemes • Marking strategies • Factors influencing marks awarded

33. Feedback • Problems with feedback • Characteristics of good feedback • The feedback ‘sandwich’ • Providing feedback efficiently • Care required…….

34. Good Practice in Assessment Consider: 1. What kinds of things do you want them to learn? 2. What learning opportunities will be provided? 3. What assessment tasks will be set? 4.What criteria will be used?