Assessment Resource Comparator

1. Learning and Teaching Institute Integrating Technology Enhanced Assessment Methods for student support and self-regulation – The ITEAM Project Assessment Resource Comparator Rationale Assessment is important for learning Good assessment supports students’ learning Staff effortshould be recognised Timely feedback is crucial Technology enhanced solutions can be beneficial • Outline of the design • Types of assessments included • Essay • Exam • Objective testing • Group work • OSCE (Objective Structure Clinical Exams) • Elements considered • Academic and professional staff effort was considered at each of the following stages, which are the same for all types of assessments: • Lecturer preparatory stage • Student preparatory stage • The assessment and feedback stage • Every stage includes a variety of activities, which may vary for different types of assessment. • Design tool: Microsoft Excel Summative essay The essay and variations of the essay is frequently used as an assessment task and it is considered to test higher order thinking skills with each discipline requiring subtleties. While there are advantages to this task, there are also disadvantages. In the resource calculator for the summative essay task figure 1, the activities were set out and the time it takes to complete each part of the process considered. Fig 1. The resource calculator for a summative essay. Details of the activities (left) and a graph of staff time versus student number (right) Time estimated for each activity can be adjusted using the sliding bar with ‘Minutes’. Task activities marked with a *(star) are student number related. This number can adjust as required. It can be seen that with a cohort of 300 students, academics and administrative staff would have to spend over 9000 minutes, which is just above 20 working days to execute this assessment task. Objective testing Objective testing includes multiple choice questions (MCQ) and multiple response questions (MRQ), which can be either paper based, and can be marked using optical marking reader (OMR), or computer based, for example, QuestionMark Perception (QMP), or using electronic voting system (EVS). Time spent on each activity can be adjusted according to the nature of the assessment (QMP, EVS, OMR) and in some cases if it was applicable the student number can be adjusted too. See Figure 2 for details. Fig 1. The resource calculator for a summative essay. Details of the activities (left) and a graph of staff time versus student number (right) It was noted that with a class of 300 students, an estimation of 1700 minutes (3.9 days) staff time was associated with the objective testing. • Assessment comparator • The comparison of staff effort between summative essay and objective testing is illustrated below in figure 3: • Fig 3. Comparison of staff time versus student number between summative essay and objective testing. Large group of students (left) and relative small group of students (right). • The transition point where the two types of assessment cross over in staff effort was at a relatively low student number. It can be seen from the right side of the graph that with this particular example, objective test uses lower staff time when the student number is above 14. Discussion This assessment resource comparator has explored the efficiency of the assessment process from the academic perspective. When considered against the benefits to students it can be readily identified that some types of assessment can be beneficial to students due to the immediate nature of feedback. Well-constructed objective test are recognised as an appropriate tool in testing higher level cognition [1]. The challenge in terms of efficiency in this type of test is the development of good questions. It should be used as reference and guideline when design assessment according to the assessment for learning principles [2], not as a driving force or as a management tool. The next stage of development will be expanding this tool to encapsulate other types of assessment that are used in Higher Education for example peer assessment. Following on from this initial phase of design, the student workload in term of their effort in the assessment process needs to be considered. References [1] Schuwirth, L.W.T., & van der Vleuten C.P.M. (2003) ABC of learning and teaching in medicine. British Medical Journal 326 (7390) 643-645. [2] Assessment for Learning Principles (2012) University of Hertfordshire. http://www.studynet2.herts.ac.uk/intranet/lti.nsf/0/9E0C70AE221C652180257AAF0044E316/$FILE/UH%20Assessment%20Guidance%20for%20publication-%20Web%20Version-PDF.pdf Fang Lou (F.Lou@herts.ac.uk) Liz Gormley-Fleming (e.gormley-fleming@herts.ac.uk)