Download
1 / 80
800 likes | 1.01k Views
UK eUniversities Worldwide Limited Derek Morrison 25 November 2003. Assessment Menu [ short ]. What do we assess?. Is assessment in e-learning different?. Assessment alignment / integration. Examples. Finish. War in the Modern World. Assessment Examples. Leeds/Soton GIS. Bath’s GroupLog.
E N D
UK eUniversities Worldwide LimitedDerek Morrison25 November 2003
Assessment Menu [ short ] What do we assess? Is assessment in e-learning different? Assessment alignment / integration Examples Finish
War in the Modern World Assessment Examples Leeds/Soton GIS Bath’s GroupLog Objective test examples LearnWright WIMBA ePortfolio examples
Knowledge What should we / do we assess? How do we assess?
What should we / do we assess? How do we assess?
What should we / do we assess? How do we assess? “… examinations are very poor predictors of any subsequent performance, such as success at work … first degree results explain less than 10% of the variance in postgraduate performance” (Warren 1971) in Gibbs G & Simpson C [publication pending]
What should we / do we assess? How do we assess? “… the kind of learning that coursework involves has long term consequences while the kind of learning involved in revision for exams does not … in experimental studies in which students have either studied exam-based or assignment-based courses, the quality of the learning has been shown to be higher in the assignment-based courses.” (Gibbs G & Simpson C [publication pending])
Is assessment in e-learning different? • There is an assumption that technological literacy and other prerequisites will be met • Students are not necessarily in the same time zone/place • Cultural influences and expectations • The informal assessment practices of face-to-face are impossible • Processes need to be clearly defined. • High quality feedback is critical. • How do we design a range of indicators or evidence gathering processes for online assessment to measure how students solve ill-defined problems, transfer learning, and achieve higher order outcomes (classification, interpretation, abstraction, rule application, judgement > analysis, synthesis, evaluation?) “Patterns indicate that freshmen enter ENG105 with relatively high word-processing, editing, email, and web research skills, but with lower skills in online collaboration, multi-media presentation, web analysis, and web design.” (Norris, JM (2003), Assessment of Students in Electronically-Mediated Settings, Northern Arizona University, http://www4.nau.edu/assessment/main/electronic/E-LearningAssessmentReport2002-2003.pdf ) Should assessment (like learning itself) be a process or an event?
Assessment Outcomes How integrated/aligned/blended are your parts? “Key to the outcomes approach to assessment is the use of ‘authentic assessment’. This approach stresses creating assignments and assessments that simulate as much as possible the situations in which students would make use of the knowledge, skills and values emphasized in the course (Battersby M, So What’s a Learning Outcome Anyway?, Centre for Curriculum, Transfer and Technology http://www.c2t2.ca/GoodPractice/Exchange/curriculum-frameworks/sowhtsa.html “… the standard written tests will seldom be appropriate to assess outcomes – because what the students will do with the knowledge and skills they are learning is not usually writing tests or essays – unless we believe that the point of the course is primarily to succeed in subsequent courses.” (ibid)
Assessment Outcomes How integrated/aligned/blended are your parts? “The more outcomes which must be passed for the course to be passed … the more magnified are any unreliabilities in assessment. And, from the student’s perspective, each distinct essential learning outcome is an additional opportunity to fail.” (Baume D, 2001, A Briefing on Assessment of Portfolios p15, LTSN Generic Centre. http://www.ltsn.ac.uk/application.asp?app=resources.asp&process=full_record§ion=generic&id=6) “… learning is rarely the linear process of movement towards immovable goals that the use of learning outcomes can sometimes suggest.” (Baume D, 2001, A Briefing on Assessment of Portfolios p18, LTSN Generic Centre. http://www.ltsn.ac.uk/application.asp?app=resources.asp&process=full_record§ion=generic&id=6)
Assessment Outcomes Resources Events Activities How integrated/aligned/blended are your parts? “Many of the flaws in existing e-learning programs stem from lack of alignment.” (Reeves TC, Aggen WD, Enhancing E-Learning Assessment and Evaluation Strategies, http://www.learnwright.com/ELearn 2002 Reeves Aggen Paper.pdf ) “Assessment if most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.” (American Association for Higher Education, http://www.aahe.org/assessment/principl.htm)
Assessment Outcomes Resources Events Activities How integrated/aligned/blended are your parts? What’s the result of a lack of alignment/integration? “… academic developers are often disappointed by the degree to which learners actually use these [electronic journals, real time databases, and online discussion forums] and similar online resources.” (Reeves TC, Aggen WD, Enhancing E-Learning Assessment and Evaluation Strategies p4, http://www.learnwright.com/ELearn 2002 Reeves Aggen Paper.pdf )
Assessment Outcomes Resources Events Activities How integrated/aligned/blended are your parts? “However, learners may simply perceive that these resources have little relationship with assessments in the course (such as writing a paper or taking a final exam) and so they wisely choose to target their cognitive efforts on those learning activities that they perceive as more clearly aligned with attaining the highest scores on assessments.” (Reeves TC, Aggen WD, Enhancing E-Learning Assessment and Evaluation Strategies p4, http://www.learnwright.com/ELearn 2002 Reeves Aggen Paper.pdf)
Assessment Outcomes Resources Events Activities How integrated/aligned/blended are your parts? “… students in the UK allocate their time largely to assessed tasks and that this becomes a more narrow focus over time as they become more experienced students, allocating as little as 5% of their time to unassessed study tasks by year three.” (Gibbs G & Simpson C [ publication pending]) • ASSESSMENT DRIVES STUDENT LEARNING & IS CENTRAL TO THEIR LEARNING EXPERIENCE • Becker et al (1968) • Snyder (1971) – the ‘hidden curriculum’ • Miller & Parlett (1974) - cues and rewards • Rowntree (1987) • Ramsden (1992) • MacFarlane (1992) • Biggs (1999) • McConnell (2002) The ‘cue conscious’ got better degree results than the cue-deaf’. So how can we make best use of this driver?
Assessment Outcomes Resources Events Activities How integrated/aligned/blended are your parts? • PARTICIPATORY / CONTRIBUTIONS ORIENTED LEARNING • Assessment activities: • Recognise students as contributors • Involve opportunities for students to communicate, contribute to, and participate in an online community • Reflect the status of students as contributors to course content, and creators of new knowledge products. McLoughlin C, Luca J (2003), Quality in Online Delivery: What does it mean for assessment in e-learning environments?, Eric Clearinghouse on Information & Technology, March 2003, http://www.ericit.org/fulltext/IR021479.pdf
Group Exercise [20% of module mark] • “In this learning object you will deepen your knowledge of the Cuban Missile Crisis by playing the role of a number of historical figures – American, Soviet, Cuban, British, and German …You depend greatly on the advice of your cabinet but the final call is yours. Victory conditions: • Avoid nuclear war, 100 points • Force the Soviets to withdraw missiles from Cuba, 80 points • Depose Castro, 80 points • Cause the Soviets to give up security commitment to Cuba, 50 points” Extract from Group Activity: Simulation of the Cuban Missile Crisis, War in the Modern World, Kings College London Assessment example: King’s College London – War in the Modern World Two Short Answer Exercises of the Student’s Own Choice[ 2 x 15% of module mark] “The Student will draft a 750 word answer to the question set for the chosen unit, will post that draft to a discussion forum for peer review, will defend the answer in the forum and will then submit a revised answer for assessment, along with a log and reflective commentary on the peer review process.”
Assessment in e-learning resources - 1 http://www.qaa.ac.uk/public/dlg/contents.htm http://kerlins.net/bobbi/education/onlineed/24benchmarks.html http://www.ihep.com/Pubs/PDF/Quality.pdf or http://www.nea.org/he/abouthe/Quality.pdf http://assessment.cetis.ac.uk/ http://bsonline.techindex.co.uk/ (BS 7988:2002) http://www.caacentre.ac.uk/ http://www.ltsn.ac.uk/genericcentre/ Scottish Centre for Research into On-Line Learning and Assessment (SCROLLA) http://www.scrolla.ac.uk/about.html
Assessment in e-learning resources - 2 Computer-Aided Assessment in Mathematics http://ltsn.mathstore.gla.ac.uk/index.asp?cat=137 Scottish Computer Assisted Assessment Network http://www.scaan.ac.uk/ Tools for Interoperable Assessment (JISC X$ http://www.jisc.ac.uk/index.cfm?name=project_toia) McLoughlin C, Luca J (2003), Quality in Online Delivery: What does it mean for assessment in e-learning environments?, Eric Clearinghouse on Information & Technology, March 2003 http://www.ericit.org/fulltext/IR021479.pdf McConnell D, (2002), Collaborative Assessment as a Learning Event in E-learning Environments, Computer Support for Collaborative Learning (CSCL) 2002, Boulder, Colorado http://newmedia.colorado.edu/cscl/54.pdf
Evaluation and redevelopment • Capture participant feedback • from specifiers, authors, tutors, reviewers, etc • Build evaluation into course • questionnaires • interviews • email analysis • behaviour metrics • assignment performance • Research Centre can advise • Annual evaluation report and action plan required by UKeU • Allow 20% of initial development cost for first annual revision (minimum)
Evaluation Synthesis Analysis Application Comprehension Knowledge Objective tests and higher level learning? “The capacity of objective tests to assess a side range of learning is often underestimated … It is commonly assumed that objective tests are only useful for examining the first three or four levels defined by Bloom et al (1956). However, some educationalists including McBeath (1992) suggest that all six levels can be tested using objective test questions.” (Bull J, McKenna C et al, 2001, Blueprint for Computer-assisted Assessment pp 14-21, CAA Centre, http://caacentre.ac.uk )
