120 likes | 294 Views
Designing for online assessment. Getting started Ingrid Nix and Ali Wyllie (Lecturers in Teaching and Learning) 2 May 2007. Outline. Building on experience Building a process for production Designing to motivate Providing feedback Learner pathways (including statistics)
E N D
Designing for online assessment Getting started Ingrid Nix and Ali Wyllie (Lecturers in Teaching and Learning) 2 May 2007
Outline • Building on experience • Building a process for production • Designing to motivate • Providing feedback • Learner pathways (including statistics) • Looking ahead
Building on experience • Previous experience in OU – science, maths, OUBS using different (computer based) systems • History of paper-based CMAs • OpenMark sophisticated functionality, while Moodle catch-up • S151 Maths (summative online use) • Previous experience in own work: • used to designing exercises incl. computer-based (drawing on EFL language teaching background) • familiarity with range of exercise types e.g. text input, matching, multiple choice, use of visuals to convey meaning, etc. • interest in instructional design, use of media, student experience • awareness of needs of HSC students: supported, incremental
Building a process for production • Driver: assessing ICT skills throughout SW degree at levels 1 - 3 • Word templates/ proformas for each question type • provide framework of requirements [see handout], feedback structures • authored by academic, built by programmer (+ designer input) • learning curve for all team, constantly improving on previous Qs • build bank of questions e.g. use of substitutable datasets (weighting affects % to be renewed each year if summative, per course or randomised per student individually) • Pedantic criticism is good! • edited while in Word (preferably subject expert) critiquing logic • developmentally test to find anything ‘grey’ before students do • course-wide adjustments can be made before scores released • (NB. difficulty keeping qs. up-to-date where changes occur regularly e.g. ICT skills, government policies, etc.)
Designing to motivate • Typology created (framework of qs types) [see handout] to give range of approaches and pathway from concepts (knowledge) to skills (application) • thematically linked clusters of questions • linked sequence of questions to lead through processes • creating a learnable continuum of qs types to self-select from (see LINA project) • Building confidence through: • engaging designs, usability • relevant context • feedback to encourage next attempt • what makes Qs easy or difficult: e.g. # of options, quantity of info
Providing feedback • Examples of approaches: • Generic across a question • Customised per option selected • Different technical approaches (ticks and crosses, wrong answers leap out) • Formative: generous, role is to support and teach, encourage to revisit • Summative: provide prompts to remind of previous learning • Future consideration: Confidence based marking • 1 self-assessment before seeing options • 2 self-assessment after seeing options but before answering
Examples of questions and Feedback • [Show real examples online]
Learner pathways • Formative CMAs: different student uses of questions • Same day session repeating question • Subsequent days return to redo until get full marks • Repeat despite full marks • Statistics • [see handout]
Looking ahead • COLMSCT CETL amassing OU experience, developing expertise • Teaching Fellows: • LINA project (Learning through Interactivity in Assessment) • integrating media e.g. audios • continuum of questions enabling student to self-assess what level question to attempt • ‘confidence-based marking’ – exploring its benefits • MAZE project • iCMAs within i-skills