1 / 51

eAssessment innovations in HSC

eAssessment innovations in HSC. Ali Wyllie, Ingrid Nix, Verina Waights, Mick Jones, Caroline Malone COLMSCT CETL Teaching Fellows, HSC F aculty briefing 24th February 2010. Outline. Emerging eAssessment practice: CETL research projects emerging from Faculty programmes

baby
Download Presentation

eAssessment innovations in HSC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. eAssessment innovations in HSC Ali Wyllie, Ingrid Nix, Verina Waights, Mick Jones, Caroline Malone COLMSCT CETL Teaching Fellows, HSC Faculty briefing 24th February 2010

  2. Outline Emerging eAssessment practice: • CETL research projects emerging from Faculty programmes • Innovation in learning design in LINA and CDM projects Highlighting innovation in Faculty programmes: • K101 • Social Work • Nursing Looking to the future: • Evaluation • Where do we go from here

  3. Emerging eAssessment practice Course production & assessment • Social work – ECDL equivalent • Pre-Registration Nursing Programme (PRNP) – Anatomy, physiology and medication calculations Innovation in learning design: Engaging students interest, relating Qs to practice, offering choice. CETL research projects: • LINA (Learning with Interactive Assessment) • CDM (Clinical Decision-Making)

  4. LINA: Extended narratives in a practice context • Review ICT skills • Choice of questions within sequences • Narrative within sequences: e.g. Sue and Claire discuss how to create a Care Plan or find a document (audios) • Contextualised by variety of rich media: audios, animations, images, tables, texts • Confidence-indicator tool promotes reflection • Learning log captures progress and reflective comments http://learn.open.ac.uk/course/view.php?id=4348

  5. LINA: Sequence, narrative, & variety of media

  6. Confidence Indicator tool

  7. Learning log

  8. Learning Design Features: LINA • Choice in question sequences • Narrative within sequences • Variety of media: audios, animations, images, tables, texts • Tools to promote reflection: Learning log, Confidence indicator tool • Progressive feedback at each of 3 attempts.

  9. Complex decision-making • Context: Student nurses develop practice-based clinical decision-making (CDM) strategies • Narrative case study of wound care - ulcer • Contextualised by rich media: audios, images, charts, texts • Reflective log to capture decisions, reasoning and reflections • Multiple level options with alternative final outcomes create intelligent tutoring maze evolving over time • 2 possible attempts with scoring and detailed feedback at end http://learn.open.ac.uk/mod/resourcepage/view.php?id=79337&superfluous=357

  10. Clinical Decision-Making (CDM)- first consultation

  11. CDM – next consultation

  12. CDM – variety of possible outcomes

  13. Feedback in Reflective Log

  14. Learning Design Features: CDM • Experiential learning with student options scaffolded • Choice in decisions • Case study narrative • Variety of media: audios, animations, images, tables, texts • Tools to promote reflection: Reflective log, score, visual narrative • Detailed feedback at final attempt.

  15. Highlighting innovation in Faculty programmes: K101 • Ingrid: K101 and Social Work

  16. Developing number skills on K101 ‘learning a way of talking about the world in numerical terms’ ‘The purpose of each quiz – to get your mind tuned in, so that you are ready to listen to the way tables & charts are talked about and the way conclusions are drawn from them.’ (K101 course text, Bl1, Unit 4 Activity 3)

  17. Number skills topics

  18. Formative – DVD-ROM activity Context links to quiz link to quiz

  19. Formative – DVD-ROM activity Correct answer triggers feedback (audio + animations explaining the answer)

  20. Audio Feedback: ‘…What do the numbers on the left tell you? They go up in twenties and the red line …’ Play demo Animation enhances explanation

  21. Summative online Moodle Quiz

  22. Social work degree rationale • Increase assessment points to cover curriculum areas • Needed esp. for ICT skills due to ECDL requirements • Avoid need for tutor marking • enabling AL focus on other areas of teaching • Self-directed learning (diagnostic self-assessment) • useful where find variations in skills/ knowledge levels so students can catch up at own speed

  23. Programme-wide development K113 K216 K315

  24. Adaptive feedback for 3 attempts

  25. Identifying information • Level One – Social work (K113)

  26. Interpreting information • Level One – Social work (K113)

  27. Matching or labelling… • Level One – Social work (K113)

  28. Sequences: constructing a reference • Level Two – Social work (K216)

  29. Entering text information: reference details • Level Two – Social work (K216)

  30. Entering information – numeric (spreadsheet formula) • Level Two – Social work (K216)

  31. Recognising and identifying principles • Identify which principles apply to which sections (i, ii, iii, iv) of the scenario … • …relating to the experience of Sheri, a newly qualified social worker. • Level 3 – Social Work (K315)

  32. Highlighting innovation in Faculty programmes: Nursing • Verina : Pre-Registration Nursing Programme

  33. Evaluation • Are the students engaging? • Does e-Assessment work?

  34. Are students engaging? • HOW DO WE KNOW? • Gradebook – to tell us about the student experience/ actions • CTs decide who to make it available to: • Students (individual view) • Tutors (tutor group view + cohort averages) • CTs (course wide view of TGs + cohort averages + sort functions to display information + generate reports)

  35. Gradebook: for students • Benefits: • Enables students to review their use of iCMAs • e.g. Answers, marks, times, dates

  36. Gradebook: for tutors Lack of score shows student needs chasing for next iCMA Average across 3 iCMAs After cut-off a score shows student has submitted Compare TG average with cohort average

  37. Gradebook: further information • VLE choices website (co-ordinated by Janet Macdonald) • http://learn.open.ac.uk/site/vle-choices • Teaching and facilitation > Section 3.3 Feedback on learning • Fiona Barnes on K101 AL use (PDF) • Ingrid Nix on CT considerations (Audio )

  38. Course perspective • The CT will be involved: • At score approval stage • At end of presentation(s)

  39. Course perspective • K101 09B • - iCMA grades distributed across iCMA42 • - How well each question helped sort strong students from weak

  40. Cohort averages viewed in Gradebook K101 09B K101 08J

  41. Cross-presentation comparisons

  42. Students not engaging – K101

  43. Student patterns of engagement • Sally Jordan (Science Faculty) analysis of student engagement: • - last minute activity just before cut off (S104) - does easy questions first, returns several times to do other Qs (S104). - completes when directed by course units after each chapter (S154)

  44. Browsing versus engaging • Sally Jordan (Science Faculty) analysis of student engagement: • Student responses to the same question • Grey = blank responses; green = correct =;red = incorrect. • Where the same colour is shown at 1st/2nd/3rd attempt, an identical student response has been entered. • diagnostic formative summative

  45. Looking to the future • Mick Jones and Caroline Malone

  46. Cost and time • Our experience with time-consuming and expensive production and maintenance of iCMAs has put us off, however: • - they have worked well with students • processes and guidance has improved significantly (led by HSC!) • - there are now exemplars and 'templates' to follow • - academic expertise is growing.

  47. Course design • With the current course design imperatives iCMAs and online quizzes should have a place in SOME courses and programmes as: • - part of online activity sequences • - increased online interactivity • - need to increase formative assessment and effective scaffolding of learning • - opportunity to gather rapid feedback.

  48. Process improvement • We need to continue with process improvement, design guidelines and staff development: • - the power lies in increased interaction and more frequent feedback • - relatively simple approaches can work well • - avoid unnecessary complexity • - but they do need higher standards of precision and care than most learning materials.

  49. Other approaches • Some approaches we haven't really tried yet: • - commissioning ALs, expert staff or appropriate consultants to produce • - ALs build up a question bank over time • - collaborate with other institutions • - use externally produced questions from publishers/commercial companies.

  50. To find out more • Learning about eAssessment (Phil Butcher) • http://learn.open.ac.uk/course/view.php?name=ICMA • E-Assessment section of Media Development wiki • http://intranet4.open.ac.uk/wikis/MDWG-wiki/EAssessment • VLE Choices website • http://learn.open.ac.uk/site/vle-choices

More Related