1 / 35

CREATING ON-LINE QUIZZES: DO YOURS MAKE THE GRADE

Overview. Part 1CMS and the online quiz review of the literatureOn-line quiz options UW ACE / AngelPart 2 Features of good question designHands on exercise . 2. OLA Super Conference 2008: Creating On-line Quizzes. Advantages/Disadvantages . Advantages :Easy (relatively) to create and dis

brant
Download Presentation

CREATING ON-LINE QUIZZES: DO YOURS MAKE THE GRADE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. CREATING ON-LINE QUIZZES: DO YOURS MAKE THE GRADE? Jackie Stapleton Sandra Keys University of Waterloo January 30, 2008

    2. Overview Part 1 CMS and the online quiz – review of the literature On-line quiz options – UW ACE / Angel Part 2 Features of good question design Hands on exercise 2 OLA Super Conference 2008: Creating On-line Quizzes

    3. Advantages/Disadvantages Advantages : Easy (relatively) to create and distribute in CMS environment Feedback – instant, automatic, consistent, Digital format preferred by new generation of students/active learners (Johnson, GM, 2006) Disadvantages: Lack of quiz security Surface understanding assessment only Technical problems Johnson, G. M., & Johnson, J. A. (2006). Learning style and preference for online learning support: Individual quizzes versus study groups. 18th Annual World Conference on Educational Multimedia, Hypermedia, and Telecommunications, Orlando, FL. 1861-1868. 3 OLA Super Conference 2008: Creating On-line Quizzes Advantages – distance education, large class sizes It is up to you to decide if the online quiz meets your assessment needs. We hope this presentation will make you aware of what is available in the CMS environment and provide suggestions to improve the quality of your quiz questions. Advantages – distance education, large class sizes It is up to you to decide if the online quiz meets your assessment needs. We hope this presentation will make you aware of what is available in the CMS environment and provide suggestions to improve the quality of your quiz questions.

    4. What does the literature say? The next series of slides will highlight some interesting case studies and outcomes of research studies in the library and education literature. 4 OLA Super Conference 2008: Creating On-line Quizzes With the introduction of CMS systems in the teaching environment, many libraries are taking advantage of this delivery mechanism for delivery of instruction and resources into the classroom Many articles discuss using an online quiz as an assessment tool, both in the education and library literature Whether or not it is a good thing, it is so easy to create and so accessible, it is being utilized With the introduction of CMS systems in the teaching environment, many libraries are taking advantage of this delivery mechanism for delivery of instruction and resources into the classroom Many articles discuss using an online quiz as an assessment tool, both in the education and library literature Whether or not it is a good thing, it is so easy to create and so accessible, it is being utilized

    5. Assess beyond surface learning Dr. Diana Quinn and Ian Reid, University of South Australia “Quizzes can be developed in ways that they play less of an assessment role, and more of a teaching role. “ Provide an outline of several quiz designs which claim to: Support personal construction of understanding Simulate group feedback Promote self assessment Quinn, D and Reid, I. (2003). Using innovative online quizzes to assist learning, Accessed on October 21, 2007 at http://ausweb.scu.edu.au/aw03/papers/quinn/paper.html. . 5 OLA Super Conference 2008: Creating On-line Quizzes Dr. Diana Quinn and Lecturer, Learning Connection Magill Mr. Ian Reid , Coordinator Online Services, Flexible Learning Centre [University of South Australia Devised different types of quizzes with the intention to move beyond the surface learning which is typically thought to result from multiple choice question quizzes – multiple choice questions in which there are no ‘right’ or ‘wrong’ answers identified. Instead, answers are used to present multiple viewpoints – comprehensive, constructive feedback is provided for each answer or – simulate being part of a group, social learning and role play. Online quiz is constructed to allow individuals to participate in mock role play activities – very detailed feedback Many example quizzes provided. The researchers claim that these quizzes are well suited to the examination of ethical and social issues. May be applicable for librarians using quizzes to address academic integrity issues. Dr. Diana Quinn and Lecturer, Learning Connection Magill Mr. Ian Reid , Coordinator Online Services, Flexible Learning Centre [University of South Australia Devised different types of quizzes with the intention to move beyond the surface learning which is typically thought to result from multiple choice question quizzes – multiple choice questions in which there are no ‘right’ or ‘wrong’ answers identified. Instead, answers are used to present multiple viewpoints – comprehensive, constructive feedback is provided for each answer or – simulate being part of a group, social learning and role play. Online quiz is constructed to allow individuals to participate in mock role play activities – very detailed feedback Many example quizzes provided. The researchers claim that these quizzes are well suited to the examination of ethical and social issues. May be applicable for librarians using quizzes to address academic integrity issues.

    6. CMS and other quiz software: RH Locklin and PR Howell, Pennsylvania State University Used Angel for practice and pre test quizzes, Questionmark for end of unit quizzes Open book vs closed book quizzes (different delivery options) Student satisfaction survey revealed: 80% of students preferred open book quizzes over closed book exams Majority believed that they would obtain a higher grade with an open book format (this did not prove to be true) 90% of students claimed to read the feedback Locklin, R. H., & Howell, P. R. (2005). The design, implementation and assessment of an on-line, open-book quizzing environment for an introductory materials science course. Materials Research Society Symposium Proceedings, 909, 23-41. 6 OLA Super Conference 2008: Creating On-line Quizzes Online, introductory materials science course, for non science majors First part of study: You don’t need a CMS system to create an online quiz. There are many other quiz design software or even using html and FLASH is an option. These researchers used the CMS system for practice and pre test but used a commercial quiz design software for the graded tests. They were still able to automate the transfer of grades to the CMS gradebook. Second part of study looked at student satisfaction with open book format Students satisfaction survey reveals: prefer open book format open book format provided same grades as previous years closed book format like feedback – 90% claim to read it Positive result for instructors using online quiz Online, introductory materials science course, for non science majors First part of study: You don’t need a CMS system to create an online quiz. There are many other quiz design software or even using html and FLASH is an option. These researchers used the CMS system for practice and pre test but used a commercial quiz design software for the graded tests. They were still able to automate the transfer of grades to the CMS gradebook. Second part of study looked at student satisfaction with open book format Students satisfaction survey reveals: prefer open book format open book format provided same grades as previous years closed book format like feedback – 90% claim to read it Positive result for instructors using online quiz

    7. Innovative training opportunities Kathryn Skhal and Catherine Thureson, University of Iowa Library Used CMS to create an online training program for library science students who staff the reference desk. Included article discussion forum, online quizzes, ‘challenging questions’ module Outcomes: Social community for reference students who often work in isolation Students posted many more questions to the ‘challenging question’ forum. Librarians have become much more efficient in the use of the CMS and able to promote their skills to faculty Skhal, K. J., & Thureson, C. (2007). ICONic training: Use of a course management system to provide continual reference student education. Medical reference services quarterly, 26(2), 15-25. 7 OLA Super Conference 2008: Creating On-line Quizzes Training program included: Content pages Article discussion forum relevant to health sciences librarianship Online quizzes on library resources and services “challenging questions’ module where students and staff discuss questions coming to the reference desk Outcomes: social community useful for any potential isolated staff (night/weekend staff, staff outside ref dept) students became more involved in the ‘challenging question’ forum. This was significant as reference students did not historically refer many questions to the daytime staff, causing concern that complicated questions were not being adequately answered. Training program included: Content pages Article discussion forum relevant to health sciences librarianship Online quizzes on library resources and services “challenging questions’ module where students and staff discuss questions coming to the reference desk Outcomes: social community useful for any potential isolated staff (night/weekend staff, staff outside ref dept) students became more involved in the ‘challenging question’ forum. This was significant as reference students did not historically refer many questions to the daytime staff, causing concern that complicated questions were not being adequately answered.

    8. Not cheating – group collaboration! Amanda Fales-Williams et al, Iowa State University College of Veterinary Medicine Online pre exam quiz in WebCT for veterinary anatomy course Questions were designed to be challenging, did not expect student to be able to answer the question on their first attempt Instructor noted an increased rate of identical answers Not cheating, but group collaboration Fales-Williams, A., Kramer, T., Heer, R., & Danielson, J. (2005). A quiz becomes a multidirectional dialogue with web-based instructional tools for an anatomical pathology rotation. Journal of veterinary medical education, 32(1), 144-149. 8 OLA Super Conference 2008: Creating On-line Quizzes Used the quiz as a teaching tool rather than a testing situation Quiz was not graded Students could make repeated attempts Wrong decisions could be explained in a safe environment Instructor noted an increased rate of identical answers – not cheating, but indirect evidence of group collaboration Results: increased feedback on quizzes has curbed some common mistakes and misconceptions students were more engaged with the material at an earlier point in the semester. Used the quiz as a teaching tool rather than a testing situation Quiz was not graded Students could make repeated attempts Wrong decisions could be explained in a safe environment Instructor noted an increased rate of identical answers – not cheating, but indirect evidence of group collaboration Results: increased feedback on quizzes has curbed some common mistakes and misconceptions students were more engaged with the material at an earlier point in the semester.

    9. Creating a quiz in CMS I consider it a three step process: Think about the learning objectives of your instructional material and then the purpose of your quiz. Assessment of learned material? Learning/teaching opportunity? Create the questions Many different question types. Decide on delivery options 9 OLA Super Conference 2008: Creating On-line Quizzes

    10. Question types

More Related