1 / 25

Pros and Cons of on-line testing using Moodle

Preamble. TEL initiatives by the Japanese Section of DLLL Server-based web course developmentDeveloping interactive instructional materials for self-studyDeveloped a distance education course for the elementary level Japanese language course (tested at Glendon for 4 yrs) using videoconferencingI

gaura
Download Presentation

Pros and Cons of on-line testing using Moodle

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Pros and Cons of on-line testing using Moodle A case study of the elementary Japanese course

    2. Preamble TEL initiatives by the Japanese Section of DLLL Server-based web course development Developing interactive instructional materials for self-study Developed a distance education course for the elementary level Japanese language course (tested at Glendon for 4 yrs) using videoconferencing Introducing Media Site Live for video-streamed lectures Developing web-based on-line tests for the elementary Japanese course assisted by ATS (2005-06) Developing a distance education course for the advanced Japanese course for St. Mary’s University in Halifax (to be implemented in September 2006)

    3. Web-based Testing (WBT) Computer-based Testing (CBT) vs. WBT On-line course delivery software products WebCT Hot Potatoes Sakai FLE3 (Future Learning Environment 3) Moodle (Modular Object-Oriented Dynamic Learning Environment) Moodle Free Open source – customizable Non-proprietary User-friendly Ease of installation Flexible Language support Modular & Comprehensive Why on-line testing? Reducing marking time Developing distance education courses

    4. Challenges for on-line testing for languages Concerns by language teaching professionals limited types of questions lack of analytical tools (natural language parsing) lack of qualitative evaluation lack of evaluation for communicative competence security issues

    5. Implementation and Objectives On-line testing is NOT comprehensive On-line testing is to access each learner’s knowledge and recognition of: Vocabulary Expressions Conjugations Sentence structures Basic kana characters and basic sino-Japanese characters (kanji) Simple context (communicative understanding) Sociolinguistic and pragmatic aspects

    6. Developing tests Transforming paper-based tests into web-based versions Modifying types of questions Developing new types of questions Reviewing students’ answers and modifying possible answers Readjusting answers to introduce partial marking system Developing a questionnaire for students’ feedback

    7. Course Management System (CMS aka LMS) Similar to WebCT, or Blackboard Our Moodle runs on Linux OS using PHP and a MySQL database backend. Open-source, modular and flexible Large, growing community Community-driven support Overview of Moodle

    8. Overview of the Japanese Tests Tests run in the Multimedia Language Lab (MLC) 50 mins per test 2 to 5 tests run a day, over 2 days, 5 mins between tests Students self-register for accounts using Moodle Ran an Orientation Session at beginning to create accounts and try out sample test Tests were originally developed in MS Word. Imported into Moodle via web interface

    9. Question Types Aside from the normal types of questions, the tests also included: Fill-in-the-blanks (multiple blanks within a paragraph, using Cloze or Embedded Answers type questions) Wildcard characters in the answers (for partial marks) Custom question type: 50 Japanese characters in a large image (png), each given a unique number Students answered the question by typing the number of the character into the box, in the correct order. (able to use wildcards for partial marks here too).

    10. Security Measures IP-restricted (MLC only) Each test has unique password Time-based release of each test During the allowed test time, students can review answers Allowed a maximum of 2 attempts Moodle ‘secure window’ test

    11. Technical Challenges Accurate representation of Japanese characters Clarity of images Import not functioning in Moodle, so questions were cut/pasted Time-consuming to enter questions initially Cloze type question still sensitive to illegal characters Various minor software glitches (eg. Half marks getting rounded down)

    12. For the Future Moodle has proven to be a successful way of deploying language tests online All of the major/minor software problems are going to be fixed over the summer 2006 Eventually will be bringing Language Placement Tests on board (French and Italian)

    13. What is tested? Evaluation criteria for JP1000 Attendance & Participation (Sept.-April): 10% Oral Presentation (4 times a year): 25% Quizzes (dictation, every week) & Homework (practice of Japanese characters): 15% Tests (4 times a year, on-line testing): 50% -grammar & structure; vocabulary; idiomatic expressions -reading of Japanese scripts

    14. Types of questions Pen & Paper vs. On-line Reading words “What would you say when…?” Translation (E?J) -Complete a dialogue -Short answers vs. jumbled words Kanji (sino-Japanese characters) Translation (J?E) -Short answers vs. multiple-choice

    15. New attempts Short answers Multiple-choice questions Questions using a table Questions using pictures “Read a passage and answer questions”

    16. ‘Cons’ for Students

    17. ‘Pros’ for Students Quiet atmosphere, easier to focus Easier to answer questions (multiple-choice, kanji recognition, re-attempts) Easier to correct answers Immediate feedback No concern about bad handwriting; hands do not get tired from writing; no need to use pencils Less stress for those familiar with technology, more fun Gain more knowledge about computer technology - important for academic and professional life

    18. On-line vs. Paper Test Average (%)

    19. Students’ Self-reflection (1) Q: How satisfied are you with the time you have allotted to prepare for the tests? A: dissatisfied 6.67% somewhat dissatisfied 46.67% satisfied 34.67% very satisfied 12%

    20. Students’ Self-reflection (2) Q: How satisfied are you with the efforts you have allotted to prepare for the tests? A: dissatisfied 8.79% somewhat dissatisfied 44.59% satisfied 38.51% very satisfied 8.11%

    21. ‘Pros’ for Instructors (1) Less time spent on marking, less tired, improved quality of teaching A test can be automatically re-graded for the entire class if a mistake is discovered in a question or more possible answers are added to the system Easier to deliver the correct answers to students Typed text easier to read than unintelligible handwriting

    22. ‘Pros’ for Instructors (2) Easier coordination between instructors regarding reviewing students’ answers, modifying possible answers, marking Faster processing of the test results Objective assessment (not a comprehensive approach) Increased marking consistency

    23. ‘Cons’ for Instructors Technical problems unsolvable by instructors Too sensitive to spaces, hyphens, unpredictable bugs in the program Location and scheduling restrictions Limitations regarding the types of questions Academic honesty issues

    24. Cheating… ? Weaker classroom control Assigned seats, gaps in the seating ? Inability to cut off the Internet ? ‘Monitored activities’ ? Multiple attempts, higher test scores ? Limited attempts (two attempts only) ? Easy to duplicate the test and export it to Word ? Question pools, randomized questions, different order of answers; run Moodle from a CD

    25. Outcome Learned more about limitations, bugs and positive features re Moodle Why were test scores lower this year? Academic honesty issues Learning process for faculty Faculty’s willingness to learn and no quick resistance to Moodle Cooperation between IT consultant and faculty with good working relationship and initiatives Tech problems are hard to deal with by faculty alone: requires a tech support person at the test site. Limitations re the test site

    26. References Abhijeet Chavan (2004) Open-Source Learning Management with Moodle http://www.linuxjournal.com/article/7478 Moodle (2006) Moodle for Language Teaching http://moodle.org/course/view.php?id=31 Aditya Nag (2005) Moodle: An open source learning management system http://business.newsforge.com/article.pl?sid=05/05/09/2117200 Den Pain and Judy Le Heron (2003) WebCT and Online Assessment:  The best thing since SOAP? http://www.ifets.info/journals/6_2/7.html Röver, C. (2000)Web-Based Language Testing: Opportunities and Challenges http://www2.hawaii.edu/~roever/wbt.htm _________(2001) WEB-BASED LANGUAGE TESTING http://llt.msu.edu/vol5num2/roever/default.html Sabine Siekmann (2006) CALICO Software Report Which Web Course Management System is Right for Me? A Comparison of WebCT 3.1 and Blackboard 5.0 http://calico.org/CALICO_Review/review/webct-bb00.htm University of Ontario (2006) WebCT http://www.uoit.ca/EN/main/11258/12122/17767/learning_webct.html

More Related