1 / 23

Using QuestionMark Perception for summative on-line assessment

Using QuestionMark Perception for summative on-line assessment. Marija Cubric May 2006. Context. Level: UG1 Module: Introduction to programming Practical “hands-on” subject that requires some amount of thinking  Cannot be learnt from books only Number of students: ~100

gaitherm
Download Presentation

Using QuestionMark Perception for summative on-line assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using QuestionMark Perception for summative on-line assessment Marija Cubric May 2006

  2. Context • Level: UG1 • Module: Introduction to programming • Practical “hands-on” subject that requires some amount of thinking  • Cannot be learnt from books only • Number of students: ~100 • Mix of BISF, AMIS, BS, MB, .. • Last year failure rate: 56% • Main Issues: • Low attendance (>50% never showed up!) • Low motivation (does not count towards the final grade) • Not enough contact time (1hpw tutorial – not sufficient)

  3. Solution (first iteration ) • Attendance • Weekly on-line test (20% of final grade) • Motivation • Publishing “league tables” (top 10, 10 most improved etc) • Contact time (tutorials) • Increased from 1hpw to 2hpw

  4. Theory • Objective questions (MCQ/MRQ) can be designed to test all levels of Bloom’s educational competencies: • Knowledge • Comprehension • Application • Analysis • Evaluation • Objective questions can be used in summative/ not only formative assessment, but need to be mixed up with other forms of assessment • Creativity, design and development, non-cognitive skills, writing style etc – cannot be tested with objective questions. • Receiving immediate feedback is important part of any constructivism*-based learning cycle such as Kolb’s learning cycle • (*Constructivism - knowledge construction is social process that occurs through collaboration with others) • Amanda Jeffreys & Andy Oliver (BLUE team) run w/shops on how to design objective tests

  5. Tool • QuestionMark Perception • 3rd party sw, licensed to UH (trial) • Not part of StudyNet • http://www.questionmark.com/uk/home.htm • Tool administrator: Angus Shaw <r.a.shaw@herts.ac.uk> • To setup account for you • To create participant groups (module code)

  6. Features • Roles: administrator or participant • Administrator functions: • Question management (C/E/D) • Assessment management • Schedule management • Group management • Report management • Participants functions: • Start & submit test • Review test results

  7. Login • http://www.perception.herts.ac.uk/em/login.asp

  8. Question database

  9. Types of questions

  10. Configuring assessment

  11. Configuring schedule

  12. Report example (full score list report) • Result index • Assessment ID • Assessment name • Assessment author • Assessment last modified • Participant • Participant group • Participant details • Hostname • IP address • Status • Date/time started • Date/time finished • Total score • Maximum score • Percentage score • Time taken • Topic percentage score • Topic result

  13. Types of reports

  14. Test example • Login to: • http://www.perception.herts.ac.uk/q/perception.dll

  15. Feedback • Not immediate (except the score) • Run through the answers immediatelly after the test (extra 5-10 minutes)

  16. Results (tutor’s perception) • Increased attendance: • Average tutorial attendance: 75% • Increased motivation: • “League tables” very popular! • Better results? • Yet to be seen • Expecting raise of 10% in pass rate • “Liminality” (Glynis Cousin) • “mimicking” rather than “thinking” (“cut&paste” strategy)

  17. Results (students’ perception) • Data: anonymous questionnaire (74 respondents)

  18. Results (usability)

  19. Workload • Creating questions (10min/q) • Creating assessments & schedules (30min/w) • Importing reports to Excell & generating league tables (30min/w) • Beware: extra time >4hpw (incl. 20 new questions)

  20. Questions? • Can questions be parameterized & parameters within question randomized • NO • Can questions database be exported • Questions-yes, not sure how to expeort answers??? • Can it be used outside of the campus • YES (with usual proxy settings for remote access)

  21. URLs • Admin login: • http://www.perception.herts.ac.uk/em/login.asp • Participant login: • http://www.perception.herts.ac.uk/q/perception.dll • More on QuestionMark Perception: • http://www.questionmark.com/uk/home.htm

  22. Top 10 – see StudyNet for full score table

  23. Top 10 (most improved) – see StudyNet for full score table

More Related