Download
1 / 35
350 likes | 435 Views
Assessment of large student cohorts in an e-learning environment. Naveed A. Malik Virtual University of Pakistan. Overview. Background The problem Dimensions of the problem Major issues/conflicting requirements IT to the rescue. Background.
E N D
Assessment of large student cohorts in an e-learning environment Naveed A. Malik Virtual University of Pakistan
Overview • Background • The problem • Dimensions of the problem • Major issues/conflicting requirements • IT to the rescue LINC 2010
Background • Virtual University of Pakistan (VU) delivers education through innovative uses of IT • VU conducts its courses under a formal semester system with a defined academic calendar • Video lectures are delivered through: • free-to-air television on VU’s TV channels • Data DVDs • Youtube: http://www.youtube.com/vu LINC 2010
Background • Student-teacher interaction is through a comprehensive Learning Management System • http://vulms.vu.edu.pk • Examinations: • Formal proctored examinations • Twice a semester (midterm and finals) • Conducted at dedicated exam centers (f2f) LINC 2010
Problem Dimensions • Large student body • Nearly 60,000 students and growing! • Scattered all over the country • 95 cities and more… • Admissions twice a year • Average annual intake is ~25,000 students • 160+ courses offered every semester LINC 2010
Problem Dimensions • Full time students • Normal course load • Mostly as per scheme of studies • Working students • Partial course load • Variety of course distributions • Limited study times/exam days/hours • Stragglers and certificate students • Can take any course in any semester LINC 2010
The Problem How can the University conduct mid and final term examinations in a secure, convenient, time-limited and efficient manner? LINC 2010
Major Issues • How to create a datesheet (exam schedule) that satisfies all students • How to ensure: • Secure distribution of question papers • Sanctity of examinations across the country • Efficient and secure collection of answer sheets • Grading within an acceptable time frame LINC 2010
Conflicting Demands • Exam centers must run at capacity if exams are to be conducted in a reasonable amount of time (number of days) but • Working students prefer evening hours or weekends for exams • Full time students prefer morning times, weekdays for their papers LINC 2010
Conflicting Demands • Commuting students prefer mid-day times! • Students prefer to have gaps between papers Management Issues: • Mismatch in exam start timings can lead to examination leaks across the country • Ensuring proper invigilation across all centers is a nightmare LINC 2010
The Idea Allow students flexibility in choosing their examination dates/times on a first-come first-served basis and run centers at full capacity to minimize the total duration of the exams LINC 2010
IT to the rescue • Allowing students to create their own individual datesheets on a first come first served basis: • Can only work if students receive distinct question papers • Can be done by creating a question bank and automating the process! LINC 2010
Key Concepts • Question Bank • Student created individual datesheets • Distinct paper generation • Encryption and security • Conduct • Grading • Result LINC 2010
Question Bank • Includes MCQs and Essay type questions • Questions identified by • Syllabus topic / Lecture number • Difficulty level • Type of skill being assessed • Etc. • New questions added on daily basis • Old/overused questions retired after each examination LINC 2010
Planning an Examination • Determine: • the total number of (Students*Subjects) • the capacity of exam centers (computer seats) And hence • Determine and define grid of: • Exam Centers • Exam days (including weekends) • Exam times (multiple sessions in a day) LINC 2010
Making a Datesheet • Students log-in to http://datesheet.vu.edu.pk • Select the City where they wish to appear • Choose Exam Center within the selected city • Their current semester subjects are displayed • For each subject, they: • Select day/date • Select session time • Confirm LINC 2010
Making a Datesheet LINC 2010
Making a Datesheet LINC 2010
Paper Generation • Based on datesheet data, exam dept knows: • Center • Date • Time • Subject, No. of papers required • Subject, No. of papers required • ..... • ..... • ..... • ..... LINC 2010
Paper Generation • Academic staffs provides paper parameters • System generates required number of distinct question papers LINC 2010
Paper Generation • All e-papers for a center are collected • Dispatched by email / transferred to a CD and sent via courier • Cannot be opened before date/time and then only with passwords which are NOT provided beforehand • Questions are doubly encrypted anyway and cannot be read except through software LINC 2010
Conduct • Exam software and encrypted data are loaded onto exam center servers using file password • Session password is sent to the invigilator by email/SMS on exam day 5-10 minutes before exam • Data for only a single session is unlocked • Students log in and are given their question paper according to their datesheet LINC 2010
Conduct • At end of each day, an exam file containing student answers is sent back to VU by email from each center • Contains data for all sessions for the day • Files are uploaded into the grading system and MCQs marked automatically LINC 2010
Grading • Student attempts are collected and sorted by question • All attempts for one question are given to a single tutor for grading • Grading rubric/reference is provided by QB • Ensures anonymity and uniformity LINC 2010
Result Publication • Result is collated by the system • Declared at one go • Students receive their results in their Learning Management System (LMS) gradebooks • Semester and subject positions and campus-wise results also declared LINC 2010
IT Works! LINC 2010
Question Bank LINC 2010
Question Bank LINC 2010
Planning an Examination LINC 2010
Planning an Examination LINC 2010
Paper Generation LINC 2010
Question Paper Criteria LINC 2010
Paper Generation LINC 2010
Conduct LINC 2010
Grading LINC 2010
