The Problem with Exams M. Selen, UIUC Physics

1. The Problem with ExamsM. Selen, UIUC Physics • Why exams are a sensitive issue… • Hand graded versus multiple choice. • How we do it in the intro Physics courses. • What is the cost?

2. Exams typically represent > 50% of the final course grade. • Students really focus on exams (perhaps too much). • Any perceived problems (real or imagined) will generate enormous resentment. • Faculty often don’t focus on exams. • Doing it right is time consuming, in particular of you are working alone. • “Fairness” is not a trivial issue. • Exam content. • Grading.

3. Hand Graded vs. Multiple Choice • Hand Grading (what we used to do): • PROS: • Students feel like everything they write is taken into account (i.e. partial credit). • Any mistakes in the exam can be adapted to (exam is more forgiving to the professor…less quality control is needed). • CONS: • Hard to make grading fair & consistent. • TA, Handwriting, time, before/after pizza etc.. • Whiners are rewarded (i.e. re-grades).

4. Multiple Choice (what we do now): • PROS: • Uniform & Fair. Everyone is treated exactly the same. • Fixing a bug in a problem help everyone. • Lends itself to electronic publishing. • WEB interface possible for practice (before exam night) and help/explanations (after exam). • Very useful for analysis. • tracking changes, education research, exam problem quality control, problem “bank” etc. • CONS: • Harder to give partial credit (but not impossible…). • More care is needed when preparing exam. • Considered “inferior” by some (mostly faculty).

5. About 1/3 of exam score is conceptual(2 & 3 choice) Quantitative problems(5-choice) allow students to select up to 3 answers. Partial credit ! Conceptual and quantitative problems are often paired.

6. Analysis of exam “data” is very interesting (and useful for education research). Physics 101 Midterm Exam 1, Spring 2000

7. not so good good More sophisticated analyses can be used to rate the effectiveness of each exam questions:

8. Good (strong correlation withthe rest of the exam) Not so good (weak correlation withthe rest of the exam) We can look at the discriminating power of each problem: We can learn, quantitatively, how to build better exams.

9. Instant exam feedback is possible: • The minute they leave the exam, students can go on the web, enter their answers into a web version of the exam they just took, and see what their raw score is: • After the exam has been graded (next day) students can find detailed statistics on each problem on the web. Students LOVE this !

10. The Cost (is it more work?) • These courses are team-taught. • Typically 3 faculty • Lecturer, Discussion director, Lab director • The team course works together to produce exam. • The subject material is divided up and each faculty submits a set of problems. • Old problems can be used for guidance. • Format is fixed (MS word in our case). • One of the faculty is in charge of assembling exam (secretarial staff can help). • The team meets several times to discuss & iterate the problems until the final draft is ready. • A senior TA works the exam out and provides comments on difficulty, length etc. • Three version of exam are produced by scrambling the order of the problems. • Secretaries do this. • After the exam, OIR “machine reads” the scantron forms and gives us the “raw data”. We have scripts that do the final analysis to yield grades etc.

11. From Physics 101 web page • We are getting good at this. Exam averages are consistently 70-75%. • Curving is unusual. • We never curve down. • We can honestly tell students that they are not competing against each-other. • Everyone could, in principle, get an “A” in Physics 11x. • This is a great motivator. • We can tell students on the first day of the semester what final semester score they need to get the various letter grades: