Using Online Quizzing to Improve Student Preparedness and Learning Outcomes

1. Using Online Quizzing to Improve Student Preparedness andLearning Outcomes John Broida Psychology Department University of Southern Maine Broida@usm.maine.edu

2. Topics I hope to cover • Why Quiz? • What is web-based (on-line) quizzing? Why quiz on-line? • When to quiz? • How many questions should be on a quiz? • How many points should each quiz be worth? • How many times should students be allowed to take a quiz? • How to prevent/reduce cheating on quizzes? • How much advance warning should students get before a quiz? • How much time should students get to answer each question? Should quizzes have time limits?

3. Why Quiz? Part 1 • The simple answer: It helps students learn the material. • Quizzing has been used for many years, because it works • The average student does not do optional work, has other ways to spend their time • Quizzes increase the time students spend studying • Students who expect a quiz may more than double the time spent preparing for class • One result is that students come to class with more understanding of the material • Quizzes spread study out through the entire semester (reduces reliance on cramming) • Cramming improves performance on exams, but it does little to foster long-term retention

4. Why Quiz, part 2 • Quizzes help those who are struggling, and reward those who understand the material • For students, they provide feedback about what they know and do not understand • For instructors, who does not know (needs help), what should be covered in class. • Quizzes have a down side. • They take time to create, to deliver, to grade. • Web-based quizzing deals with these issues very effectively and efficiently.

5. What is web-based (on-line) quizzing? Why quiz on-line? Part 1 • In-class quizzes usually require students to answer the same questions at the same time. • In class quizzes are an evaluation, a “got ya” • In class quizzes are very time consuming, to administer and grade • This explains why many instructors do not use this tool • Though it may seem obvious, web-based quizzing is quizzing done on-line, not in the classroom. • This means that class time can be devoted to something other than the quiz. • Also, some, if not all, questions can be computer scored, reducing faculty workload • And many publishers provide on-line quizzes with their texts, further reducing workload

6. What is web-based (on-line) quizzing? Why quiz on-line? Part 2 • Web-based quizzing is a recent improvement over in-class quizzes. • Web-based quizzing enables quizzes to become “formative” • Multiple attempts facilitate learning/mastery • As with in-classes, faculty can see who does not know, what should be covered in class • Note, web-based quizzes can also be an evaluation, if students can take them only once. • The drawback of web-based quizzing is you never know who is actually answering the questions • Are students relying on their text (control this by setting a time limit) • Do they have help while they are taking the quiz? • As I use quizzing, the goal is to get students to read the text

7. When to quiz? Part 1 • The most effective quizzing occurs before you talk about the material in class. • Students get some understanding as the result of reading • Practiced teachers find that they eliminate between 30 and 50% of their standard lecture,. Because the students already understand it. • We find that more students ask questions in class when they understand the topic • They are more prone to ask about implication and analysis, not definition! • I personally like to quiz students once a week, given I cover about 1 chapter each week. • The quiz is due an hour before the first class covering the chapter • If you are using a modular textbook, you may want to quiz students more frequently • If you teach more than one chapter in a week, think about combining chapter quizzes • That said, some colleagues like to offer online quizzes two or three times per week.

8. When to quiz? Part 2 • The best advice is, do what feels comfortable to you and best matches your course organization. • I am not aware of research that suggests there is an optimal quizzing frequency • The other best advice is to make quiz deadlines consistent— • For example, all quizzes are due at 2 PM on Tuesday • Also, it is a good idea to have a consistent number of quizzes each week. • Having 1 quiz this week, 2 quizzes the next and 3 two weeks later is confusing • Communicate your expectations for quizzing clearly

9. How many questions should be on a quiz? Part 1 • There are a few general best practices but no absolute rules. • Consider my recommendations in the light of your own course needs and preferences. • I typically use 20 questions • I’ve tried 10, 25, and 30 but have found 20 to be most effective. • I have colleagues who use 10, 15, 20, 25 and 30. • Asking around, I have discovered that 20 is certainly the modal value. • In part, the number of items on a quiz often is based on how many topics are covered • Most chapters in the social sciences have fewer than 20 topics • The idea is to ask a question on each topic on every quiz, more or less

10. How many questions should be on a quiz? Part 2 • Longer quizzes (with 30-50 questions) feel more like tests • They introduce compliance challenges.. • Data suggests that some students taking long quizzes respond randomly towards the end. • Why not shorter quizzes? • 5-10 questions is generally not representative enough for a full chapter quiz. • Fewer items may work for a modular book. • I believe it important not to ask multiple questions on a single topic while ignoring other topics. • Quizzes set up by Worth are designed such that there is about 1 question per topic

11. How many points should each quiz be worth? Part 1 • This really depends on you and your course structure • I can certainly talk what several early adopters have found • Use a round number (10 or 20 rather than 17.436 points per quiz) • Fewer people use 13, or 7, because it makes it hard to determine the percentage. • The number of points available on each relates directly to your overall points in the course. • Some people use this as the determining factor in how many points quizzes are worth. • Make sure that each quiz is worth the same number of points • The literature say students are confused when some quizzes are worth more than others

12. How many points should each quiz be worth? Part 2 • The literature suggests that quizzes be “low stakes,” meaning than any 1 is not worth a lot • This reduces anxiety associated with the quiz without reducing compliance • Missing a quiz (due to illness perhaps) is not disastrous to your grade nor is failing a quiz. • Each quiz may not be worth much, • Multiplied by the number of quizzes, the total percentage of a grade can be considerable. • An eighth to a third is typically used, but some use 100%.

13. How many points should each quiz be worth? Part 3 • In my intro class, I have 1000 points available, 300 from quizzes. • A colleague allots 50% of the course grade to quizzing • Another colleague has reported that students work hard for ¼ point. • His class had something like 10,000 possible points. • Students hear the word points • To sum up: • make each quiz a low points activity, • quiz often and consistently throughout the term, • and determine what works best for quizzing vis a via your overall course grade.

14. How many times should students be allowed to take a quiz? Part 1 • Exams grade what students know, quizzes assess what students need to know and can still learn. • Quizzes can be a learning tool and formative assessment tool, • They can also be a summative assessment, but this is not considered as helpful • Quizzes are an ideal way to get students to read and study more regularly. • As a result, I allow more attempts than people use quizzes as summative assessments. • Other colleagues limit the number of possible attempts to 3-5 times. • Some faculty suggest that limiting the number of attempts focuses student study • They know that they must answer all the items correctly soon

15. How many times should students be allowed to take a quiz? Part 2 • Offering multiple attempts is best executed with randomized quiz items. • Assume that you have a pool of 100 items. • The machine randomly picks, for example, 20 for the quiz. • Thus, students do not see the same questions repeatedly • They learn to focus on the topic, not the question. • In general, the more often unprepared students take quizzes, the more concepts they master. • Better prepared students don’t need as many attempts to achieve perfection • The better students often continue to take quizzes, because they find them helpful

16. How many times should students be allowed to take a quiz? Part 3 • If students can take a quiz multiple times, think about how to credit their attempts. • There are several options • First quiz encourages student preparation, eliminates any incentive to retake the quiz. • Average of all quizzes taken also encourages preparation, but grade may go down! • Most recent quiz does not encourage preparation, and again grade may go down • Most faculty use highest grade. No chance a student score can go down.

17. How to prevent/reduce cheating on quizzes? • Cheating is typically defined as having someone else do the work • Perhaps using the text while taking the quiz or having someone else do the work • I do not see using the book as a problem while taking a quiz • The purpose of my quizzing is to get students to read the book. • If you want to limit reliance on the text, limit the amount of time to complete the quiz • How to stop someone else from doing the work • If you have many quizzes, few people outside of the class itself are willing to do the work • And, if you make the questions hard, even advanced students will have a problem • Another tactic is to put quiz questions on the exam! • Tell the students you are doing this – they will want to take the quizzes • At least 50% of the items on my exams come from the quizzes.

18. How much advance warning should students get before a quiz? Part 1 • Pop quizzes are helpful if you want students to study, but do not want to grade quizzes. • The threat of a quiz increases the likelihood that students will study, like quizzing • There is, however, some data suggesting that they study less than when quizzed • If one uses quizzes to evaluate students, then giving little advance warning makes sense • Those who do not prepare do not do well… • This makes no sense if quizzes are used as a study guide, to provide feedback • My thinking is that there is no reason to use pop quizzes if you administer quizzes online • The machine does the work (creating the quizzes, scoring them, entering the grades) • You need to set up the parameters and select the topics and/or questions • Thus, I schedule 1 quiz a week, all are available from the start of the semester • Many people make the quizzes due for a short time, a week or two before lecture.

19. How much advance warning should students get before a quiz? Part 2 • If you want to evaluate students, you should decide how much advance warning to give • If 24-48 hours makes sense to you, or a week… • Be consistent in your announcements, so that students know what to expect. • My advice is that you decide what you want to do and do it • Whatever makes sense given your purpose in these quizzes.

20. How much time should students get to answer each question? Should quizzes have time limits? • There are two schools of thought on this one • Limit time, to get at what students know. • I have a colleague who limits time to 1 minute per question. • Another colleague limits the time to 45 seconds per question, • and yet another 75 seconds. • The less time students have, the more anxious they are • Especially when they have a limited number of attempts. • The more time they have, the more likely they are to look up answers • Limiting time creates the feeling of an assessment, an evaluation. • Do not limit time, to get at what students can find out. • Makes sense if you see quizzing as a formative assessment • Encourages students to study more, • Enables students to identify their weak areas of knowledge

21. How much time should students get to answer each question? Should quizzes have time limits? -2 • I set very high time limits – 1 hour for 20 questions • I want students to look up answers, to read and find the answers, with minimal stress. • Quizzes are seen as a study guide, not an assessment. • My main goals are to get students to read the book and come to class better prepared; • Extra time reduces the aversiveness of the task • I do not recommend setting up quizzes with no time limits. • Students will start the quiz, and complete it after the deadline!

22. Do not hesitate to contact me • If you think that I can help you help your students. • John Broida 207 780 4255 • broida@usm.maine.edu