What are Mastery Learning, GRIT, and IRT?

1. Mastery Learning: Does mastery-based assessment increase student learning? Neil Heffernan (PI), Zach Broderick, and the ASSISTment Team Worcester Polytechnic Institute Figure 2 – WPI Mastery Learning Content A team of undergraduates at WPI this past year worked non-stop to create mastery learning content focused on a standardized set of skills. Each problem set corresponds to one of these skills with the intent of enabling the student to demonstrate “mastery” of that particular skill. Each set was made using one or more variabilized templates (see figure 1). By default, students can test out if they answer the first two problems correctly. Otherwise, they must answer 3 in a row correctly in order to master the assignment. If the student does 5 problems on the first attempt without demonstrating mastery, he or she is told to consult his or her teacher for help and try again the following day. The student can then do 7 problems the subsequent days before he or she is told to seek outside assistance. What are Mastery Learning, GRIT, and IRT? • Mastery Learning is a style of assessment in which the student must demonstrate mastery of the assignment by correctly answering a certain number of problems in a row. These problems are generally focused on one particular skill (see figure 1). • GRIT measures a student’s self-discipline, calculated using a survey in which students are asked to reflect on and evaluate their self-control. • IRT measures a student’s mathematical proficiency, or knowledge level, and is calculated using performance data from previously completed ASSISTment assignments. Figure 1 – Variabilized ASSISTments Mastery learning problem sets are created using variabilizaed ASSISTments (top picture). These special type of ASSISTments are similar to traditional problems except that they contain variables (denoted by %v{…} ) instead of certain values. Variabilized ASSISTments act as templates that generate other ASSISTments (bottom two pictures) by having the computer algorithmically fill in the variables in the template with random values. In the above example, the sides of the triangle and the bottom-right angle are all variabilized. This allows us to generate problem sets that focus sharply on one particular skill. Research Goal • Determine whether or not mastery learning increases student learning. Additionally, discover which students, categorized by their GRIT and IRT scores, benefit the most from mastery learning. Experiment • 8th grade students at a local middle school were given pre- and post-tests on several different math units over a one month period. In between those tests, alternating groups of students were given mastery learning assignments (figure 2), while the others were not. Performance gain between the two tests was taken as a measure of student learning. Figure 3 – Results The above charts show the gain in percentage points (in decimal form) between the pre and post tests for each condition, averaged over each of 5 math units and Z-scored to control for the relative difficulty of the math content in each unit. Notice that in the top chart, on average students in the control group actually decreased in score between the two tests about 16 and a half percentage points, whereas the mastery learning group increased by 18 and a half. In the second and third tables, notice that the high GRIT and IRT students either did worse on the post-test or did not improve at all, as did the low GRIT and IRT students in the control groups. However, the low GRIT and IRT students in the mastery learning condition gained a surprising amount of percentage points, suggesting that these students benefit the most from mastery learning. Preliminary Results “It feels like I’m conquering the world!” • On average, the group of students treated with the mastery learning condition learned significantly more than their counterparts in the control group. Additionally, there is strong evidence that students with lower IRT and lower GRIT scores are the ones who benefit the most from mastery learning (see figure 3). Figure 4 – Anecdotal Results Throughout the 30 day study, we talked to students regularly about their feelings on mastery learning, as well as observed their response to it. The environment in which we conducted our trials is notoriously low-morale, as students are forced to do math problems for a full hour each day. After mastery learning was introduced, there was a noticeable increase in shouts of triumph—students very much liked it when the computer told them they had “mastered” an assignment. It gave them a much needed sense of purpose. The above quote is the sentiment expressed to us by one particular student.