Differences Between Explanation and Answer-Only Students in Learning with a Cognitive Tutor

1. Previous results showed that students required to explain each step of their answers acquired deeper knowledge than students only required to answer problems. Can we predict these differences on the post-test from student interactions with the tutor? • Predicted differences between ‘explanation’ students and ‘answer only’ students Aleven & Koedinger (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26(2)

2. Explanation Condition • Predicted differences between ‘explanation’ students and ‘answer only’ students

3. Answer-only Condition • Predicted differences between ‘explanation’ students and ‘answer only’ students

4. Is there a difference between the Learning Curves? Surprisingly, the Explanation Condition may have a slightly higher error rate.

5. Predicted differences between ‘explanation’ students and ‘answer only’ students Models For Example: High Log-Odds Good Student + Easy Problem + 5th Try Sample R Code: log.model = glm(Success ~ Student+Skill+(adj.count-1):Skill:Condition-1, family=binomial(), data=new.data)

6. Predicted differences between ‘explanation’ students and ‘answer only’ students More Models In this model, there is a different learning rate between the two conditions, indicated by δ. When we fit this model, δwas significantly negative, indicating that students in the Explanation condition did not learn as rapidly from repeated problems. This is unexpected, since the Explanation group performed as well or better than the Answer-only group on the post-test. In this model, each student may improve at a different rate. The correlations in the table are from this model.

7. Predicted differences between ‘explanation’ students and ‘answer only’ students Is there a correlation between predicted probability of success and Post-Test outcomes?

8. Predicted differences between ‘explanation’ students and ‘answer only’ students This model better predicted Post-Test performance for the Answer-only group than for the Explanation group. This model might be missing important interactions for the Explanation group.

9. One possible explanation: Students in the Explanation condition appear to make more mistakes before they ask for a hint.

10. However, they are really only making more mistakes on the reason components.

11. Are the problems simply harder? Are they guessing? They can select reasons from a menu, perhaps this encourages guessing? Do the students not find the hints helpful? • Predicted differences between ‘explanation’ students and ‘answer only’ students Why are students making more errors before asking for a hint on the reason steps?

12. …Thanks, Ken! What we have learned… • Building cognitive tutors isn’t as easy as it looks. • Better English • How to use Excel • Excel and R are good at different things. • Fitting Logistic-Regression Models • Interpretation of regression coefficients.