1 / 22

In-vivo Experimentation

In-vivo Experimentation. Steve Ritter Founder and Chief Scientist Carnegie Learning. An attempt to find meaning in three acts. Design : Geometry Contiguity (Vincent Aleven , Kirsten Butcher) Modeling : Adjusting learning curve parameters (Cen, Koedinger , Junker)

zaide
Download Presentation

In-vivo Experimentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. In-vivo Experimentation Steve Ritter Founder and Chief Scientist Carnegie Learning

  2. An attempt to find meaning in three acts • Design: Geometry Contiguity (Vincent Aleven, Kirsten Butcher) • Modeling: Adjusting learning curve parameters (Cen, Koedinger, Junker) • Personalization: Word problem content (Candace Walkington)

  3. Design

  4. Geometry angles

  5. Contiguity Early Version Commercial Version (Carnegie Learning) Research Version (Carnegie Mellon) Butcher, K., & Aleven, V. (2008). Diagram interaction during intelligent tutoring in geometry: Support for knowledge retention and deep transfer. In C. Schunn (Ed.) Proceedings of the Annual Meeting of the Cognitive Science Society, CogSci 2008. New York, NY: Lawrence Earlbaum. Hausmann, R.G.M. & Vuong, A. (2012) Testing the Split Attention Effect on Learning in a Natural Educational Setting Using an Intelligent Tutoring System for Geometry. In N. Miyake, D. Peebles, & R. P. Cooper (Eds.), Proceedings of the 34th Annual Conference of the Cognitive Science Society. (pp. 438-443). Austin, TX: Cognitive Science Society.

  6. Early Tutor

  7. Revised (commercial) tutor

  8. Geometry Contiguity • Design and field experimentation • Butcher and Aleven (2008) • Diagram interaction led to better transfer and retention • Analysis of impact • Hausmann and Vuong (2012) • Unit-level effects mixed • Advantage for harder skills

  9. Geometry Angles

  10. Lessons • Change is constant • Transition from research to production always requires adaptation

  11. Modeling

  12. Skillometer

  13. Expression Writing

  14. What gets learned?

  15. Bayesian Knowledge Tracing Cognitive tutor traces these skills differently

  16. Learning Curve Parameter Fitting • Field study looking at learning area of geometric figures • One group used adjusted learning parameters based on previous year’s data • Optimized group took 12% less time to reach same performance • Significant learning gain in both groups • No difference in learning gain between groups (p = 0.772 )

  17. Lessons • Learning efficiency is a great outcome • Small, systemic changes can have big impact • Optimizing skills requires appropriate skill model • Koedinger, McLaughlin and Stamper (2012) - LFA

  18. Personalization

  19. Word problem customization

  20. Personalization field study • Students who got problems related to their interests made fewer errors • Also affected subsequent unit • Interaction with readability

  21. Lessons • Content matters • Challenge for knowledge component modeling • Are we personalizing preferences, reading level or both?

  22. Summary • It’s not about whether A is better than B • It’s about whyA is better than B

More Related