1 / 24

QA on “The behavior of tutoring systems”

QA on “The behavior of tutoring systems”. CPI 494 Feb 3, 2009 Kurt VanLehn. What’s a “task domain”?. Solving linear algebraic equations? Solving 2x+3=5? Solving math problems? Reciting the Gettysburgh Address? Painting color-by-numbers picture? Dancing the Macrarena ? Folk dancing.

theola
Download Presentation

QA on “The behavior of tutoring systems”

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. QA on “The behavior of tutoring systems” CPI 494 Feb 3, 2009 Kurt VanLehn

  2. What’s a “task domain”? • Solving linear algebraic equations? • Solving 2x+3=5? • Solving math problems? • Reciting the Gettysburgh Address? • Painting color-by-numbers picture? • Dancing the Macrarena? • Folk dancing. • Writing a funny TV sit-com script

  3. What’s a “task”? • Writing a hiaku on death of a loved one? • Solving 2x+3=5? • Writing the digit 5? • Reading • Reading an electronic schematic for a 2-transitor amplifier.

  4. What is a “step”? • What are the steps in • Solving 2x+5=11?  2x=11-5  2x=6  2x/2=6/2 • Writing the digit “4” • Dividing 10 by 2? • Phonic spelling of the last name of the current US president? • Writing a hiaku on death of a loved one? • Painting a color-by-numbers picture?

  5. What is a “Knowledge component”? • Definition (hint: Uses “independently”)? • What are choices of granularity in solving 2x+5=11?

  6. How to use transfer to check if KC’s are defined correctly • Use KCs to represent task domain, e.g., multi-column addition (496+307) & subtraction • Determine which KCs must be mastered for one set of tasks, e.g., addition  A • Determine which KCs must be mastered for a second set of tasks, e.g., subtraction  S • Experiment: • Measure time for group S to master subtraction • Measure time for group AS to master subtraction after mastering addition • Group AS should take less time because some of the subtraction KCs are also addition KCs, and thus have been mastered already. • |S-A| / |S| ≈ time(AS)/time(S) • where |X| is the cardinality of set X • Where S-A is the set difference

  7. What is a learning event? • Definition? • Give example from algebra • Give example from teaching your mom how to enter a phone number into her cell phone.

  8. Fourth Third Second First Fifth While studying an example, tries to self-explain; fails; looks in text; succeeds 25 sec. While solving a problem, looks up example; recalls explanation; maps it to problem 20 sec. Recalls explanation; slips; corrects Duration of the instructional event 15 sec. Solves without slips 10 sec. 5 sec. Chronological position of the learning event

  9. Using learning curves to check KCs • Proper KCs should have smooth learning curves Duration of learning event Duration of learning event 1 2 3 4 5 6 7 … 1 2 3 4 5 6 7 … Learning event s in chronological order Learning event s in chronological order

  10. How to collect a learning curve • Track learning of N students • At each of their learning events • Measure time (plot duration on y-axis) • Measure errors (plot error rate on y-axis) • Measure hint requests (plot hints+errors on y-axis) • Aggregate over the N students • E.g., at the 3rd learning event of all N students, count number of students who make error & divide by N  plot on y-axis, with x=3.

  11. Two nested loops of an immediate feedback step-based tutoring system • Task (outer) loop: • Select task • Do task • Repeat until student has mastered task domain • Step (inner) loop • Tutor and student do a step, somehow • T:hint; S: trys; T: wrong!; S: trys again; T: Good! • Repeat until the task is finished

  12. What are the 2 loops for CTAT fraction addition tutor? • Outer? • Inner?

  13. What are two loops for Steve?

  14. What are two loops for Andes?

  15. What are two loops for Cognitive Algebra Tutor?

  16. What are two loops for AutoTutor?

  17. What are the two loops for SQL-Tutor?

  18. Main issue for the task (outer) loop? • How to select the next task? • Where to get tasks to select from?

  19. 5 common services for steps • Minimal feedback • Hints on the next step • Error specific feedback • Assessment of knowledge • Review of solution

  20. Main issues for minimal feedback • Correctness? • When?

  21. Main issues for next-step hints • When? • What step to suggest? • How to give hints?

  22. Main issues in error-specific feedback • How to classify errors? • How to give feedback? Debugging… • When/whether to give feedback?

  23. Main issues in assessment • What should be assessed? • Grain size of assessment? • How to measure progress, hints & errors? • How to combine progress, hints & errors?

  24. Main issues in reviewing solutions • Who controls the review? • What depth? • Anything suppressed? • Questions allowed?

More Related