Constraint-based tutoring

1. Constraint-based tutoring CPI 494 Feb 17, 2009 Kurt VanLehn ASU

2. Outline • SQL-Tutor (today) • ER-Tutor & Normit (next class) • Technical details (next class) • Evaluation & Dissemination (next class)

3. Framework for comparing step-based tutoring systems

4. In class-exercises • Install SQL tutor • Try at least 3 problems • Check our your student model periodically • Try different levels of help • Try different ways of expressing the same step • Try different ordering of steps • What steps does it suggest? • Try to game it • Try fancy ways to say simple things: E.g., instead of referring to Stanley Kubrick by name, retrieve the director of “Clockwork Orange”

5. Step granularity & user interface • Type of user interface? Canvas, forms (i.e., goal boxes), text, simulation based? • Learnability? Every problem the same UI? • In general, what are the steps? The Answer? • On the problem “Display the titles of movies that are comedies or dramas”, what are the steps? • Would a finer granularity make sense? For who?

6. Framework for comparing step-based tutoring systems

7. Interpreting student steps: How does tutor know what is correct? • Is it example-based or knowledge-based or both? • “Constraint-based modeling (CBM) is…not interested in the exact sequence of states in the problem space…” • “CBM does not require an executable domain model, and is applicable in situations in which such a model would be difficult to construct.” • “CBM eliminates the need for bug libraries”

8. A syntactic constraint • Msg: "The SELECT clause is a mandatory one. Specify the attributes/expressions to retrieve from the database.“ • Clause applied to: SELECT • Relevance condition: t • Satisfaction condition: (not (null (select-clause ss)))

9. A semantic constraint • Msg: "Check whether you should have ascending or descending order!“ • Clause: “ORDER BY“ • Relevance: (and (not (null (order-by is))) (not (null (order-by ss))) (bind-all ’?n (names (order-by ss)) bindings) (match ’(?*d1 ?n "DESC" ?*d2) (order-by ss) bindings) (not (qualified-name ’?n))) • Satisfaction: (match ’(?*d3 ?n "DESC" ?*d4) (order-by is) bindings) • Comment: “verifies that if the results of a query are sorted in decreasing order in the ideal solution (is), the the student’s solution (ss) uses the same ordering.”

10. Interpreting a student’s step:Details of the matching • How is an individual step interpreted? • Spaces? Capitalization? • Ordering of conjuncts and disjuncts? • Ordering of steps? • Fancy ways to say simple things? E.g., instead of referring to Stanley Kubrick by name, retrieve the director of “Clockwork Orange”

11. Framework for comparing step-based tutoring systems

12. Suggesting good steps • What does its method/ordering appear to be? • What if there are multiple correct solutions?

13. Framework for comparing step-based tutoring systems

14. Feedback and hints • What are the hint levels? • How can one game? • Does it detect cut-and-paste? • Does it detect common mistakes e.g., quotes, “WHERE” as first word of WHERE clause… • Are the hints pedagogical?

15. Framework for comparing step-based tutoring systems

16. Task selection • What happens if you select next task in order? • Anyone graduate from novice to higher? • Do clauses make a good index for student task selection? • Does student have enough info to do task selection?

17. Framework for comparing step-based tutoring systems

18. Assessment • What are the KCs? Clauses or constraints? • Displayed? • Learning curves? • How many? • Are the colored sections of the bars informative? • How updated? • Is history window useful?

19. Framework for comparing step-based tutoring systems

20. Outline • SQL-Tutor (today) • ER-Tutor & Normit (next class) • Technical details (next class) • Evaluation & Dissemination (next class)