Tutoring and Help Systems

1. Tutoring and Help Systems Tell me and I forget. Show me and I remember. Involve me and I understand. - Chinese proverb

2. Previous Approach • Used for over 20 years • Computer-based training (CBT) • Computer aided instruction (CAI) • Effective in helping learners, but do not provide the same attention a human tutor can provide • Approach to a solution • Individual problem solving style

3. A New Approach • Previous approaches focused on scripted information about the domain • New approach must reason about both the domain and the learner • Allowing greater versatility in systems interaction with students

4. Intelligent Tutoring System Possesses two intelligent properties: • Generate problem solution • Flexibility within problem domain • Able to explain errors • Adapt to user needs • User knowledge models

5. Model of Traditional ITS

6. Components in Tutoring System Domain Knowledge Pedagogical Module Communication Model Student Model Expert Model

7. Student Model • Used to tailor instruction for each student • Must represent the student’s knowledge with respect to domain • Choice of representation • Must store pedagogical information about the student • Student’s preferences • Problem solving style • General information • Acquisition and retention

8. Student Model Representation • Typically represented with overlays • Student’s knowledge as a subset of expert’s knowledge

9. Overlay Student Model Student’s Knowledge Expert’s Knowledge

10. Overlay With Buggy Extensions Expert’s Knowledge Student’s Buggy Knowledge Shared Knowledge

11. Pedagogical Module • Uses information from the student model to determine what to present to learner • New Material from the Domain • Review of Previous Topic • Feedback on Current Topic • Teaching Meta-Strategy vs. Low Level Issues

12. Low Level • Topic Selection • Examine student model for areas of focus • Problem Generation • Difficulty based on student’s ability level (taken from student model) • Size of question depend on granularity of domain • Feedback • How much & What kind

13. Meta-Strategy • Implementing strategy has been a formidable problem • Ideal to have many strategies to choose from based on student model • Realistically many ITSs only have one • Difficulty in representing knowledge impedes some methods • Socratic method requires “Common Sense”

14. …Enter CBR • Individualizing will depend on two issues • Information about how learner solved tasks • Using this information in subsequent tutorial decisions • Storing this information builds cases • Cases from other learners • Pre-stored cases - Pitfalls domain experts have foreseen

15. Two Goals of CBR Tutoring • Case-based Adaptation • Adapt interface components to the user’s needs • CBR that not only uses pre-stored cases but also stores new cases can be adapted • CHEF: Recipe and Taste • Case-based Teaching • Provide user with cases that help solve current problem • Observe user solving problem – cases can be used as a reminder

16. Two Ways to Store a Case • Case is stored as a whole • Most systems use this approach • Show examples or give advice • Case is stored as a snippet • Describes sub goals of problems within particular context • Used to find problem solving path • Application used in ELM

17. Episodic Learner ModelELM • Analyzes solutions (or partial solutions) to programming problems in LISP • Looks for problem solving errors and returns feedback • Used in diagnostic process • Able to return examples and remindings • EBR: Explanation-based retrieval

18. ELM • Stores user model in a collection of episodes (cases) • User code is analyzed to create a derivation tree consisting of concepts and rules • These concepts and rules are instantiations of units from the knowledge base

19. ELM Knowledge Representation • Represented in hierarchically organized frames • Concepts • Knowledge about the language (concrete procedures and semantic concepts) • Schemata of common algorithmic and problem solving knowledge (eg recursion) • Additional information • Plan transformations for semantically equivalent solutions • Bug rules for derivations which may result from confusion

20. Bug Rule • Bug Code • Ideal Code Append: (APPEND “a” “bcd”) (APPEND (a) (bcd)) Append: (APPEND ‘(a) ‘(bcd))

21. ELM Diagnostic • Code is at least syntactically correct • Starts with task description related to higher concepts in the knowledge base • Most concepts have transformations describing semantically equivalent variations • Ordering of clauses or sequence of arguments • The sequence of testing transformations is determined by the student model

22. ELM Diagnostic Cont. • A set of rules is indexed by concepts describing different ways to solve the goal • Good • Bad • Buggy • Applying a rule results in comparison between plan and student’s code • Diagnostic process is called recursively on further concepts • Results in derivation tree

23. ELM Derivation Tree • Information in tree added to episodic model • Instances of concepts and rules • Context • Transformations and argument bindings • Each concept (level) in tree creates a frame • The set of episodic frames of a particular episode constitutes a case • Can later be indexed by first frame in case to rebuild tree

24. Derivation Tree Student Code Simple And: (defun simple-and(li) (cond ((null li) t) ((null (car li)) nil) (t (simple-and (cdr li))))) • Partial Derivation Tree: • (NIL-TEST(FIRST-ELEMENT(PARAMETER?LIST))) • NIL-TEST • Empty-List-Nil-Test-Rule • (NULLTEST(FIRST-ELEMENT(PARAMETER?LIST))) • NULLTEST • Unary-Func-Rule • (NULL-OP)(FIRST-ELEMENT(PARAMETER?LIST)) • NULL-OP FIRST-ELEMENT • Correct-Coding-RuleUnary-Func-Rule • null • (FIRST-ELEM-OP)(PARAMETER ?LIST) • FIRST-ELEM-OP PARAMTER • Correct-Coding-RuleCorrect-Param-Rule • carli

25. LISP Code Task Description Diagnosis (Explanation) Domain Knowledge Learner Model Generalization Derivation Tree (Explanation Structure) ELM

26. Explanation-Based Retrieval • System generates a solution based on concepts and rules and temporarily stores this solution in case base • All episodic frames that are neighbors contribute to computing weights for similarity • Most similar case is retrieved (based on previous explanations) and temporary solution is deleted

27. ELM-Programming Environment • Intelligent analysis of task solutions • Diagnostic tool based on ELM • Gives user feedback on purposed solution • Directs user with hints • Example-based Programming • Can reuse code from pre-installed cases or the user’s own previous experience • Example-based Explanation • Shows examples based on matching of expected solution with previous cases already in learner model

28. ELM-Adaptive Remote Tutor • HTML Implementation of ELM-PE • Conceptual network of topics • Red light, green light • Example-based programming • Can find the most relevant example from case history • Demonstration of ELM-ART • http://apsymac33.uni-trier.de:8080/elm-art/login-e

29. Static vs Dynamic CB Teaching • Static • Problem design facilitates the diagnosis of failure • Cases (failures) are associated with supporting case to help overcome failure • Limited by case-base • Dynamic • Problems solved twice, by learner and system • System solution used as an index for supporting cases • Model Tracing • Similar to dynamic, but solution used for direct feedback (could limit multiple solution paths)

30. Case-based Chess Endgame Tutor • Dynamic Teaching • Chess heuristics are not given, instead must be inferred • First given examples to watch • Next examples to solve • CACHET structures this learning by recognizing sub optimal moves and providing hints that lead in the right direction

31. CACHET Case Libraries • Pre-defined cases • Prototypical informative games • Cases generated on demand • Able to generate scenarios for learner • Cases produced by the learners themselves • Self-generated cases are very successful for remindings • Useful to system as point of intervention

32. Roger Schank • President and CEO of Socratic Arts • Founder of Institute for the Learning Sciences • Research on AI and cognitive learning theory • Focus on e-learning

33. Cognitive Learning Theory • A general approach that views learning as an active mental process of acquiring, remembering, and using knowledge. • Learning is evident by a change in knowledge which makes a change in behavior possible. • Learning itself is not directly observable.

34. Schank’s Criticisms • Schools act as if learning can be disassociated from doing • Schools believe they have the job of assessment as part of their natural role • Schools believe they have an obligation to create standard curricula • Schools believe studying is an important part of learning • Schools believe students have a basic interest in learning whatever it is schools decide to teach them

35. Schank’s Idea • Case-based reasoning: Understand the universe by matching incoming events to past experiences • The Steak and the Haircut • Knowledge is built on the ability to index and make sense of cases • It is not a set of facts! • You must question to learn

36. Conclusion • CBR can effectively be applied to enhance tutoring systems • Cases can be complete or snippets • Cases can include buggy information • Cases are applied either diagnostically or adaptively • ELM-PE and ELM-ART use cases diagnostically forming a derivation tree