Author: Fang Wei, Glenn Blank Department of Computer Science Lehigh University July 10, 2007

1. A Student Model for an Intelligent Tutoring System Helping Novices LearnObject Oriented Design Author: Fang Wei, Glenn Blank Department of Computer Science Lehigh University July 10, 2007

2. Intelligent Tutoring System (ITS) • A computer-based instructional system • has knowledge bases for instructional content and teaching strategies • uses a student’s level of mastery of topics to adapt instruction dynamically • A cost-effective means of one-on-one tutoring to provide novices with individualized attention • Computer Assisted Instruction (CAI) system does not model what a student is learning and cannot adapt to student • CAI provides same instruction, problems and feedback to every student

3. Intelligent Tutoring System • Typically contains three main components: • An expert evaluator that observes a student’s work and identifies errors in his/her solution • A student model that diagnoses gap in student’s knowledge • A pedagogical advisor that provides feedback to student

4. Student Model • Maintains a model of students’ current knowledge state by representing and updating • Provides information for intelligent pedagogical decisions and actions including: • curriculum sequencing • interactive problem solving support • pedagogical tutoring customized to each individual student’s learning state

5. Student Model in Wei & Blank (2006,2007) compared with other BN Student Models

6. Layers of Student Knowledge(Self 1994) • Domain knowledge layer • explain all vocabulary for discussing or solving problems • Reasoning knowledge layer • contain reasoning relationships between propositions in domain knowledge • Monitoring knowledge layer • specify how to solve a problem using reasoning knowledge and domain knowledge • Reflective knowledge layer • specify appropriate strategies students should have in a learning environment

7. Three Layered Architecture • CM recognizes cognitive strategies that a student is using • HM simulates students’ hierarchical knowledge in a history • PDM simulates current students’ hierarchical knowledge

8. Curriculum Information Network actor double_int actor_object variable_attribute numeric datatype object object_class object_constructor class_method class_attribute variable_parameter variable_returntype method_parameter datatype variable attribute_constructor method_constructor pass in only constructor attribute_method int double int_string actor_method string double_string datatype_returntype object_method datatype_variable object_attribute class class_constructor method attribute returntype parameter method_returntype A B A is prerequisite of B attribute_parameter

9. Two kinds of concepts • Unique concept, such as attribute or parameter • Relationship concepts, such as attribute_parameter • Relationships emerge because of student’s confusions between concepts • E.g., student defines movieTitle as a parameter when he has already defined movieTitle as an attribute

10. Prerequisite relationships • Prerequisite is relationship between concepts: • The concepts a learner needs to understand before understanding a concept • E.g., one needs to understand int and double in order to understand numericDatatype • Relationship concepts are prerequisites of unique concepts and vice versa • E.g., class_constructor -> constructor • Understanding constructor doesn’t imply understanding of class, just how to define a constructor for a class

11. ku au Connecting Knowledge with Performance • Student action unit and knowledge unit make a pair(KU,AU) • Infer understanding of a concept (KU) from a student solution step (AU) • Action unit (AU): • A single action or step in a student’s solution • E.g., add an attribute to a class • Knowledge unit (KU) – concept a student need to learn • KU directly causes a student action unit • KU is a concept in Curriculum Information Network (CIN)

12. Atomic Bayesian Network (ABN) …… d-prereq(ku)N d-prereq(ku)1 d-prereq(ku)2 Noisy-and generalizes logical-and ku Students must understand all direct prerequisites of the concept kuin order to understand ku au

13. How to generate an ABN • Student model generates an ABN in response to a student solution step • First, define the structure of an ABN, i.e., the causal relationship between KU and AU, and the direct-prerequisites of KU • Second, determine conditional probability tables for this ABN

14. 0 d-p(ku)N 1 … d-p(ku)N … 0 1 d-p(ku)2 0 d-p(ku)1 1 d-p(ku)1 ku 0 ku 1 d-p(ku)2 0 au 1 au Atomic Dynamic Bayesian Network (ADBN) for HM layer

15. How to generate an ADBN • Student model generates an ADBN in response to a student solution step • First, look for the ABN in response to previous student solution step • Second, generate an ABN in response to current student solution step • Third, determine conditional probability tables for the ADBN

16. Concrete Example • Student defined movieTitle as a parameter for method displayMovieTitle after she has already defined movieTitle as an attribute to a class TicketMachine • EE determines that movieTitle should not be a parameter • SM determines that the center concept of an ABN is attribute_parameter, andfinds all direct prerequisites, attribute and parameter, from CIN

17. Concrete Example • attribute’s prior can be found from the database • parameter’s prior is 0.5, students’ knowledge state is assessed based on the difference between prior and posterior probabilities (VanLehn et al. 1998, Millán & Pérez-de-la-Cruz 2002) • SM determines: • student has good understanding of class, attribute,methods, and parameter but low understanding of attribute_parameter • the tutoring need is: explanation of attribute_parameter

18. Concrete Examplefeedback • “Since you have added movieTitle as an attribute to the class TicketMachine, you shouldn’t also make it a parameter to the method displayMovieTitle. To decide whether movieTitle should be an attribute or a parameter, remember: attributes are accessible anywhere within the scope of a class, while parameters are accessible only within the scope of a method”

19. Conclusions • Student models with ADBNs can diagnose student knowledge states accurately in real-time • Accuracy of ADBN-based student model is significantly higher than ABN student model

20. Future work • Implement cognitive model to simulate monitoring knowledge and reflective knowledge • Consider students learning gain from reviewing feedback • how do we determine the conditional probability table for the ADBN so as to simulate the real student learning? • how do we update the new ADBN? • how do we convey empirical studies with simulated students and human subjects? • Diagnose students’ learning state in other domains, such as object-oriented programming