Problem Order Implications for Learning Transfer

1. Problem Order Implications for Learning Transfer Nan Li, William Cohen, and Kenneth Koedinger School of Computer Science Carnegie Mellon University

2. Order of Problems • One of the most important variables that affects learning effectiveness • Blocked order vs. interleaved order • Interleaved is better! Why? Most existing textbooks Numerous previous studies

3. Need for Better Theory • Studies • Contextual interference (CI) effect (Shea and Morgan, 1979) • Mixed results on complex tasks or novices • … • Hypothesis • Elaboration hypothesis (Shea and Morgan, 1979) • Forgetting or reconstruction hypothesis (Lee and Magill, 1983) • … • Proposed Approach • A controlled simulation study • Using a machine-learning agent, SimStudent • Given problems of blocked orders or interleaved orders Lacks the precision of a computational theory. A precise implementation. Easier to inspect SimStudent’s learning processes and outcomes.

4. A Brief Review of SimStudent • A learning agent that • Acquires production rules • From examples and problem-solving experience • Given a perceptual representation, a set of feature predicates and operator functions Matsuda et al., CogSci-09

5. SimStudent Learns Production Rules • Skill divide (e.g. -3x = 6) • Retrieval path: • Left side (-3x) • Right side (6) • Precondition: • Left side (-3x) does not have a constant term => • Function sequence: • Get-coefficient (-3) of left side (-3x) • Divide both sides by the coefficient

6. Retrieval Path Learner • Retrieval path: • Left side (-3x) • Right side (6) • A perceptual learner • Finding paths to identify useful information (percepts) in GUI • E.g. <-3x, 6>  <Cell 11, Cell 21> • <4x, 12>  <Cell 12, Cell 22> • Specific  general • E.g. Cell 21  Cell 2?  Cell ?? • The most specific path that covers all of the training percepts

7. Precondition Learner • Precondition: • Left side (-3x) does not have constant term • A feature test learner • Acquiring the precondition of the production rule • Given a set of feature predicates • A boolean function that describes relations among objects • E.g. (has-coefficient -3x), (has-constant 2x+5) • Utilize FOIL (Quinlan, 1990) • Input: • Positive and negative examples based on the percepts • <percept1, percept2> • E.g. positive: <-3x, 6>, negative: <2x+4, 8> • Output: • A set of feature tests that • describe the desired situation to fire the production rule • E.g. (not (has-constant ?percept1)) • Different problem orders  Different intermediate production rules  Incorrect rule applications  Different negative feedback

8. Function Sequence Learner • Function sequence: • Get-coefficient (-3) of left side (-3x) • Divide both sides with the coefficient -3 <-3x, 6> (coefficient -3x) (divide -3) (divide -3) • An operator function sequence learner • Acquires a sequence of operator functions to apply in producing the next step • Given a set of operator functions • E.g. (coefficient -3x), (add-term 5x-5 5) • Input: • A set of records, Ri = <perceptsi, stepi> • E.g. <<-3x, 6>, (divide -3)> • Output: • A sequence of operator functions, op = (op1, op2, … opk), that explains all records • E.g. • (bind ?coef (coefficient ?percepts1)), (bind ?step (divide ?coef))

9. Problem Order Study • Blocked order vs. Interleaved order • Three domains • Fraction addition • Equation solving • Stoichiometry • Training and testing problems • Solved by human students in classroom studies • SimStudent • Tutored by automatic tutors that simulate the automatic tutors used by human students

10. Fraction Addition • Problem • Types

11. Equation Solving • Problem S1 + S2V = S3 • Types

12. Stoichiometry • Problem • How many moles of atomic oxygen (O) are in 250 grams of P4O10? (Hint: the molecular weight of P4O10 is 283.88 g P4O10 / mol P4O10.) • Skills • Unit conversion: 0.6 kg H2O = 600 g H2O • Molecular weight: There are 2 moles of P4O10in 283.88 * 2 g P4O10 • Composition stoichiometry: There are 10 moles of O in each mole of P4O10 • Types

13. Problem Orders Blocked-Ordering Curricula Interleaved-Ordering Curricula

14. Speed of Learning Fraction Addition Equation Solving Stoichiometry

15. Cause of the Effect • SimStudentvs. Human Student • More controllable • More observable • Conjecture: • Interleaved order  Receive feedback from all three types • Blocked order  Receive feedback from some types • Interleaved order  More explicit negative feedback  More effective learning

16. Explicit Negative Feedback • More negative feedback  More effective precondition learning • Opportunities to expose to over-general preconditions

17. Example • E.g., S1+S2V=S3 (Type I) • Negative feedback: Subtract both sides of S1V+S2=S3 by S1V(Type I) • Subtract both sides by Si,if Siis a signed number • Subtract both sides by S1 • Negative feedback: Subtract both sides of S1/V=S2 by V or S1(Type III) • Subtract both sides by Si,if Siis a signed number & there is a “+” • Subtract both sides by S1

18. Summary • Interleaved order  More negative feedback  Better precondition learning • SimStudent with limited memory • Blocked order  More training examples  Better function sequence learning • Future studies • Generality across problem sets • SimStudent with limited memory • A study on human students

19. Thank you!