Theoretical Relevance: Lecture 2 for the IV track of the 2007 PSLC Summer School

1. Theoretical Relevance: Lecture 2 for the IV track of the 2007 PSLC Summer School Robert G.M. Hausmann (Holodeck version of Kurt VanLehn)

2. From this morning’s newspaper “The first lesson in sales is, ‘Look for the pain.’”

3. Look for the pain in the… • Literature • Classroom • Science of learning Your customers

4. Literature: It is painful not to know the answer to a known question • Known questions appear at the ends of papers, reviews, etc. • At least one informed person cares about the answer. • Common (bad) ways to pose research questions • Cool software • Pop psychology • I learn this way, so… No customers

5. Select a question to add information and clarity to the literature • Information value (in Shannon’s sense) • High if prior probability of the answer is very different from the answer obtained in the experiment. • Low if experiment just confirms the expected answer. • Clarification value (real pains here) • Low if the literature is a mess, and the experiment just adds one more fact to the mess. • High if the experiment somehow clarifies the mess. • Moderate if there is little prior literature.

6. Look for (and relieve) the pain in the… • Literature • Known question • Answer would add information and/or clarity • Classroom • Sciences of Learning Next

7. What pains the classroom? • Ask the instructor (you?) what’s most frustrating • Teaching a certain concept? • Transfer to real world? • Depth of understanding? • Ask the students…

8. Andes is not “selling” (can’t give it away!) • Andes teaches quantitative problem solving. • Most instructors think this is not a bottleneck. • Instead, qualitativeproblem solving is their concern.

9. Look for (and relieve) the pain in the… • Literature • Known question • Answer would add information and/or clarity • Classroom • Instructors consider the question important • Sciences of Learning Next

10. Where is the pain in the Learning Sciences? • Too many results • No organization of the results • No theory • No clear implications • No classic results that everyone knows • No accretion • Progress is more like politics than medicine

11. To cure the pain, Learning Science needs a theoretical framework • Not like physics • A few basic principles from which all else follows. • More like Medicine • A few basics (anatomy, physiology, genetics) • Many specializations e.g., lymphatic cancers • Few principles; many diseases, syndromes, therapies • A standardized, rigorous terminology • Digital libraries becoming essential

12. Types of theories Computational models Shared theoretical vocabulary Boxology “How People Learn” principles

13. PSCL theoretical framework Computational models Shared theoretical vocabulary Boxology “How People Learn” principles

14. PSLC theoretical framework • Shared terminology • Research clusters • Analytic framework Next

15. Shared terminology • Micro-level • Knowledge component: A principle, concept, fact, schema, strategy, meta-strategy… • Learning event: An application of a knowledge component • Macro-level: A taxonomy of robust learning processes • Sense-making • Fluency-building

16. Knowledge of the solo student Knowledge of the solo student Micro level is just (good, old fashioned) cognitive psychology Instructional activities Prior knowledge Learning processes can be decomposed and taxonomized Cognitive processes Knowledge components Knowledge can be decomposed Observable outcomes

17. Macro level is a taxonomy of learning process • Sense making • Coordination of multiple types/sources of learning • Example: step plus a rule • Interaction of the student with other agents • Agents can be peers, experts, or tutoring systems. • Fluency • Three Mechanisms: • Strengthening • Deep-feature perception • Headroom

18. PSLC research clusters • Coordinative learning • How do students coordinate multiple sources of information, media, representations, strategies? • Interactive communication • How does interaction between a student and a peer, tutor or teacher affect learning? • Fluency and refinement – How does skill become fluent?

19. Coordinative learning • Co-training (Blum & Mitchell) • Learning from multimedia (Clark & Mayer)(Tversky) • Learning from analogies (Novick & Holyoak, 1991) (J.R. Anderson, Fincham, & Douglass, 1997) (VanLehn, 1998) • Learning from multiple representations & multiple solutions (Ainsworth, 1999) • Learning from agents (Lester, Converse, Stone, Kahler, & Barlow, 1997) (Graesser et al., 2003) (Moreno, Mayer, Spires, & Lester, 2001)

20. Interactive communication • Feedback and hint effects(J. A. Kulik & Kulik, 1988) (McKendree, 1990) (Hume et al., 1996) (Kluger & DeNisi, 1996) *(Corbett & Anderson, 2001) (Mathan & Koedinger, 2005) (V. J. Shute, 1992) • Learning from examples, self-explanation and fading *(Collins, Brown, & Newman, 1989) (Nguyen-Xuan, Bastide, & Nicaud, 1999) (Kalyuga, Chandler, Tuovinen, & Sweller, 2001) (Renkl, Atkinson, Maier, & Staley, 2002) (Kalyuga, Ayres, Chandler, & Sweller, 2003) (Atkinson, Renkl, & Merrill, in press) *(M. T. H. Chi, 2000) (M.T.H. Chi et al., 2001) (V. Aleven & Koedinger, 2002) (Siegler, 2002) (Corbett, Wagner, lesgold, Ulrich, & Stevens, 2006) • Tutorial dialogues vs. monologues *(VanLehn et al., in press) (Vincent Aleven, Ogan, Popescu, Torrey, & Koedinger, 2004) • Learning with a peer, including collaborative learning, peer tutoring, learning by teaching (Reif & Scott, 1999) (Okita & Schwartz, 2006)

21. Fluency and refinement • Practice effects, including spacing and part-whole training effects *(Newell & Rosenbloom, 1981) (J.R. Anderson et al., 1997) (Pavlik & Anderson, 2005) • Macroadaptation and mastery learningeffects (Bloom, 1984) *(C. Kulik, Kulik, & Bangert-Drowns, 1990) (V. J. Shute, 1992) (V.J. Shute, 1993) (Corbett, 2001) (Ainsworth & Grimshaw, 2004) (Arroyo, Beal, Murray, Walles, & Woolf, 2004) • Implicit (practice only) vs. explicit (direct) instruction. *(Berry & Broadbent, 1984) (Singley, 1990) (K. Koedinger & Anderson, 1993) (Klahr & Nigam, 2004) (VanLehn et al., 2004)

22. Enabling technology 7 Current research projects

23. PSLC Theoretical framework • Glossary of theoretical terms • Micro-level • Macro-level • Analytic framework Next

24. Fourth Third Second First Fifth Learning events over time While studying an example, tries to self-explain; fails; looks in text; succeeds 25 sec. While solving a problem, looks up example; recalls explanation; maps it to problem 20 sec. Recalls explanation; slips; corrects 15 sec. Duration Solves without slips Solves without slips 10 sec. 5 sec.

25. A new analytic framework, based on an analogy • A problem is to a problem space asa learning event is to a ______________

26. A new analytic framework, based on an analogy • A problem is to a problem space asa learning event is to alearning event space.

27. Key ideas • A learning event space is a set of paths determined by the instruction and the student’s prior knowledge, • but it is the student who chooses which path to follow • different paths have different outcomes: • Deep learning • Shallow learning • Mis-learning • Etc.

28. You get to choose the granularity • Coarse grain-size: Only observable actions • Correct vs. incorrect steps • Feedback from tutor • Finer: Reportable mental actions • Recall vs. construct • Even finer?

29. How to use learning event spaces Construct a learning event space such that… • it is consistent with observable actions, and… • the top level question, “Why did they learn?” • becomes two easier questions: • Path choice: Why did students tend to choose as they did? • Path effects: Given that a student went down a path, why did that cause the observed learning/outcomes?

30. A simple illustration • Maxine Eskenazi & Alan Juffs hypothesize that using authentic texts will increase vocabulary acquisition in ESL. • Students read text with a few target unfamiliar words. • Texts come either from web or from existing primer. • Clicking on an target word displays its definition. • Why would authenticity increase learning? How?

31. Learning event space (one per target word) Start • Ignore the word • Exit, with little learning • Infer meaning from context • Exit, with “implicit” learning • Click on the word; definition is displayed • Read & understand the definition • Exit, with “explicit” learning • Go to Start

32. Why should authentic text help?Hypotheses based on path choices Start • Ignore the word • Exit, with little learning • Infer meaning from context • Exit, with implicit learning • Click on the word; definition is displayed • Read & understand the definition • Exit, with explicit learning • Go to Start Authentic text should decrease this Authentic text should increase this

33. Why should authentic text help?Hypotheses based on path effects No change Start • Ignore the word • Exit, with little learning • Infer meaning from context • Exit, with “implicit” learning • Click on the word; definition is displayed • Read & understand the definition • Exit, with “explicit” learning • Go to Start Cue validity of this path increases No change???

34. To summarize the theoretical framework… • Glossary • Macro-level • Sense-making • Coordinative Learning • Interactive Communication • Fluency • Micro level: Knowledge components, learning events… • Learning events space • Decomposes “why did they learn?” into • Path choices: Which paths were chosen? • Path effects: For each path, what was learned?

35. Find the pain (and relieve it) in the… • Literature • Known question • Answer would add information and/or clarity • Classroom • Instructors consider the question important • Science of learning • Glossary of theoretical terms • Learning event spaces