CLT Conference Heerlen

1. Does Cognitive Load Theory account for the beneficial effects of worked examples in tutored problem solving? CLT Conference Heerlen Ron Salden, Ken Koedinger, Vincent Aleven, & Bruce McLaren (Carnegie Mellon University, Pittsburgh, USA)

2. Worked examples and tutored problem solving • Worked examples mostly investigated in untutored problem solving environments • Cognitive Tutor is Intelligent Tutoring System  provides step-by-step guidance during complex problem solving practice

3. Worked examples and tutored problem solving • Cognitive Tutor provides a tougher control condition of tutored problem solving • It is this tutored part that in our view reduces extraneous load • And, sometimes, increases germane load

4. Cognitive Tutors • Longstanding tradition in improving students’ learning • Grounded in cognitive theory (ACT-R, Anderson & LeBière, 1998) • Methods for reducing WM load • Scaffolding or prompting of sub-goals • Step by step feedback & hints (i.e., guided learning) • Use cognitive model of student thinking • Many full-year classroom evaluations show improved math competence (Koedinger & Aleven, 2007)

5. Cognitive Tutors Elements that reduce extraneous cognitive load.

6. Standard Cognitive Tutor: control condition 1a. Corrective feedback Pittsburgh Science of Learning Center

7. Standard Cognitive Tutor: control condition 1b. Implicit positive feedback Pittsburgh Science of Learning Center

8. Standard Cognitive Tutor: control condition 2. Stepwise hints: last hint level is bottom-out hint  problem fading into example

9. Standard Cognitive Tutor: control condition 3. Problem sub-goals are given

10. Standard Cognitive Tutor: control condition 4. Student’s self explanation This feature is not so much about reducing extraneous load but about increasing germane load Pittsburgh Science of Learning Center

11. Studies • Shih et al (Geometry) • McLaren et al (Chemistry) • Salden et al (Geometry) Not addressed in this talk • Anthony et al (Algebra) obtained similar results as the other three studies  Does Cognitive Load Theory explain beneficial effects of examples in tutored problem solving?

12. Shih et al • Study: re-analyzing prior study (Aleven & Koedinger, 2002) • Logged response data on bottom-out hint usage • One type of “Gaming the system” behavior • Can be hint abuse due to students skipping abstract hints to obtain the concrete answer • Can also be helpful when bottom-out hints act as worked examples

13. Shih et al • Developed a model to distinguish between good student use of bottom-out hints from bad student use of bottom-out hints • Two key elements of model are time spent on: • Reflecting about prior step (after bottom-out hint) • Thinking about next step (prior to next action) • Subtraction method to isolate reflection (self-explanation) time • Use other data, when bottom-hints are not requested, to estimate next step time

14. Shih et al results • High correlation of time spent reflecting on bottom-out hint with learning (pre-to-post gain) • Spending time on hints is beneficial to learning for all students • Difference between students’ hint usage: • Good usage = spending more time on bottom-out hint • Bad usage = spending less time on bottom-out hint • Thus students who study bottom-out hint as worked example obtain higher learning gains

15. McLaren et al • Conducted three studies comparing • Tutored Alone vs. Worked Examples + Tutored • Examples are alternated with isomorphic problems

16. Stoichiometry Tutor: control condition

17. Worked Example condition • Students watch video of a worked example plus do prompted self-explanations following the example:

18. McLaren et al results • No differences on posttest performance • BUT, students in Examples condition did learn more efficiently, using 21% less time to finish same problem set

19. Salden et al • Conducted lab and classroom study comparing: • Tutored problem solving • Fixed example fading • Adaptive example fading • Adaptive fading based on students’ self-explanations of the example steps • Students who self explain well receive fewer examples than students who self explain poorly

20. Standard Cognitive Tutor: control condition Pittsburgh Science of Learning Center

21. Example-enhanced Tutor: experimental condition Worked out value step with calculation shown by Tutor Student still has to self explain the worked out value step!

22. Salden et al results • Lab study: • Adaptive fading condition needed fewer examples than fixed fading condition • Adaptive fading > both fixed conditions on posttest and delayed posttest • Classroom study: • Adaptive fading condition needed fewer examples on several theorems than fixed fading condition • Adaptive fading > problem solving on delayed posttest

23. Summary of results • Shih et al: Students can effectively use bottom-out hints as worked examples and achieve higher learning gains • McLaren et al: Students working with examples can complete learning phase needing 21% less time while obtaining the same learning outcomes • Salden et al: Students learning from adaptively faded examples obtained higher immediate and delayed posttest performance • Fourth study by Anthony et al (using Algebra Tutor): Students who learned with examples attained better long term retention • Also measured mental effort: examples = tutored problem solving

24. Does CLT explain these beneficial effects of worked examples in tutored problem solving? • Cognitive Tutor is a harder control condition than untutored environments • Students can effectively use bottom-out hints as worked examples • The tutoring seems to reduce possible extraneous cognitive load • Anthony study even showed no difference in mental effort between control and experimental condition • Stepwise feedback & hints, self-explanation prompts geared to increase germane cognitive load

25. Does CLT explain these beneficial effects of worked examples in tutored problem solving? Possible explanations • Without the information (guidance) provided by examples, students waste time tackling new skills during problem solving • McLaren study: examples lead to same learning gains but needed 21% less time • Two Freiburg lab studies: examples lead to same learning gains needing roughly 17.5% and 25% less time • Motivation • Goal of understanding v. performing (Shih et al) • Frustration after unsuccessful solution attempt • Where is the cognitive load?

26. Questions?