Candace Walkington Assistant Professor of Teaching and Learning Southern Methodist University

1. Using Adaptive Learning Technologies to Personalize Algebra Instruction to Student interests: The Impact of Relevant Contexts on Robust Learning Candace Walkington Assistant Professor of Teaching and Learning Southern Methodist University

2. Presentation Overview • Personalization of Learning • Theoretical Framework - Interest • Pilot Work • Study in Algebra I classrooms • Summary, Conclusions, Next Steps

3. Student Motivation in the 21st Century • Important issues with student motivation face schools today (Hidi & Harockwicz, 2000), especially in secondary mathematics (Mitchell, 1993) • Interest in mathematics declines over adolescence generally, and algebra classes specifically (Fredicks& Eccles, 2002; Frenzel, Gotez, Pekrun, & Watt, 2010; McCoy, 2005) • Algebra I a gatekeeper to higher-level mathematics (Kaput, 2000), with significant implications for equity & access (Cogan, Schmidt, & Wiley, 2001; Moses & Cobb, 2001)

4. Student Motivation in the 21st Century • Failure rates in Algebra I continue to be high, especially among low-income students and student of color (Allensworth, Nomi, Montgomery, & Lee, 2009; McCoy, 2005) “I think we're growing serfs in our cities, young people who graduate with eighth grade education that can't access economic arrangements to support families. Kids are falling wholesale through the cracks – or chasms – dropping out of sight… people say they do not want to learn. The only ones who can dispel that notion are the kids themselves.” ~Bob Moses, Algebra Project

5. Student Motivation in the 21st Century How can learning technologies be utilized to enhance student motivation and promote achievement in difficult secondary subjects like Algebra I? “I think we're growing serfs in our cities, young people who graduate with eighth grade education that can't access economic arrangements to support families. Kids are falling wholesale through the cracks – or chasms – dropping out of sight… people say they do not want to learn. The only ones who can dispel that notion are the kids themselves.” ~Bob Moses, Algebra Project

6. Personalization of Learning • Learning technologies emerging that personalize instruction to background, goals, preferences, and prior knowledge (e.g., Papert, 1980, 1983; Carnegie Learning, 2011)

7. Personalization of Learning • Learning technologies emerging that personalize instruction to background, goals, preferences, and prior knowledge (e.g., Papert, 1980, 1983; Carnegie Learning, 2011) • Learners accustomed to customization, interaction, and control when seeking knowledge outside of school (Collins & Halverson, 2009) • National Academy of Engineering named “Advancing Personalized Learning” Grand Challenge for engineering in 21stcentury

8. Context Personalization • Matching instructional components with students’ personal interests and experiences (e.g., sports, gaming, movies, etc.). • Most research has been conducted in elementary mathematics, with mixed results • Domain: Algebra I You work at a furniture store and make $10.50 per hour. How much money will you make in 5 hours? You work at a video game store and make $10.50 per hour. How much money will you make in 5 hours?

9. Theoretical Framework • Interest: the psychological state of engaging and the predisposition to re-engage with certain objects, events, or ideas (Hidi & Renninger, 2006) • Context personalization may elicit topic interest – triggered when learners are presented with a specific topic or theme (Ainley et al., 2002) • Activating interest can involve: • Affect: Emotions accompanying engagement with the topic (Hidi & Renninger, 2006) • Value: The feelings of importance or worthwhileness the learner ascribes to a topic (Schiefele, 1991, 2001) • Knowledge: Learner’s knowledge of the procedures and discourse related to the topic (Renninger et al., 2002)

10. Theoretical Framework • Personalizing instruction may spur affect or stored value for a topic • May also trigger relevant stored knowledge about the topic, that is related to the task at hand: • Reasoning with familiar quantities in algebra (Carraher et al., 2006; Chazan, 1999; Lampert, 2001) • Groundingof abstract ideas in concrete experiences (Goldstone & Son, 2005) - redundancy with everyday knowledge supports inferences (Koedinger, Alibali, & Nathan, 2008)

11. Theoretical Framework • Interest has been associated with: • Attention, persistence, engagement (Schiefele, 1991; Schiefele & Krapp, 1996; Hidi, 1995, 2001; McDaniel, Waddill, Finstad, & Bourg, 2000; Renninger & Wozinak, 1985; AinleyHidi, & Bendorff, 2002; Ainley, Hillman, & Hidi, 2002; Flowerday, Schraw, & Stevens, 2002) • Motivational variables like self-efficacy, self-regulation, achievement goals (Harackiewicz, Durik, Barron, Linnenbrink-Garcia, & Tauer, 2008; Hidi & Ainley, 2008; Sansone et al., 2011) • Learning (Ainley, Hillman, & Hidi, 2002; Ainley, Hidi, & Bendorff, 2002; Harackiewicz et al., 2008; Hulleman & Harackwicz, 2009; Schiefele 1990; 1991)

12. Pilot Study • 24 Algebra I students given story problems (y = mx+b) that were either normal or personalized to their interests • Modifications based on a pre-interview • Personalized problems easier to solve for struggling students, and harder linear functions • More informal strategies, less conceptual errors • Students reported personalized problems as easier, more related to their lives • What about long-term learning? “Every time I tweet my numbers go up - I get more followers… [I have] 156 followers…”

13. Research Question Can a context personalization intervention aimed at matching instruction to topics students are interested in promote long-term learning in algebra? • How does personalization impact performance while the intervention is in place? • How does personalization impact performance once the intervention is removed? • What is the impact of personalization for students who struggle with algebra?

14. Participants • 145 Algebra I students at a Pennsylvania high school • School used Cognitive Tutor Algebra curriculum • Adapts hints, feedback, and problem selection • Story problems & multiple representations • Unit 6 “Linear Models and Independent Variables”

15. Method • Students given open-ended survey about their out-of-school interests in 9 topic areas (sports, music, movies, TV, games, computers, art, food, shopping) • Results of survey used to write problems that were “personalized” to these 9 different topic interests • 4 variations written for each original problem in Unit 6

16. Method

17. Method • Participants randomly assigned to 2 conditions in Unit 6: • Control: Receive normal problems for Unit 6 • Experimental: Receive 1 of 4 personalized versions of same problems for Unit 6, based on interests survey

18. Method

19. Analysis • Tutoring environment tracked different concepts or knowledge components (KCs): • Easy: entering a given, identifying units and quantities • Medium: RU/SU, write expression slope only • Hard: Write expression with slope and intercept

20. Performance Effects • Significant impact for easy (3% difference, p < .001) and hard (10% difference, p< .001) • Personalization significantly reduced time spent writing symbolic expressions (6.93 second reduction, p < .05) *** ***

21. Performance Effects Condition* Opportunity interaction significant, p < .05

22. Performance Effects • Students with low performance in algebra identified through curriculum progress measures (13 C, 12 E) • Personalization had a significantly greater impact on performance for struggling students (24%, p < .05) * * (Hard only) * p < .05

23. Gaming the System • Issue with Intelligent Tutoring Systems • Enter answers quickly and repeatedly • Click through to “bottom out” hint • Baker and deCarvalho (2008) developed “gaming detector” that utilizes log data • Personalization significantly reduced gaming behaviors in Unit 6 (p < .05, Cohen’s d = 0.35)

24. Learning Effects • Next expression-writing section (Unit 10) • Stories and equations more complex • Experimental group still significantly better at writing expressions in Unit 10 (p < .01) • 6% difference in expression-writing in Unit 10 (40% control vs. 46% experimental) • Also maintain learning efficiency gain (6.26 second reduction; p < .01) • Learning impact greater (3 times larger) for weaker students (attrition)

25. Conclusions • Interventions designed to elicit interest have the potential to support learning, even in advanced domains like algebra • Adaptive technology environments that personalize instruction can impact learning of difficult skills • Need to further explore how interest can be leveraged in adaptive environments, more authentically • Develop stronger theory behind learning from personalization interventions

26. Future Directions • Expand intervention to 4 units • Collect measures at multiple grain sizes of how personalization interacts with: • Affective states (boredom, frustration, engagement) • Triggered and maintained situational interest • Individual interest and utility value • Self-efficacy • Achievement goals • Metacognitive strategies and “gaming the system”

27. Future Directions • More authentic interventions: • Classroom study of teachers implementing “personalized” versus “normal” units – richer problem contexts or mini-projects • Classroom study where Algebra I students generate their own “personalized connections” – utility value interventions • “Personalized” visual representations (NCCMI) • Analysis of Mathia data sets (used in over 100 schools in first year)

28. Significance • A recent national survey of Algebra I teachers found that: • “Working with unmotivated students” as most challenging aspect of teaching algebra • Second place: “Making mathematics accessible and comprehensible to all students.” (Loveless, Fennel, Williams, Ball, & Banfield, 2008) • Adaptive learning technologies offer a potentially powerful means to enhance motivation and achievement for students who struggle with mathematics

29. Acknowledgements • Milan Sherman • Anthony Petrosino • Mitchell Nathan • Jim Greeno • Ken Koedinger • Ryan Baker • Vincent Aleven • The Pittsburgh Science of Learning Center • Carnegie Learning

30. Papers • Walkington, C., & Maull, K. (2011). Exploring the assistance dilemma: The case of context personalization. In L. Carlson, C. Hölscher, & T. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 90-95). Boston, MA: Cognitive Science Society. • Walkington, C., & Sherman, M. (2012). Using adaptive learning technologies to personalize instruction: The impact of interest-based scenarios on performance in algebra. In Proceedings of the 10th International Conference of the Learning Sciences. Sydney, Australia. • Walkington, C., Sherman, M., & Petrosino, A. (2012). ‘Playing the game’ of story problems: Coordinating situation-based reasoning with algebraic representation. Journal of Mathematical Behavior, 31(2), 174-195. • Walkington, C., Petrosino, A., & Sherman, M. (in press). Supporting algebraic reasoning through personalized story scenarios: How situational understanding mediates performance and strategies. Mathematical Thinking and Learning. • Walkington, C. (under review). Using learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes. Invited to special issue of Journal of Educational Psychology.