Can Personalized Learning Improve Motivation and Student Success in Online Learning?

1. Can Personalized Learning Improve Motivation and Student Success in Online Learning? Anne E. PembertonAssociate Director, Library Assessment & Instructional Services GLTC: October 11, 2013

2. Literature Review: Overview • Identification of problems in online learning • Attrition (e.g. drop out) • 10 to 20% higher attrition rates compared to f2f courses • Lack of success (e.g. lower grades, less satisfaction) • Motivation • Critical issue in learners’ success or lack of success in all learning environments, but particularly in online environments • “Personalized Learning” • Potential solution?

3. Literature Review: Online Learning • Average annual increase in students taking online courses: • Over 18% from 2002 to 2010 (Allen & Seaman, 2011) • At least 31% of higher education (HE) students taking at least 1 online course (Allen & Seaman, 2011) • Increase from 1.6 million HE students taking at least 1 online course in 2002 to over 6.1 million students taking at least 1 online course in 2010 (Allen & Seaman, 2011)

4. Literature Review: Motivation • Motivation • One of the factors most frequently identified as contributing to drop out and/or success in online courses (Bekele, 2010; ChanLin, 2009; Clem, 2004; Hartnett, 2010; Hartnett, St. George, & Dron, 2011; Holder, 2007; Ivankova & Stick, 2005; Keller, 1999, Keller, 2008; Kim & Frick, 2011; Knowles & Kerkman, 2007; Levy, 2007; Lim & Kim, 2003; Lynch & Dembo, 2004; Müller, 2008; Parker, 2003; Sansone, Fraughton, Zachary, Butner, & Heiner, 2011; Wighting, Liu, & Rovai, 2008; Xie, DeBacker, FergusoXie, DeBacker, & Ferguson, 2006)

5. Literature Review: Personalized Learning • Solution to issues relating to attrition, specifically motivation, in online learning? • Relevance and satisfaction (part of motivation) are two specific factors contributing to attrition in online courses (Park and Choi, 2009) • Learners are less likely to drop out when they feel satisfied with courses and when courses are relevant to their lives (Chapman, 2006; Park and Choi, 2009)

6. Literature Review: Personalized Learning • Personalized Culture • Everything is customizable, personalized • Companies: “Market of One” • Amazon.com (individual book recommendations) • Grocery stores (a coupon for what you just bought …) • Facebook.com (individual ads) • Match.com (personalized dating) • iTunes (make everything “your own”) • Data collection • Why can’t education do the same for each student to understand one’s learning performance level and learning preferences? (Wolf, 2010)

7. Literature Review: Personalized Learning • Such a Change in Education May Not Be So Easy • Big shift in culture • Students AND educators may not be prepared; transition difficult • Teacher-centric culture long standing • Students depend on instructor for motivation (due dates, grades, reminders, etc.) • Might be especially difficult for online learners who already struggle with motivation

8. Literature Review: Personalized Learning • Defining “Personalized Learning” • Many definitions • No consensus

9. Literature Review: Personalized Learning • “Personalization refers to instruction that is paced to learning needs, tailored to learning preferences, and tailored to the specific interests of different learners. In an environment that is fully personalized, the learning objectives and content as well as the method and pace may all vary.” • Encompasses individualization and differentiation, but also allows students to draw on their personal interests to direct learning objectives and content • U.S. Department of Education’s Office of Educational Technology (2010)

10. Literature Review: Personalized Learning • History (New and Not So New) • Important to acknowledge this is not a “brand new” concept • Not widely accepted or well defined • Oral histories, apprenticeships, one-on-one tutoring, mentoring vs. formal, compulsory delivery to batches of students (Anderson, 2011) • Early 1900’s: Progressive education, child-centered movement, John Dewey • Early 1930’s: Individual differences; later learning environments • 1950’s: Bloom’s mastery learning • 1960’s / 1970’s: Personalized System of Instruction (PSI) • 1970’s to 1990’s: Special education, “personalized education” • 1990’s to Present: PSI studied; call for “personalized learning” (K-12)

11. Literature Review: Personalized Learning • What’s Changed? • Technology • Vast amounts of information available • Anywhere, any time access • Various formats of information • Collaboration and sharing tools • Ability to explore individual interests • Track and manage data • By integrating the principles of personalized learning with the tools of technology, some educators argue that they can create the kind of customized learning environment that has the potential for breaking schools out of the industrial-age model of education and bring about true 21st century school reform (Demski, 2012a). http://www.geekyblog.net/2012-01/

12. Literature Review: Personalized Learning and Motivation • Rich discussion; little research • No clear research to indicate what aspects of personalization increase learner motivation • No best practices

13. (“My”) Framework for Personalized Learning • Could consider a variety of factors, conditions, ideas, etc. for personalized learning • Started with Martinez’s “Whole Person Approach” • Comprehensive view of psychological factors • Achieved through Learning Orientations • Learning Orientations Questionnaire helps identify Learning Orientation • Used to identify what is important to learners (values, emotions, etc.) combined with • Consideration of learners’ interests and goals

14. Design Strategies for Learning Orientations

15. Design Specifications for Personalization Learners choose objectives

16. Methodology: Design • 3 versions of an online course module developed using previously outlined framework/design • 6 sections of UNCW’s First Year Seminar divided into 3 groups: • 1 control group plus 2 treatment groups • Roughly 50 students in each group • All students completed a pre and post-test • Timing depended on group placement

17. Methodology: Procedure • Group 1: Control Group • No differentiation, individualization, or personalization • Meant to simulate “typical” online instruction (but?) • Materials selected for broad range of learners • Materials “gathered” from previously developed materials • Aligned with instructional objectives, strategies, and assessment items • Sequenced based on task analysis/scaffolding • Not to toot horn, but design better than “typical”? • Order: Pre-test, then completed module, then post-test, and motivation instrument

18. Methodology: Procedure • Group 2: Differentiated Instruction • Same as Group 1 + • Further “grouped” and then modified into Learning Orientation “packages” • Used Bb’s “Adaptive Release” • Conforming learners given all details; Transforming learners given “big picture” • Order: Pre-test, then Learning Orientations Questionnaire, researcher scored and organized module, then students completed module, then post-test and motivation instrument

19. Methodology: Procedure • Group 3: Personalized Instruction • Same as Group 2 + • Further personalized using student’s name, student’s selected goals based on pre-test results, and student’s interests in majors to create “Personalized Learning Environment” (PLE) visible only to individual students • Order: Pre-test, then Learning Orientations Questionnaire, researcher scored, then students completed majors questionnaire and selected objectives of interest, researcher organized module, student completed module then post-test and motivation instrument

20. Methodology: Procedure Group 3: Jane Doe is a Transforming Learner and is interested in majoring in Psychology, Sociology, or Social Work. Based on the results of her pre-test and her own interests, Jane selected objectives 1, 2, 3, 7, 8, 9, and 12 to achieve.

21. Measurement and Assessment Instruments • Pre-test and Post-test (researcher designed) • Learning Orientation Questionnaire (LOQ) • Margaret Martinez • 25 item survey • Identifies individual’s “Learning Orientation” • Instructional Materials Motivation Survey (IMMS) • John Keller • 36 item survey • Each statement measuring scale in ARCS (Attention, Relevance, Confidence, and Satisfaction) • Group 3: Additional inventory to select objectives and select up to 3 majors of interest (researcher designed)

22. Results: Description of Participants • 146 students invited to participate • 80% completed ALL study components (117) • A participant might have completed the LOQ and the pre-test but not the post-test or IMMS • These participants were excluded from data analysis

23. Results: Pre-Test / Post-Test • Multiple analyses run on pre-test and post-test scoring • No statistically significant differences between groups for pre-test scores • No statistically significant differences between groups for post-test scores • Only statistically significant difference found was between groups for the difference between pre-test score and post-test score: (i.e. post-test score – pre-test score = “difference”)* * Some Group 3 participants scored lower on post-test!

24. Results: Pre-Test / Post-Test • Examined objective by objective results: • Whether or not selecting an objective had an impact on post-test scores • They were no more successful than the other groups

25. Results: Learning Orientations • No participants were found to be “Resistant” • More “Performing Learners” (48) than “Conforming” (18) and “Transforming” (10) combined • No statistically significant differences found between Learning Orientation groups with regard to pre-test or post-test or motivation

26. Results: Motivation • No statistically significant differences found between groups in relation to Instructional Materials Motivation Survey (IMMS) scores

27. Discussion: Conclusions • Personalized learning did not have a positive OR negative impact on either learner outcomes or learner motivation • Good news/bad news: • LMS can be used for PLE (Bb’s Adaptive Release) • Is possible to create personalized learning • Time consuming • Overloaded FYS students may not appropriate audience for personalized learning

28. Discussion: Limitations • Competing interests with Group 3? • Completed module, post-test, and motivation instrument later in semester than other groups • May have had other commitments • Time between pre-test and post-test an issue? • Prior knowledge should have been more carefully assessed • Assessment items should have been multiplied and more carefully crafted

29. Discussion: Limitations • Social conceptualization not possible in study due to time constraints and culture • Timing and Motivations • Group 1 completed all components at one time • Extrinsic motivations • Extra credit • Dropping quiz grades • FYS instructor priorities, messages, etc. • General motivation levels not measured

30. Discussion: Future Research • Was Group 1’s instructional design better than typical online learning? Was personalization negated for 3? • Include social aspect • One large section of an online course • Control for timing, extrinsic rewards, etc. • Different levels (undergraduate vs. graduate) • Discussions with students • Objective selection (why?) • Their perceptions of their own motivation

31. Discussion: Future Research • More research needed on motivation in relation to personalized learning • Should students really judge their own mastery in order to choose objectives? • K-12 to HE • Change in both cultures? • No consistency with online learning or personalized learning in K-12 currently

32. Questions? pembertona@uncw.edu