Flip the Script to Improve Learning Measurably

1. Flip the Script to Improve Learning Measurably How Learning Engineered Online Courses Coupled with Learning Analytics Improves Learning Outcomes. Michaele J. Brown, PhD Educational Technology Consultant

2. Agenda

3. Trends in Higher Education • Greater emphasis on feedback & formative assessment • Flipped instruction • Learning dashboards displaying real-time student performance • Adaptive instruction informed by real-time student performance

4. Learner-Centered Instruction • Learning results from what the student does andthinks and only from what the student does and thinks. The teacher can advance learning only by influencing what the student does to learn. • Herbert Simon, 2001 It’s not teaching that causes learning. Attempts by the learner to perform cause learning, dependent upon thequality of feedback and opportunities to use it. Grant Wiggins President, Center of Learning Assessment

6. Practice and Feedback • Clear, observable, measureable, performance goals • Models of goals • Practice directed toward specific levels of performance • Feedback on performance against goals and models • Self-assessment and opportunities for further practice

7. Flipped Learning Before Class In-Class Facilitated Instruction • View video lecture • Read & reflect on text • Learn by doing Individualized Learning Collaborative Work Ref: Flipped Learning Networkhttp://fln.schoolwires.net/FLN Founders: Aaron Sams and Jonathan Bergmann

8. Flipped Learning • Four Pillars of F-L-I-P: • Flexible environments • Anytime, anywhere access, at your own pace. • Intentional content • Video lectures, text resources, and active learning • Learning culture • Learner-centered instruction • Professional educators • Coaches, guides, tutors, and facilitates Ref: Foundations of Flipped Learning http://fln.schoolwires.net//site/Default.aspx?PageID=39 Founders: Aaron Sams and Jonathan Bergmann

9. Transformation of Higher Education Flipped Instructional Pedagogy Goal Directed Practice & Feedback Adaptive Learning Real-time Learning Analytics

10. Open Learning Initiative

11. Openonline learning environments based on the science of learning An approach to designing, developing, delivering and improving learning experiences What is the Open Learning Initiative?

12. Goal Directed Practice & Targeted Feedback Learning activities that enact instruction: Intelligent tutoring systems Simulations Videos & animation Virtual Labs

13. Team-based Design and Development

14. Data drives powerful Feedback Loops Feedback: Changing the Productivity of Learners and Teachers

15. OLI Research Results • OLI students completed course in halfthe time with half the number of in-person meetings • OLI students showed significantly greater learning gains on the national standard “CAOS” test for statistics knowledge • No significant difference between OLI and traditional students in: • Amount of time spent studying statistics outside of class • Follow-up measures given 1+ semesters later M. Lovett, O. Meyer, & C. Thille, C., “The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning,” Journal of Interactive Media in Education (2008).

16. Community College OLI Domain experts outside of the core development team provide vital feedback

17. Multi-Institution Participation • “Improvement in post secondary education will require converting teaching from a solo sport to a community based research activity.” Herbert Simon Nobel Laureate & CMU Professor • Three levels of participation by faculty SMEs • Institutional participants: • Butte Community College • California Polytechnic State University • National University • Prince George Community College • Redwood Community College • Salisbury University • Towson University • UMUC

18. Aligning Learning Outcomes and Learning Objects

19. The Course Design Triangle Objectives Descriptions of what students should be able to do at the end of the course Assessments Tasks that provide feedback on students’ knowledge and skills Instructional Activities Contexts and activities that foster students’ active engagement in learning

20. OLI Used in Hybrid Mode

21. Analyzing Student Performance

22. Introduction to Biology Course Key: bio-mjb-v1P

23. Learning Activities Learn by Doing Did I Get That Key Terms Confidence Checks Assessments

24. Learning Dashboard

25. Student Performance

26. Student Participation High Performers

27. Student Participation Moderate Performers

28. Interaction Level Data Misconception between genes and chromosomes

29. OLI’s Data Driven Process

30. LearnLab:Transforming Education Research Ed tech + wide use = “Basic research at scale” NSF Science of Learning Center • 10 years, ~$50 million • Tech enhanced courses, assessment, & research • School cooperation for data collection

31. Learning Curve Analysis DataShop: Pittsburgh Science of Learning Center

32. Other Learning Curves

33. More helpBasics More challenge Understanding Thousands of combinations, what works best depends … Gradually widen Study examples 50/50Mix Test onproblems Study Study 50/50 Test Study 50/50 Test Mix Concrete Abstract Concrete Mix Abstract No feedback Immediate Delayed No feedback Immediate Delayed Block topics in chapters Interleave topics Fade Interleave topics Block topics in chapters Fade Ask for explanations Ask for explanations Mix Mix Explain Explain What’s best? Focused practice Distributed practice

34. Flat learning curve Discovery opportunity! Better Insight Better Courses Past research • Careful “cognitive task analysis” produces much better courses OLI Chemistry, equilibrium topic: Previously: 20% correct on exam After Redesign: 60% • New opportunity • Ed tech provides data for automated analysis

35. The “Killer App” Data Collection & Feedback Loops for Continuous Improvement

36. OLI Courses http://oli.cmu.edu/ Introduction to Biology

37. OLI Summary Apply learning science research and scientific inquiry to course development, implementation and evaluation. Develop intelligent learning platforms collaboratively with a team of content experts and novices, learning scientists, HCI, software engineers. Employ feedback loops for continuous improvement. Establish communities of use, evaluation and improvement.

38. Challenges to Implementing Adaptive Instruction

39. Challenges for Adaptive Learning Deciding on clear and measurable learning goals and objectives Providing good quality and timely feedback Visualizing and interpreting leaning analytics, and implications for adapting instruction. Faculty development for productive uses of class time for learner-centered activities

40. Adaptive Learning • Course End: • Summative Evaluation • Course Improvements • Course Delivery: • Learner-centered activities: • Opportunities for feedback & assessment • Authentic action based activities • Peer interaction and collaboration • Experiential, project and inquiry based activities • Self actualizing activities

41. Questions?

42. Thank You Michaele J. Brown, PhD Learning by design of authentic, goal-based, online experiences.

43. OLI Research Reports