Continuous Improvement in Teaching and Learning

1. Continuous Improvement in Teaching and Learning Candace Thille Director, Open Learning Initiative

2. What is Carnegie Mellon’s Open Learning Initiative? Scientifically-based online learning environments designed to improve both quality and productivity in higher education

3. Goal-Directed Practice and Targeted Feedback

4. A computerized learning environment whose design is based on cognitive principles and whose interaction with students is based on that of a (human) tutor—i.e., making comments when the student errs, answering questions about what to do next, and maintaining a low profile when the student is performing well. What is a Cognitive Tutor?

5. Practice Synthesizing and Applying Skills & Knowledge

6. Feedback: Changing the Productivity of Learners and Teachers

8. LD project lead: Dr. Marsha Lovett

10. “The Killer App” Feedback Loops for Continuous Improvement

11. Learning Curve Analysis DataShop: Pittsburgh Science of Learning Center

12. Apply learning science research and scientific method to course development, implementation and evaluation Develop Interactive Learning Environments collaboratively. (teams of content experts and novices, learning scientists, HCI, software engineers) Feedback loops for continuous improvement OLI Review: What Difference Does it Make?

13. Accelerated Learning Results • OLI students completed course in half the time with half the number of in-person course meetings • OLI students showed significantly greater learning gains (on the national standard “CAOS” test for statistics knowledge) and similar exam scores • No significant difference between OLI and traditional students in the amount of time spent studying statistics outside of class • No significant difference between OLI and traditional students in follow-up measures given 1+ semesters later M. Lovett, O. Meyer, & C. Thille, C., “The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning,” Journal of Interactive Media in Education (2008).

14. Other Class Results • Large Public University: OLI Online vs. traditional. OLI 99% completion rate vs 41% completion rate traditional. • Community College accelerated learning study in Logic: An instructor with minimal experience in logic. Students obtained high levels of performance on more advanced content (~33%) not covered in traditional instruction. • OLI stoichiometry course:The number of interactions with the virtual lab outweighed ALL other factors including gender and SAT score as the predictor of positive learning outcome.

15. Quotes • Student Quote:“This is so much better than reading a textbook or listening to a lecture! My mind didn’t wander, and I was not bored while doing the lessons. I actually learned something.” • Instructor Quote: “The format [of the accelerated learning study] was among the best teaching experiences I’ve had in my 15 years of teaching statistics.”

16. Strategy for Educational Improvement

17. “Improvement in Post Secondary Education will require converting teaching from a ‘solo sport’ to a community based research activity.” —Herbert Simon

18. Community College Open Learning Initiative (CC-OLI) APLU/AACC Planning Grant WA & TX State Evaluation Projects OPEN Support for DOL TAACCCT Grantees Carnegie Foundation StatWay UMUC Course Adaptation ITHIKA OLPU 4 NGLC Wave 1 projects Get Involved: OLI Current Projects

19. 25% jump in successful course completion rates in 4 CCOLI gate-keeper/gateway courses at participating colleges 40 colleges participating in CCOLI course development, adaptation, evaluation An established model, platform & tools for testing and scaling innovation CC-OLI Outcomes

20. Select 5-7 high demand transfer level courses for OLI development Build agreements with APLU and AACC institutions to participate in development project. Develop a plan for the on-going support, improvement and maintenance of the courses. OLI, APLU & AACC Planning Grant