Managing Quality in Learning Effectiveness

1. Managing Quality in Learning Effectiveness John Sener, Sener Learning Karen Swan, RCET, Ohio State Karen Vignare, Michigan State

2. Managing Quality in Learning Effectiveness • Learning outcomes measure learning effectives—right? • Quality model must begin with inputs, move to process and then measure outcomes • This session will use this model versus a focus on learning outcomes

3. John Sener • Founder, Sener Learning Services • Supporting the evolution of online learning environments • Practice areas: knowledge development, evaluation, learning design, strategic planning • Some current projects: • MarylandOnline’s Quality Matters • Michigan State University’s Citizen Planner Online • Maryland Students Online Consortium (K-12) • Editorial Board, JALN; Contributing editor, EdPath • 25+ yrs in education; Unique mélange of experience

4. Karen Swan • Research Professor in the Research Center for Educational Technology at Kent State University. • Her current research focuses on online learning, mobile computing and on student learning in ubiquitous computing environments • Long-term research has been focused mainly in the general area of media and learning • She served as a project director on several large scale grants for the US Department of Education, the National Science Foundation, and the NYC Board of Education • She is an Effective Practices Editor for the Sloan Consortium, the Special Issues Editor for the Journal of Educational Computing Research, and Editor of the Journal of the Research Center for Educational Technology.

5. Karen Vignare • Director MSU Global Ventures • Eight years various DL experience • Faculty, marketing, research & administrative experience • Research interests—blended, OL business strategies, OL demographics & retention • Attending doctoral class at Nova Southeastern

6. LEARNING EFFECTIVENESS COST EFFECTIVENESS ACCESS Quality STUDENT SATISFACTION FACULTY SATISFACTION Sloan-c Pillars

7. Learning Effectiveness • Sloan goal--online is as good as other methodologies • The quality of learning is deemed to be as good across learning methodologies • Learning output can be measured • Moving beyond traditional outcomes—interaction, higher order thinking, networks and community

8. Learning Effectiveness Metrics • Grades, GPA, outcomes, withdrawals, failures, repeats • Graduation and retention • Employer satisfaction • Certification tests and exams • Learner perceptions • Reported interaction

9. Learning Effectiveness • Meta-analyses show ALN as statistically equal to F2F • Furthermore, studies which show BL in the ALN pool tend to have better results • Single case study comparisons show variations in what method is best • Variations in outcomes seem to exist in academic disciplines

10. Decision Tree Modeling Of Factors Influencing Student Success In Online Courses(Charles Dzuiban & Patsy Moskal, UCF) Overall 85.9% n=11,286 Arts & Sciences, Business Admin., Hospitality Mgmt. Health & Pub. Affairs Engineering Education 85.8% n=6,460 72.7% n=378 91.5% n=2,079 86.7% n=2,369 O F2F, E, M F2F E, M, O F2F E, M 89.1% n=1,043 79.6% n=230 94.1% n=1,036 64.7% n=148 74.8% n=821 86.5% n=5,639 females males A&S BA & Hosp. mgmt 84.1% n=2,376 78.5% n=526 88.4% n=3,263 68.9% n=298

11. Learning Effectiveness • Traditional graduation, retention measures remain unclear because of definitions • Work on ALN completion which includes BL shows lower than F2F • Single cases (RIT, UCF, UW (Madison, Milwaukee)) show completions or drop outs similar • Difficult to compare until graduation rates are reported

12. Course Completion rates Number Programs in Sample 358 Two-year schools 58 56.87% Four-year schools 5 84.31% Upper division only 6 84.94% Graduate schools 7 85.53% Totals for Responding Programs 61.23% Ingle, F. K. Student retention and completion rates in a postsecondary online distance learning environment.

13. Comparative Data • Course Completion rates for traditional courses much harder to locate • Focus is on graduation rate • Comparative Graduation rates • Two-year 35% • Four-year 50% • Doctoral 35% (although course completions are generally high—ABD factor)

14. Blended Learning • BL adds to learning environment—visible constructivism & social networking • BL learning strategies clearly allow for more learner-centered strategies for students • Very difficult to measure the blend and whether it is contributing positively to learning effectiveness

15. Measuring Quality • For-profits leading total quality assurance movement • Metrics include inputs—class size, standardized design, content & services, technology • Process is systematized • Outcomes measured by market share, student success and profits

16. Other Improvement Models • ISO Standards • Products, Services—determines outcomes, organizations agree to change inputs and processes to achieve • Baldridge Quality (UW Stout) • Customer Relationship Management

17. Value of Contribution • Factorial analysis • Example: Tinto’s retention research • Design experiments to test contributing factors • Classroom Community Index (Rovai) • Shea decision trees using CCI

18. Learning effectiveness Summary • Benchmark (tools like IQAT) • Measure impact of changes • Look outside of education for process improvement models • Challenge Status Quo