Quality Matters: Inter-Institutional Quality Assurance in Online Learning

1. Quality Matters: Inter-Institutional Quality Assurance in Online Learning Sponsored by the U.S. Dept. Education Fund for the Improvement of Postsecondary Education (FIPSE) SREB – DECEMBER 7, 2005 Wendy Gilbert, Director, MarylandOnline

2. Quality Matters: Inter-Institutional Quality Assurance in Online Learning • Grantor: FIPSE • Grant period: 9/03 – 8/06 • Award: $509,177 • Grantee: MarylandOnline • Statewide consortium: 14 community colleges, 5 senior institutions • http://www.QualityMatters.org

3. FIPSE Interested Because … • Quality assurance of online courses is important • Voluntaryinter-institutional assurance has never been done before • This can serve as a national model Quality Matters!

4. MarylandOnline • Statewide consortium dedicated to support of distance learning in Maryland • Partners: 15 community colleges, 5 senior institutions • Goals • Web gateway for online higher education in Maryland • Faculty training • Facilitate online course and program sharing • Facilitate collaborations among member institutions • Provide statewide leadership in distance education

5. Course Meets Quality Expectations Course Revision Course Peer Review Process • Institutions • CAO’s • AR’s Faculty Course Developers National Standards & Research Literature Course Rubric Faculty Reviewers Training Peer Course Review Feedback Instructional Designers

6. For Our Purposes, Quality Is… • More than average; more than “good enough” • An attempt to capture what’s expected in an effective online course at about an 85% level • Based on research and widely accepted standards 85 %

7. What this process is NOT • Not about an individual instructor (it’s about the course design) • Not about faculty evaluation (it’s about course quality) • Not a win/lose, pass/fail test (it’s about a continuous improvement process in a supportive environment)

8. A QM review is Ongoing Focus: course design Outcome: course improvement Voluntary, non-threatening Team approach that includes the faculty member Full disclosure to faculty A faculty evaluation is Single point in time Focus: delivery Outcome: decision on performance for promotion/tenure Win/lose situation Confidential/secretive QM Collegial Review vs. Evaluation

9. Strengths QM is grounded in: • research literature • national standards of best practice • instructional design principles

10. What’s In It For Institutions … • Stronger reaccreditation package • Validation by external bodies • Elevation of QA as a priority activity • Access to a sustainable, replicable, scalable QA process • Informed online course training & practices • Professional development activities • Increased course & program sharing (MOL)

11. What’s In It For Faculty … • Improvement of online course • Access to instructional design support • QA validation by external peers • Expansion of professional community • Opportunity to review other courses & gain new ideas for own course • Useful tool for annual evaluations, promotion applications, professional development plan/requirements • $150 for each completed peer course review

12. Peer Reviewers • Selection Factors • Prior training to teach online • Extent of online teaching experience • Currency of online teaching experience • Content area • Requirements • Sign MOU • Attend peer reviewer/rubric training

13. Review Teams • 3 Faculty Peer Reviewers: • 1 from home institution, 2 from others • 1 from same discipline, 2 from others • mix of CC & 4 yr schools • mix of large & small schools • mix of public & private schools • Faculty Course Developer • access to rubric prior to review • involved in pre-review discussions • consulted during review

14. Rubric Training Focus on: • Application of rubric to course review • Interpretation of review elements • Providing constructive feedback • Competency-based

15. Rubric • Based in • research literature • nationally recognized standards of best practice • instructional design principles • Used by review teams to: • assess course quality in 8 key areas (40 review elements) • provide feedback to faculty course developer • provide guidance to instructional design support team

16. The Rubric • Eight standards: • Course Overview and Introduction • Learning Objectives • Assessment and Measurement • Resources and Materials • Learner Interaction • Course Technology • Learner Support • ADA Compliance Key components must align.

17. Rubric Features • Living document • Web-based • Automated compiling of team report • Annotations • Examples

18. Rubric Scoring • Team of three reviewers • One score per standard based on majority • Two criteria to meet quality expectations: • “Yes” to all 14 Essential Standards • Receive at least a total of 68 points

19. Course Reviews • To date, 50% meet expectations on initial review • Instructional design support provided • Identified 11 common areas for improvement • Target for faculty training & course development/revision: • interaction • ADA • learning objectives stated at the module/unit level • self-check/practice activities with feedback • links to school's academic supports • instructor's self-introduction • netiquette expectations

20. National Participation Scope: • individuals from 70 different institutions (including the 19 MOL schools) in 14 different states • 300+ faculty trained to review online courses using the rubric Uses of QM System: • online course development, review, and revision, faculty training, formation of distance learning policies & steering committees, institutional reaccreditation packages

21. Awards - 2005 • WCET Outstanding Work (WOW) Award • USDLA 21st Century Best Practice Award • Maryland Distance Learning Association (MDLA) Best Program Award

22. External Partners • Sloan Consortium • Southern Regional Education Board (SREB) • Western Cooperative for Education Telecommunications (WCET) • Towson University (MD) • Kentucky Virtual University • Michigan Virtual Community College Consortium • Portland Community College (OR) • Florida Community College of Jacksonville (FL) • Raritan Valley Community College (NJ)

23. Advisory Board • Middle States Commission on Higher Education • MD Higher Education Commission • MD State Department of Education • Penn State University • US Naval Academy • Miami University (OH) • South Dakota Electronic University Consortium • Minnesota Online • Northern Virginia Community College • Bucks County Community College (PA) • Defense Acquisition University • Education Direct • Kaplan College

24. Lessons Learned • QM is part of an on-going process; course review is not the first step in this process • Must address and minimize faculty anxiety prior to review • Need for faculty training at individual institutions during course design and prior to implementing a review process • Need for pre-course development checklist tied to rubric • Approach to the Rubric and the Review process needs to be holistic

25. Looking Ahead • Adapt rubric & process for: • hybrid, f2f, ConEd, commercial, professional training • specific institutional needs • Promote the integration of the QM process within institutions • Explore the “QM Institution” concept • Assess the impact on student learning through research projects • Annual rubric update cycle • Diversify training program • Sustainability plan • Develop partnerships & business opportunities

26. More Information www.QualityMatters.org QM Project Co-Directors • Chris Sax csax@umuc.edu • Mary Wells mwells@pgcc.edu MarylandOnline Director • Wendy Gilbert gilbertwa@comcast.net Project Coordinator • Kay Kane kkane@pgcc.edu