Benchmarking for Quality in Distance Learning Richard T. Hezel, Ph.D.

1. Benchmarking for Quality in Distance Learning Richard T. Hezel, Ph.D. Josh Mitchell June 11, 2006

2. Background • “Quality” focus at NUTN annual conferences, 2004-2006 • NUTN-Hezel collaboration on benchmarking research • Resulting report:Benchmarking for Quality in Distance Learning

3. What is benchmarking? • Applying common, consistent standards, methods, observations, measurement • Identifying common metrics for process, costs, and other measures of quality • Using quantitative and qualitative standards to compare performance over time • Establishing a point of reference

4. What is benchmarking? • Establishment of an agreed upon point of reference based upon comparison studies. • The continuous process of measuring your outcomes, processes and practices against strong competitors, peers, or recognized distance learning leaders.

5. Benchmarking Research • Benchmarking requires comparisons using an agreed upon vocabulary and metrics. • Research on quality in online instruction has been, primarily, within a course, program or single institution. • Guides provided by accrediting agencies and traditional higher ed evaluation practice.

6. Benchmarking Research • Campus Computing Survey • Sloan-C “Entering the Mainstream” • NUTN-Hezel Associates benchmarking research

7. Challenges to Benchmarking Efforts • Quality is contextual. • Definitions and metrics must be agreed upon and applied in context. • Example: • The Framework for Higher Education Qualifications • Subject Benchmark Statements • Quality Assurance Agency

8. Quality Standards – Corporate vs. Higher Education (HE) • Quality assumed in HE for most of its history. (Judith Eaton, CHEA) • Innovation (in technology) and accountability for funding drives the need to measure and prove quality • Application of corporate-focused quality concepts (Deming, Juran, Baldridge, Drucker, Marchese) • Establishment of standards for measurement of quality • HE slow to accept framework and vocabulary of quality from industry. • TQM and other systems have been seen as fads

9. Faculty • Nearly 3 million learners are taking online courses. A growing proportion of regular faculty are teaching online. • Online instructors include regular faculty teaching in load, regular faculty teaching overload, adjunct faculty, and graduate assistants. • Need to clearly define terms and taxonomy and standards of measurement

10. Faculty Training Benchmarks(68 schools) • 61% of schools have optional instructional design help • 51% have voluntary faculty training • 16% have mandatory training • 58% of schools evaluate course design only through end-of-course student evaluations • Source: Carole Hayes, Florida State University

11. Course completion rates (programs=358) • Two-year schools 58 56.9 % • Four-year schools 5 84.3% • Upper division only 6 84.9% • Graduate schools 7 85.5% Source: Ingle, F. K.  Student retention and completion rates in a postsecondary online distance learning environment.

12. Is this Benchmarking? • No, it is a description of the environment. • Next step, define terms and context to enable comparisons – IQAT, a new tradition. • Comparisons provide data from which to establish benchmarks; a subjective endeavor requiring collaboration. • Continuing evaluation, data collection, and active comparison develop a rich culture and base of evaluation and benchmarking as an ongoing and responsive practice.

13. NUTN-Hezel Associates Benchmarking Initiative • Benchmark framework research: 3/05-6/05 • Focused benchmark research: 6/05-11/05 • IQAT development: 12/05-present

14. NUTN-Hezel Associates Benchmarking Initiative • Key questions: • What are colleges/universities measuring? • What are they benchmarking? • What do they think are the most important elements to benchmark? • What would be useful to benchmark internally and externally?

15. Survey Results • Key findings: • 54% say they’re benchmarking • But, few similarities in the types of institutions being benchmarked against • Benchmarking by convenience • Unsystematic “benchmarking”

17. IQAT helps distance learning leaders measure and manage change.

18. How IQAT Works • Input – • The process starts with the submission of institutional data

19. How IQAT Works • Output– • IQAT generates tables and graphs that show the your institutional data benchmarked against your peer group

20. What does IQAT benchmark now? • Programs & Courses • Enrollment & Completion Rates • Tuition • Library Services • Faculty

21. Coming soon… • Revenue • Expenditures • Retention • Student Services • Student Satisfaction

22. What IQAT Offers • Defines quality operationally, providing a foundation of comparison administrators can use to make informed decisions • Builds on the distance learning community standards as the baseline for continuous improvement • Reinforces outcomes assessment and facilitates the valuation process

23. Next Steps • Join us at tomorrow’s technology forum for a live tour • Subscribe at www.IQAT.org • Special pricing for NUTN attendees • Compare your institution against others in the database • Tell your friends about IQAT • Perpetuate database growth

24. For more information • Richard Hezel • richard@hezel.com • Josh Mitchell • josh@hezel.com • Hezel Associates, LLC • Syracuse, NY • 315 422 3512