What have we learned about evaluating quality in distance education?

1. What have we learned about evaluating quality in distance education? @bsimunich Senior Research Colleague Quality Matters www.qualitymatters.org Dr. Bethany Simunich Director of Online Pedagogy and Research Kent State University, U.S.A.

2. What is quality? Why is it important? • Traditionally, quality has been determined by the perceived status of the institution and the expertise of its faculty (Rees, 2007). • Now, we are moving from trusting these “inputs” to focusing on results, or “outputs”, as there is a growing public concern for quality and affordability. • Quality is often not defined. Common attributes are excellence, consistency, value for money, fitness for purpose, and transformation (Harvey and Green, as cited in Chalmers & Johnston, 2012).

3. A Systems Perspective of Distance Education • Michael G. Moore and Greg Kearsley, 1996, 2012 • A framework in order to better understand quality. • Accounts for the interrelated parts that must function together, in a planned and strategic way, in order to achieve outcomes. • Component processes are learning, teaching, design, communication, and management.

4. Learning • Design • Teaching • Technology

5. LEARNING Grades Learning Summative assessment Exams, quizzes Formative assessment Students receive feedback to help them improve Indirect assessment Ask students

6. Determine learning outcomes Identify how the learning outcomes are addressed. Measure student achievement of learning outcomes. Ask students for feedback. Make revisions to the course. Measuring quality in student learning Walvoord, B.E. (2010). Assessment clear and simple: A practice guide for institutions, departments, and general education.

7. INSTRUCTIONAL DESIGN Content Instructional Design “… incorporates known and verified learning strategies into instructional experiences which make the acquisition of knowledge and skill more efficient, effective, and appealing.” Merrill, M. D., Drake, L., Lacy, M. J., & Pratt, J. (1996). Reclaiming instructional design.Educational Technology. 36 (5): 5–7.

8. Instructional Design and Learning • B.F. Skinner (1954), The Science of Learning and the Art of Teaching • Instructional materials should include small steps, frequent questions, and immediate feedback. • Robert F. Mager (1962), Preparing Objectives for Programmed Instruction • Create learning objectives based on what the student will be able to do in terms of performance, and how that performance will be evaluated. • Benjamin Bloom (1956), Bloom’s Taxonomy • Three domains of learning: cognitive, physical, and affective

10. Instructional Design Analyze Design Develop Implement Evaluate 9 Learning Events, including stimulating recall, providing guidance and feedback Results-focused design

11. Backward Design Wiggins, G., & McTighe, J. (2005). Understanding by Design (Expanded 2nd ed.). Alexandria, Virginia: Association for Supervision and Curriculum Development.

12. Course Overview and Introduction Learning Objectives (Competencies) Assessment and Measurement Instructional Materials Course Activities and Learner Interaction Course Technology Learner Support Accessibility and Usability Quality Instructional Design https://www.qualitymatters.org/rubric Contact: Yaping Gao, Ed.D Senior Academic Director, Programs & Services ygao@qualitymatters.org

13. Quality Matters rubric for instructional design • Course Overview and Introduction: do students know how to begin? Do they know what their course is about and who their professor is? • Learning Objectives: Are the objectives measurable? • Assessment and Measurement: Do the assessments measure student achievement of the learning objectives? Are students able to assess their progress? • Instructional Materials: Do they support students in performing well on the assessment? Do they know how they should use the materials?

14. Quality Matters rubric for instructional design • Learner Interaction: Do the activities provide opportunities for interaction that support active learning? • Course Technology: Is technology used to support the learning objectives? • Learner Support: Do students know where to go for help? • Accessibility and Usability: Is content accessible? Does the course have good navigation and organization?

15. TEACHING Grading Teaching Gagne: - Providing relevance so that students will know what they will be able to do because of the instruction- Providing guidance to students to help them understand- Providing feedback to students to help them improve

16. Transactional Distance Theory • Developed by Michael G. Moore in 1970s/1980s • The theory of cognitive distance between instructors and learners • It is a psychological and communication space that must be minimized in order to maximize learning • Components of dialog (interaction), structure (of the instruction) and learner autonomy • Transactional distance can leave students feeling isolated and disconnected which can reduce motivation and engagement, and can also lead to students dropping the course.

17. Community of Inquiry framework Garrison, Anderson, & Archer, 2000

18. Quality Online Teaching Based on Chickering & Gamson (1987), Seven Principles for Good Practice in Undergraduate Education http://facdev.e-education.psu.edu/sites/ default/files/PeerReview_OnlineCourses_PSU_ Guide_30November2015.pdf

19. Peer Review Guide for Online Course Penn State University • Frequent and timely student-faculty contact. • Interaction is used to enhance learning. • Active learning methods are used to encourage students to discover, process, and apply information. • Instructors provide frequent feedback so students can reflect and improve. • Time-on-task is emphasized. • Communicate expectations and provide encouragement. • Use a variety of assessments and make learning “personal”.

20. TECHNOLOGY More Technology Use Technology to Support Pedagogy and Support Students “The same tools used to automate, standardize, and replicate instruction can be also be used to make a more tailored experience for both the student and the instructor.”- Dr. Ron Legon

21. Who determines quality, and how? • There are varying components to quality, and the degree of importance given to those components depends on which interest group is going to interpret quality (Lampikoski, 1995) • Exoteric influences: government, employers, professional accreditation bodies • Esoteric influencers: administration, professors, students • These stakeholders value different aspects of education (Thompson & Irele, 2007)

22. The Iron Triangle: Constraints • “Widen access by packing more students into each class and people will say quality is slipping. Improve quality with better learning resources and costs will go up. Cut costs and we damage both access and quality.” • Sir John Daniel, 2011 • New educational technologies: mLearning, OER, cMOOCs, cloud computing and analytics • Stella C.S. Porto, in • Assuring Quality in Online Education, • ed. Kay Shattuck

23. The New Iron Triangle: Priorities • Power & Morven-Gould, 2011 Administrators: Cost Effectiveness Faculty: Quality • Each stakeholder group will have to meet their own needs and priorities at a predetermined threshold. Students: Accessibility

24. “Who is responsible for quality?” - Dr. Darcy Hardy All of Us

25. MuitoObrigada! Dr. Bethany Simunich bsimunic@kent.edu Twitter: @bsimunich