Research That I nforms the 2011 – 2013 Edition of the QM Rubric TM

1. Research That Informs the 2011 – 2013 Edition of the QM RubricTM

2. Quality Matters Research Colleagues • Bethany Simunich, Ph.D. , Kent State University • Joan Milkalson, Ph.D., Excelsior College • Julie Frese, Ph.D., University of the Rockies • Li Wang, Ph.D., Ashford University • Sharon Lalla, Ed.D., New Mexico State University • Kay Shattuck, D.Ed., Quality Matters

3. Planned Take-Aways for this Session • Explanation of relationship between research and the QM RubricTM • A sampling of citations from the QM Research Library • Talking points for each of the 8 QM Higher Education general standards

4. QM and Research The QM RubricTM

5. QM Research Library • https://www.qualitymatters.org/research • https://www.qmprogram.org/qmresources/research

6. What’s in the QM Research Library? • As of today, we have documented 503 citations and sources (library currently being updated as we continue our review) • Not every reference for each standard is recorded in library

7. What’s in the QM Research Library? • Standard 1 - 25 references • Standard 2 - 32 references • Standard 3 - 54 references • Standard 4 - 31 references • Standard 5 - 235 references (91 of which came from this most recent lit review) • Standard 6 - 63 references • Standard 7 - 21 references • Standard 8 - 42 references (26 of which came from this most recent lit review)

8. Talking Points: Standard 1 (Course Overview & Intro) • Research supports the importance of early engagement in online courses. • How to get started and where to find course components is critical to student success. • Students want to know the purpose of the course and what’s expected from them. • Introductions from both the instructor and students establish social presence and build a sense of community to foster learning.

9. Standard 1: Examples from QM research library • The overall design of the course is made clear to the student at the beginning of the course. This is important to begin the engagement process for the learner by initial activities that will promote social presence. Anderson, T., Rourke, L., Garrison, D.R., & Archer, W. (2001). The effects of teacher discourse in computer-mediated discussion. Journal of Asynchronous Learning Networks, 5, 1-17. • Students can be easily confused regarding their obligations in the absence of clear direction. Group goals especially require clear and achievable objectives. (Studies often inform several standards) Murphy, K., Mahoney, S. & Harvell, T. (2000). Role of Contracts in Enhancing Community Building in Web Courses. Educational Technology & Society, 3(3) [Electronic version]. Retrieved December 20, 2002, from http://ifets.ieee.org/periodical/vol_3_2000/e03.pdf

10. Talking Point: Standard 2 (Objectives/Competencies) • When designing/re-designing a course for the online environment, studies show that focusing the design around stated learning objectives results in better learning. • Learning objectives can serve as goals, but students need information that illustrates how content and assessments support the achievement of those goals.

11. Standard 2: Examples from QM research library • Findings suggest that revising a course around stated objectives resulted in better student outcomes related to them. Swan, K., Matthews, D., Bogle, L., Boles, E., & Day, S. (2010). Linking online course design and implementation to learning outcomes: A design experiment. [2010 QM Research Grant University of Illinois Springfield] • Specific learning activities in blended learning courses must be connected to specific course content, and the objectives/outcomes must be well organized and “on time through the whole semester” (p. 334). Mortera-Gutiérrez, F. (2006). Faculty best practices using blended learning in e-learning and face-to-face instruction. International Journal on E-Learning, 5(3), 313-337.

12. Talking Points: Standard 3 (Assessment & Measurement) • Assessment items should direct learners to those aspects of a course that are of primary importance. • Having a wide variety of clearly explained assignments on a regular basis is deemed an effective online assessment strategy. • Effective assessment techniques include projects, portfolios, self-assessments, peer evaluations, and weekly assignments with immediate feedback. • Formative feedback should be aligned with SLOs and be timely, personal, manageable, motivational, and in direct relation with assessment criteria.

13. Standard 3: Examples from QM research library • Key issues for assessment of activity-based learning: assessment must reflect course philosophy; assessment is essential in creating learning opportunities at critical points; and assessment provides a vital opportunity for feedback, helping to complete the reflective learning cycle. Macdonald, J. & Twining, P. (2002). Assessing activity-based learning for a networked course. British Journal of Educational Technology, 33(5), 603-618. • Students do better in the final assessment tests if in-course practice exercises are aligned with content and style; intensity and frequency of a student’s participation in group discussions also seems to have a positive influence on final performance. Lewis, B. A. (2002). Learning effectiveness: Efficacy of quizzes vs. discussions in online learning. Dissertation International Abstracts, 63(03), 911A.

14. Talking Points: Standard 4 (Instructional Materials) • Students in a fully interactive multimedia-based e-learning environment achieved better performance and higher levels of satisfaction than those in a traditional classroom and those in a less interactive e-learning environment. • Faculty need to select manageable content and clearly explain the purpose of the activities. • Students can become overwhelmed (cognitive overload). • Students can achieve the intended learning outcomes when they have appropriate supports, structure, and focus.

15. Standard 4: Example from QM research library Web module length impacts recall and other variables. Study participants simply did not complete long modules. Pomales-García, C. & Lui, Y. (2006). Web-based distance learning technology: The impacts of web module length and format. The American Journal of Distance Learning, 20(3), 163-179.

16. Talking Points: Standard 5(Interaction & Engagement) • Course design and organization are factors of teaching presence (along with instructor facilitation). • There is a need to move away from efficiency-oriented design to innovation-oriented design.

17. Standard 5: Examples from QM Research Library • Deep and meaningful formal learning is supported as long as one of the three forms of interaction (student-teacher; student-student; student-content) is at a high level. The other two may be offered at minimal levels, or even eliminated, without degrading the educational experience. Anderson, T. (2003). Modes of interaction in distance education: Recent developments and research questions. In M.G. Moore & W. G. Anderson (Eds.), Handbook of distance education (pp. 129-144). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. • Students’ sense of community (SOC) was related to the students’ sense of satisfaction with the course, but it did not correlate with course grade or retention. Students had varying levels of desire for SOC. Drouin, M. (2008). The relationship between students’ perceived sense of community and satisfaction, achievement, and retention in an online course. Quarterly Review of Distance Education,9(3), 267-284. 

18. Standard 5: Another example from QM Research Library • Impact of social interaction in online learning communities on retention rates. Concluded (n=28,000) there is a significant impact of social and cognitive presence on retention. “… students who positively perceive online learning environments, which is potentially increased by their perception that they are part of a larger (social) learning community, are more likely to have increased retention.” Boston, W., Diaz, S. R., Gibson, A., Ice, P., Richardson, J., & Swan, K. (2009). Journal of Asynchronous Learning Networks, 13(3), 67-83.

19. Talking Point: Standard 6(Course Technology) • Incorporating current learning technologies can support both interaction and collaboration. • Utilizing various course tools and media respects the diversity of learners/learning styles in online courses. • Pedagogy should drive the choice of what technologies to employ.

20. Standard 6: Examples from QM Research Library • “. . . more is not necessarily better” and warned of potential dangers resulting from imposed interaction as it might interfere with student autonomy in managing time, place, and pace of learning. Kramarae(2003), Cook (1989), & May (1993) • Confusion by too many technology options in a virtual classroom along with technical issues was frustrating to students in blended learning courses and often had negative impact on students’ involvement. McBrien, J. L. & Jones, P. (2009). Virtual spaces: employing a synchronous online classroom to facilitate student engagement in online learning. International Review of Research in Open and Distance Learning, 10(3) , 1-17. • Characteristics of high quality digital games: “promote student learning through exploration, interactivity, trial and error, and repetition in such a way that students got so lost in the fun, that they don’t realize they are learning at the same time” (pp. 5-6). Green, M. & NcNeese, M. N. (2007). Using edutainment software to enhance online learning. International Journal on ELearning, 6(1), 5-16.

21. Standard 7: Talking Points • Standard 7 is about evidence of “access” and not about evaluating the actual support services. • Learner support services help students to persist and successfully complete courses. • Students need information about support services and how to access them from within the course or learning management system. • Institutions must find meaningful ways to connect students to learner support services. • Four categories of support services: technical, accessibility, academic services, and student services.

22. Standard 7: Examples from QM Research Library • Understanding "instructions given by the system, which usually is not adequately adapted to user's prior knowledge or the vocabulary of a lay person" is a serious drawback (p. 3) Heckner, M., Schworm, S., & Wolff, C. (2009). Combining design patterns and elements of social computing for the design of user-centered online help systems. Journal of Educational Technology Systems 38(1), 3-20. • Perceptions of psychological presence a student working at a distance holds in relation with teachers, peer students, and institution can be significant predictors of learning outcomes. An institution’s transactional presence may be relatively more important than teachers’ and peer students’ transactional presence Shin, N. (2001). Beyond interaction: transactional presence and distance learning. Unpublished doctoral dissertation, Pennsylvania State University.

23. Talking Points: Standard 8 (Accessibility) • Driven by ADA Policy • Influenced by Universal Design • Envisioned to “Design Online Courses to Meet All Learner Needs” • Confronted with the Complexity of Learner Needs

24. Standard 8: Example from QM Research Library • Online classes need to be designed with all forms of disabilities in mind, not just the blind and the deaf. Success for disabled students is dependent on their individual learning needs being met. Universal design makes online instructional material accessible and understandable for all kinds of students with disabilities. Sapp, W. (2009). Universal Design: Online Educational Media for Students with Disabilities. Journal of Visual Impairment & Blindness.

25. Questions?

26. ~Sharon, Li, Julie, Joan, Bethany, & Kay