Teaching in a Web-Based Distance Learning Environment: An Evaluation Summary Based on Four Courses

1. Teaching in a Web-Based Distance Learning Environment:An Evaluation Summary Based on Four Courses Charles Graham, Joni M. Craner, Byung-ro Lim, & Kursat Cagiltay, Thomas M. Duffy Center for Research on Learning and Technology Indiana University

2. Evaluation Scenario You have just been asked by the VP of your institution to evaluate its online courses. She asks you if there is any difference in how you would evaluate an online course compared to a face-to-face course. What do you tell her? Center for Research on Learning and Technology, Indiana University

3. Center for Research on Learning and Technology, Indiana University It happened to us . . . A similar scenario happened to the director of the Center for Research on Learning and Technology at Indiana University. . .

4. Center for Research on Learning and Technology, Indiana University Context for the Evaluation • The assistant Dean in a large academic program in mid-west requested an evaluation of their online program Their evaluation objectives: • Assess the quality of online instruction and provide recommendations for improvement to program directors • Provide feedback to the instructors about the quality of their courses and provide recommendations for improvement

5. Center for Research on Learning and Technology, Indiana University Evaluation Background • What was being evaluated? • We evaluated 4 online courses • Who was conducting the evaluations? • A team of 4 evaluators from the Center for Research on Learning and Technology under the leadership of Tom Duffy

6. Center for Research on Learning and Technology, Indiana University Evaluation Criteria • What Criteria was used? • They use the 7 principles of Good Practice in Undergraduate Education (Chickering & Gamson, 1987) to guide their on-campus courses • We decided to try and use the same criteria for online evaluations

7. Center for Research on Learning and Technology, Indiana University The Challenge The 7 Principles seemed to apply to both online and face-to-face courses. BUT . . . Specific evaluation strategies provided for each principle in a f2f context often did not apply to the online environment.

8. Center for Research on Learning and Technology, Indiana University The 7 Principles of Good Practice(American Association of Higher Education, 1987) Good practice: • Encourages student-faculty contact • Encourages cooperation among students • Encourages active learning • Gives prompt feedback • Emphasizes time on task • Communicates high expectations • Respects diversity

9. Center for Research on Learning and Technology, Indiana University Data Collection • We evaluated the courses looking for examples and non-examples of the 7 principles of Good Practice • We formulated a set of guidelines for each principle based on our findings • How was data collected? • Literature review • Interviews, and • document analyses (course materials, discussion forums, etc.)

10. Center for Research on Learning and Technology, Indiana University Findings These were our findings . . .

11. Center for Research on Learning and Technology, Indiana University Online Guidelines1. Encourage Student-Faculty Contact Instructors should: • Contact students (by phone or private email) who are not participating regularly to keep students engaged • Clearly communicate email response policies (set student expectations)

12. Center for Research on Learning and Technology, Indiana University Online Guidelines1. Encourage Student-Faculty Contact • Encourage students to contact instructor through private email messages with learning problems • Invite students to have on- or off-line appointments with instructor • Be respectful to students in all interactions (listserv reply)

13. Center for Research on Learning and Technology, Indiana University Online Guidelines2. Encourage Cooperation Among Students • Begin course with a structured ice-breaking activity where students share interests/backgrounds/etc. • Develop assignments that require meaningful peer interactionthat fosters asense of community/connectedness • Encourage students to share their life experiences

14. Center for Research on Learning and Technology, Indiana University Online Guidelines2. Encourage Cooperation Among Students • Require and teach students how to evaluate each other’s work according to specific guidelines (Critical peers) • Encourage students to praise and challenge each other • Require group assignments and provide a mechanism for assessing individual contributions

15. Center for Research on Learning and Technology, Indiana University Online Guidelines3. Encourage Active Learning • Ask students to relate course content to real-world issues/experiences • Require students to do authentic projects and assignments • Require students to present their work to the class for peer feedback

16. Center for Research on Learning and Technology, Indiana University Online Guidelines3. Encourage Active Learning • Structure assignments and bulletin board discussions activities to require students to challenge the ideas of the instructor, of other students, or those presented in the readings or other course materials • Provide a mechanism for students to add relevant resources for the class

17. Center for Research on Learning and Technology, Indiana University Online Guidelines4. Give Prompt Feedback • Provide information feedback on assignments and projects according to the written policy • Provide acknowledgement feedback to students within 24 hours • Monitor group bulletin boards regularly and give specific information feedback to students

18. Center for Research on Learning and Technology, Indiana University Online Guidelines5. Emphasize Time on Task • Structure assignments with deadlines that are spread throughout the semester • Require students to participate in the online forum on a regular basis • Emphasize the importance of regular work, self-directed learning, sound self-pacing, and scheduling.

19. Center for Research on Learning and Technology, Indiana University Online Guidelines5. Emphasize Time on Task • Use easily accessible resources and prepare sufficient resources. Avoid unnecessary time spent on finding materials.

20. Center for Research on Learning and Technology, Indiana University Online Guidelines6. Communicate High Expectations • Clearly list expectations from the students on the course Web site • Provide examples of exemplary performance to students. • Publicly call attention to excellent performance by the students • Reinforce expectations during the duration of the course

21. Center for Research on Learning and Technology, Indiana University Online Guidelines6. Communicate High Expectations • Periodically discuss how well the class and individual groups are doing during the course of the semester • Develop a mechanism for evaluating individual participation and contribution to group projects.

22. Center for Research on Learning and Technology, Indiana University Online Guidelines7. Respect Diverse Talents and Ways of Learning • Learn about the different backgrounds and interests of the students • Allow students to shape their own coursework by choosing project topics • Encourage students to express their diverse points of view

23. Center for Research on Learning and Technology, Indiana University Human Computer Interaction Recommendations • Consistency of Web site layout and design • Clear Organization and Presentation of Information • Consistent and easy-to-use navigation • Aesthetically Pleasing Design and Graphics

24. Center for Research on Learning and Technology, Indiana University Instructor Sharing and Collaboration Problem Asynchronous conferencing requires different methods and strategies from face-to-face conferencing.

25. Center for Research on Learning and Technology, Indiana University Guidelines for using Asynchronous Conferencing Effectively • Require student participation (make grade dependent on it) • Students need a task to focus their discussion • The task should have a product or something that brings closure • The task must engage the learner in the content • The discussion structure should be carefully thought out

26. Center for Research on Learning and Technology, Indiana University Guidelines for using Asynchronous Conferencing Effectively (cont’) • Discussion must be evaluated based on quality of content and not length of posting or number of postings • Instructors should post examples of expectations for discussions • Students must get some kind of feedback on the discussions • Discussion groups should remain relatively small

27. Center for Research on Learning and Technology, Indiana University Course Management Problem Managing a course online was much different from managing a face-to-face course. Instructors start to burn out trying to manage interactions in the same way they might in the classroom.

28. Center for Research on Learning and Technology, Indiana University Online Course Management Strategies • Increase emphasis on peer evaluation and feedback. • Increase the emphasis on group work. • Selectively evaluate discussions. • Set clear feedback expectations.

29. Center for Research on Learning and Technology, Indiana University Questions?

30. Center for Research on Learning and Technology, Indiana University Contact Information • Presentation Online at: http://php.indiana.edu/~crgraham/dl2000.ppt • Paper Online at:http://crlt.indiana.edu/crlt00-13.pdf • Email Contact:Charles Graham: crgraham@indiana.edu Byung-Ro Lim: byunlim@indiana.eduJoni Craner: jcraner@indiana.edu