Engaging Students in Distance Learning

1. Engaging Students in Distance Learning Jim Waters Susan Gasson The iSchool at Drexel

2. If students have already paid us, why should we care if they are engaged ? • Altruism – the long view? • Pragmatism – they can always leave taking their tuition money with them • Student feedback and tenure decisions? • Word gets out! – social networks

3. Agenda • How do students engage with distance learning? • How do I know if students are engaged? • Does instructor moderation affect student engagement? • Does an entertaining instructor affect student engagement? • So what?

4. Research Study • Analyzed course interactions via discussion board on Blackboard learning system. • 12 online MS courses (info. systems./info. Science) • 313 Students, 11,497 messages • Posts to discussion board + small group discussions • Analyzed • Thread depth, thread length, participants • Cognitive content of message • Interactive intent of message • Patterns of message sequences • Examined student outcomes related to interaction • Pre and Post questionnaires • Demographics and Attitudinal data

5. Three modes of learning engagement • Individual Participation • Course Involvement • Iterative Social Engagement • Fluid: students can move between modes reacting to drivers

6. Mode 1: Individual Participation • The semi-transparent participant • Interacts with materials • Internalizes knowledge • Contractual Obligation postings • Broadcast messages • Superficial learning • Hermit

7. Mode 2: Course Involvement • Demonstrates (some) genuine interest • Interacts with peers (after a fashion) • Translates community knowledge • Relates posts to own experience or knowledge • Internalizes community knowledge • Ego-centric approach • Small group or clique interactions

8. Mode 3: Social Engagement • Motivated for interactive learning • Committed to greater group learning • Interacts freely with peers • Looped learning cycles • Iterative internalizations/externalizations • Social construction of knowledge

9. How can we tell what is going on?

10. Mode 1: Course Participation How Do We Know A Project Is On Track? Instructor 10/21/07 2:36 PM RE:How Do We Know A Project Is On Track? S18 10/25/07 12:09 PM RE:How Do We Know A Project Is On Track? S10 10/25/07 6:29 PM RE:How Do We Know A Project Is On Track? S21 10/25/07 8:30 PM RE:How Do We Know A Project Is On Track? S17 10/26/07 7:38 PM RE:How Do We Know A Project Is On Track? S22 10/28/07 6:19 PM RE:How Do We Know A Project Is On Track? S2 10/28/07 7:04 PM RE:How Do We Know A Project Is On Track? S4 10/28/07 10:24 PM RE:How Do We Know A Project Is On Track? S10 10/28/07 10:26 PM RE:How Do We Know A Project Is On Track? S7 10/28/07 10:46 PM RE:How Do We Know A Project Is On Track? S8 10/29/07 12:59 AM RE:How Do We Know A Project Is On Track? S11 12/14/07 11:34 AM

11. Mode 1: Course Participation

12. Mode 2: Domain Involvement • Unanswered questions Instructor 10/5/07 3:23 PM • RE:Unanswered questions S1 10/5/07 6:59 PM • RE:Unanswered questions S17 10/6/07 3:41 AM • RE:RE:Unanswered questions S19 10/8/07 12:33 AM • RE:RE:RE:Unanswered questions S20 10/8/07 10:52 AM • RE:Unanswered questions S13 10/6/07 10:52 AM • RE:RE:Unanswered questions S18 10/7/07 4:14 PM • RE:RE:RE:Unanswered questions S6 10/9/07 9:45 PM • RE:Unanswered questions S12 10/6/07 11:04 AM • RE:RE:Unanswered questions S20 10/7/07 10:34 AM • RE:Unanswered questions S9 10/7/07 6:49 AM • RE:Unanswered questions S21 10/7/07 4:36 PM • RE:Unanswered questions S10 10/7/07 5:31 PM • RE:Unanswered questions S4 10/7/07 10:59 PM • RE:Unanswered questions S19 10/8/07 12:07 AM • RE:Unanswered questions S8 10/8/07 2:21 PM • BOK as a communications/marketing tool S12 10/9/07 1:18 PM • RE:Unanswered questions S8 10/8/07 2:48 PM • RE:Unanswered questions S14 10/12/07 2:33 PM • RE:RE:Unanswered questions S12 10/12/07 3:22 PM • RE:Unanswered questions S15 10/12/07 3:26

13. Mode 2: Domain Involvement

14. Mode 3: Social Engagement

15. Mode 3: Social Engagement

16. What makes students engage with a distance-learning course?

17. Question Frequency vs. response

18. Question Design I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding] Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives?  Is this a "real" model that could be used on "real" projects? [Fast or Slow] I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]

19. And the Winner is I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project] I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding] Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives?  Is this a "real" model that could be used on "real" projects? [Fast or Slow]

20. Good I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work.[Cooking up a new project] • 150 posts Several sub-threads extremely deep (7 or 8 levels) Critique, feedback, support and facilitation Well-placed faculty moderation, nudges rather than cattle prods • Well-bounded but open-ended: students define problem • Deliberately pitching as cooperative task • Concrete task • Students negotiate the task meaning collaboratively

21. Average I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding] • 85 posts • Moderate sub-thread depth (mostly 3 or 4 levels) • 31% were messages from Instructor to students • 20% were messages from students to instructor • Well-placed faculty moderation, focus on challenging assumptions. • Reasonably open-ended problem • Far less cooperative inter-student activity • Not pitched as a cooperative activity • Students not answering a common question, but question is defined

22. Bad Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives?  Is this a "real" model that could be used on "real" projects? [Fast or Slow] • 46 posts • Limited sub-thread depth - mostly 2 (question then single response) • 45% were messages from Instructor to students • 37% were messages from students to instructor • 18% were student-student messages • Faculty intervention much more critical (didactic) • Five questions in one: 1 was open-ended 4 bounded • Very little cooperative inter-student activity • Not pitched as a cooperative activity

23. Good questions tended to be • Early rather than later • First question in the week • Early weeks better than later weeks • Open (scope), but bounded (problem structure) • Permitted students to call upon their personal experience with IT or organizations • Permitted many ways to approach the issues • Negotiated rather than defined • Permitted collaborative interpretation • Allowed students to contribute by defining their own take on the question. • Relevance to students helps – war stories, company policies and approaches

24. Bad questions tended to be • Following a highly-interactive question • Questions set in later weeks were much less interactive and constructive across the classes than questions set in earlier weeks. • Cognitively complex • Containing multiple parts that needed to be considered in turn, or • Overly abstract, so students could not draw on their personal experience. • Socially isolating • Fewer opportunities for interpretation and collaboration in answering the question.

25. How does course scaffolding affect student engagement?

26. Scaffolding • Task requires an extension of prior abilities • Incremental approach • Solid foundation for task • Materials • Discussion • Peer Support • Strong identification with domain

27. Poor Scaffolding You've been asked to read the ALA Code of Ethics plus two other codes of ethics of your choice.  What did you learn from this process?  Did any common themes or concerns tend to emerge?  What did you relate to in the ALA Code of Ethics?  Were there things that seemed problematic, or that you disagreed with? Codes of Ethics Instructor 1/28/08 3:15 PM RE: Codes of Ethics S5 1/31/08 7:00 AM RE: Codes of Ethics S15 1/31/08 5:49 PM RE: Codes of Ethics S13 1/31/08 9:17 PM RE: Codes of Ethics S14 2/1/08 12:05 AM RE: Codes of Ethics S16 2/1/08 12:54 PM RE: Codes of Ethics S9 2/1/08 1:08 PM RE: Codes of Ethics S17 2/3/08 1:20 PM RE: Codes of Ethics S11 2/2/08 3:26 PM RE: Codes of Ethics S18 2/3/08 4:23 PM RE: Codes of Ethics S19 2/4/08 5:33 PM RE: Codes of Ethics S6 2/5/08 6:24 PM RE: Codes of Ethics S1 2/5/08 11:03 PM

28. Better Scaffolding Can ethical behavior really be codified by a professional organization? Can ethical behavior be enforced? How?

29. Scaffolded • A description of ethical models • Worksheet for ethical decision making • Actions and consequences • Responsibilities and obligations • A theoretical and pragmatic platform from which discourse could be built • Three sparse pages of bullet-points

30. Unscaffolded • A long list of codes of ethics urls • Three abstract ethics articles • A body of solid material but which did not directly relate to the posted question or give a framework for answering the question

31. How does instructor moderation affect student engagement?

32. Moderated or leave-alone ? • Two sections of an IS course delivered at the same time – same basic syllabus • ~Same number of students (23/24) • Selected six “identical” questions on each section • Different Instructor approach • Heavy moderation vs. lightweight moderation

33. Questions and Approach (messages)

35. Instructor interaction and student posts

36. Are students more engaged when the Professor is entertaining?

37. Deconstructing the Entertaining Professor • Highly knowledgeable industry professional • Very Popular Instructor • High level of interaction with students • Regular internet chat presence • Projects personality into discussion • High percentage of social interactions • Voluntarily discusses hobbies, weather, music, Disneyworld, cooking, children, Dickens, vintage cars, pets, gardening, insects, Star Wars, birds, Nintendo, Scrabble, foreign films, beer …. • Injects lots of jokes

38. Student participation

39. Never mind the width, feel the quality • Threads show evaluation and analysis • Some hypothesis formation • Fairly advanced cognitive activity • Some student-student interaction • Pretty successful overall • High student satisfaction • Grades (5 B and 18 A) But,much chaff among the wheat • 5% Un-focused Anecdotes • 32% fluff posts: “LOL,” “Awesome [dead rock star] story!” “OMG,” “Pictures of gardening implements,” “Lawyer Jokes” • 50% Substantive Discourse advancing posts • 33% Student-Instructor messages

40. Introducing Professor Serious • Same course – Same Syllabus • Highly knowledgeable industry professional • Very skilled Instructor • Low level of direct interaction with students • Strong Topic-Focus • Little social interaction

41. Student participation

42. Quality ? • More collaborativediscourse • Messages longer and more detailed • Fairly advanced cognitive activity • Much stronger student-student interaction • Stronger awareness of value of peer interactions • Moderate student satisfaction • Grades (5 B and 17 A) So what? • High task-Focus • 2.5% fluff posts • 80% substantive discourse-advancing posts • 15% student-Instructor messages

43. The tale of the tape • Instructor participation of Prof. Entertaining inflates thread depth slightly • Student-student posts more productive than student-instructor posts • Overall productive activity was about the same for the two Professors • Student satisfaction slightly lower for Prof. Serious than Prof. Entertaining • But • Stronger thought leaders for Prof. Serious • Explicit kudos for peers in Prof. Serious

44. Conclusions • Question design can be crucial to engagement • Discussion needs to be framed as collaborative not competitive • Relevance to students helps – war stories, company policies and approaches • Course scaffolding aids engagement • Focus and framing of questions • Must support task in concrete manner

45. Conclusions • Successful course moderation hinges on quality not quantity or frequency • Knowing when to intervene • Does not mean “being absent” • Being entertaining is not essential for success • Excessive interaction is a lot of work • May shift focus from peers to instructor • No payoff in frequency of questions

46. Final Words: Value of online discussion this was so helpful because often I was struggling with the same thing so I could learn from their errors and gain new information from the answers to their questions I was moved to comment on how refreshing the lack of competition in the Communications for the online classes seemed to me. It was a discussion and a sharing of experiences No question that the on line discussion was critical to getting me through the class. There were mostly questions about how to.. I've never done this before. I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant. Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback

47. Worrying or not ? I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant. Stopped posting after week 2 Little opportunity to get drawn in What do you do if someone will not contribute ? Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback Concerned about how posts were interpreted Most posts were responded to Most posts contained positive feedback but no added content Many posts Social rather than task-oriented Rarely posted thread starters

48. Related References • Waters, J. 'Social Network Behavior, Thought-Leaders and Knowledge Building In An Online Learning Community', Proceedings of  Hawaii Intl. Conference on System Sciences (HICSS-41), Knowledge Management Track, Jan. 2008. • Gasson, S. and Waters, J. “How (not) to construct ALN course questions that encourage student participation in peer collaboration and knowledge construction,” 40th Hawaii International Conference on System Sciences, Hawaii, January 2007. • Waters, J., and Gasson, S. "Social Engagement in an Online Community of Inquiry," 27th International Conference on Information Systems (ICIS), Milwaukee WI, 2006. • Waters, J. “Determinants of Engagement in an Online Community of Inquiry,” The 12th Sloan-C International Conference on Online Learning, November 2006, http://www.sloanconsortium.org/conference/proceedings/2006/ppt/1162852287092.pot • Waters, J., and Gasson, S. "Strategies Employed By Participants In Virtual Learning Communities," Hawaii Intl. Conference on System Sciences (HICSS-38), Collaboration Systems and Technology track, IEEE Software Society, Hawaii, January 2005, p. 3b. • A full list of publications, with full copies of articles, is available at http://www.pages.drexel.edu/~jw65/publications.htm