Analysis of Student Learning in Global Change

1. Analysis of Student Learning in Global Change Eugene S. Takle1, Heather Moser1, and Elsebeth K. Sorensen2 1Iowa State University, Ames, IA 2Alborg University, Denmark gstakle@iastate.edu

2. Outline • Context for Dialog • Theoretical Background • Dialog Implementation Strategies • Diagnosis by “Theory of Language Games” • Evaluation of student learning

3. Context for Dialog • Global Change course • 3 credit , senior-level, on-campus or remote • 42 GC topics (1 each MWF), 3 5-week blocks • Online with dialog since 1995 • Currently enrolls 50 students from 23 disciplines • Course publicly available: http://www.meteor.iastate.edu/gccourse

4. Characteristics of “Structure-less” Dialog • Superficial remarks • Off-the-cuff comments • Personal agendas • Religious fervor • Cutsey comments • Occasional glimmer of critical thinking

5. The Challenge: How do we stimulate learning in online dialog?

6. Theoretical Framework:Learning in Virtual Environments • Wenger (1998) : true collaborative learning occurs through “negotiation of meaning” • Sorensen and Takle (2002) • mutual exploration of issues • mutual examination of arguments, agreements, and disagreements • mutual questioning of positions • dynamic interaction • weaving of ideas • convergence of perspectives

7. “Genuine Collaboration” “Genuine collaboration” (Salomon, 1995) is a condition of “genuine interdependence” between individuals that calls for • sharing knowledge/information • adopting complementary roles • a “pooling together of minds”

8. So how do we structure our virtual environment for dialog to ensure that learning takes place?

9. I. Active Participation Although some learning occurs in passive observation, active participation is essential for true collaborative learning Put requirements on minimum number of postings: 6 per block

10. Table 1. A = # students, B = # comments per student • per learning unit, C = #comments per student per block. • ____________________________________________ • Year A B C • ____________________________________________ • 1995 32 0.069 1.0 • 1996 31 0.106 1.6 • 1997 32 0.350 5.3 • 1998 33 0.390 5.9 • 1999 26 0.320 4.8 • 2000 45 0.480 7.2 • 2001 44 0.382 5.7 • 2002 46 0.528 7.6 • ____________________________________________

11. Result: Increased volume of comments, but content of comments was shallow and did not give evidence of critical thinking

12. II. Quality • Define critical thinking skills (CTS), give examples, and demand that student dialog must demonstrate CTS • Implement an online threaded dialog structure that requires student to state, in advance of posting, the CTS being used • Ask student in end-of-block self assessment to defend their use of CTS

13. Elements of the Virtual Portfolio Research Paper Research Paper Calc Calc Simulation Simulation Quiz Quiz Dialog Dialog Student Products Time Disc. Sum. Disc. Sum. Web Search Web Search Hypothesis Hypothesis Hypoth. Exam Exam Animation Animation Block 1 Block 2

14. Elements of the Virtual Portfolio Evaluate Product Student Products Time Block 1 Block 2

15. Elements of the Virtual Portfolio Student Products Time Evaluate Process Block 1 Block 2

16. Result: By a subjective measure (0-10), quality went from 4.4 (1995) to 3.2 (1997) to 5.3 (2000) It worked, but online entries became a collection of monologs

17. III. Action-Reaction Require students to respond to 3 other students per block and to elicit comments from 3 other students per block. Require one “social” or “community-building” comment per block.

18. Table 2. Comparison of comment characteristics when requirements for social comments were added. ___________________________________________ 2001 2002 Chg(%) ___________________________________________ Number of students 44 46 +4 Required # of posts/st. 15 18 +20 Total number of posts 760 1,045 +38 Actual # posts/student 17.3 22.7 +31 % actual to required 115 126 +10 Length of post (words) 140 155 +11 ______________________________________ True collaborative learning?

19. IV. Collaborative Learning Through Substantive Interaction • Close, but not there yet • Not sufficient evidence we have true “negotiation of meanings” (Wenger) • Or “true interdependence” (Salomon) • Next Step: • Analyze dialog by use of the theory of “language games”

20. Language Game Theory }}}} Closes language game opened by

21. Language Game Theory • A language game must have, as a minimum, the following elements: • Initiation (open a new game) • Expectation (offers that new understanding may emerge if response is provided) • Clarify a misconception, resolve a conflict • Might result in initiation of a subgame (requesting more information, clarifying an assumption, expanding the game beyond the initiation concept) • Closure • All expectations are met, conflicts resolved

22. Assumption and Hypothesis Assumption: The number of completed Language Games developed within a dialog in some way (yet to be determined) will be proportional to the amount of collaborative learning that takes place. Hypothesis The amount of collaborative learning that takes place will increased by use of the knowledge-building process.

23. Experimental Procedure and Analysis • A random number generator was used to generate numbers that directed which comments would be selected • 1,350 comments from 1996-2004 were selected for consideration • Of these, 689 comments described a situation suitable for analysis • Of these 689 comments, 356 (52%) were considered to be candidates for initiating a language game • Of the 356 potential games, 242 (68%) were actually created • Of the 242 games created, 217 (90%) were closed • Of those comments (689-356=333) for which no game was created, only 124 (37%) received follow-up comments

24. Interpretation % of candidate comments for Year which a game was completed 1996 17.2 15.7 24.7 38.1 36.5 48.8 35.7 30.1 31.6 Mean = 19% Mean = 37%

25. Conclusions • Online dialog is a powerful option for promoting reflective thinking and use of critical thinking skills • Students will respond if we tell them what we want them to do • Developing online learning environments based on sound pedagogical principles improves likelihood of success • Implementation of the knowledge-building process enhanced student learning http://www.meteor.iastate.edu/gccourse gstakle@iastate.edu