Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion

1. Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion To appear in the International Journal of Computer-Supported Collaborative Learning I appreciate the research assistance of Choi Yik Ting

2. Motivation for the Study Online, asynchronous forums • Can participate anywhere – no geographic limits • Can share ideas at any time – more time to think • But often disconnected, only lists of isolated ideas Guzdial & Turns, 2000; Herring, 1999; Thomas, 2002 Summaries • Connect previous ideas and develop them • But often occur at end of discussion & Do not benefit other members De Wever et al., 2007; Schellens et al. 2005; 2007 Encourage summaries in the middle of discussions?

3. Knowledge Construction (KC) FrameworkGunawardena et al.’s (1997) Five-Phase Model

4. Research Context for the Study Emerging Themes in Collaborative Learning Research (e.g. Arvaja, 2007; Stahl, 2004; Strijbos et al., 2004) (e.g. Chiu & Khoo, 2005; Kapur, 2001; Reimann, 2009) (e.g. Cress, 2008; Suthers & Teplovs, 2011)

5. Knowledge Construction Phase Possible KC Patterns Post Number

6. Research Questions • What patterns characterize knowledge construction processes during an online discussion? • What characterizes pivotal posts that divide a discussion into distinct segments? Summaries? • Which characteristics of a post influence the knowledge construction phase of the next post?

7. Roles Synthesizer (+) … Pivotal Post Functions Summary (+) … Time context Week Individual Control variables Gender Age … Post Control variables # of words Time of post …

8. Roles Synthesizer (+) … Knowledge Construction Functions Summary (+) … Time context Week Segment Individual Control variables Gender Age … Post Control variables # of words Time of post …

9. Methods

11. Content Analysis Unit of analysis:  Post / Note / Message  Objectively identified unit that its author defines Rourke, Anderson, Garrison, & Archer, 2001 Inter-rater reliability  Krippendorf’s  (range: -1 … 1; desired: > .67)

12. Statistical Discourse Analysis 4 types of Analytical Difficulties • Time • Outcomes • Explanatory variables • Dataset - No missing data

13. Statistical Discourse Analysis 1 2 4 6 3 5 7 8 • Strategies • Breakpoint analysis + Model • Multilevel analysis (MLn, HLM) • Test with I2 index of Q-statistics • Model with lag outcomes, KC (-1) • Store path: Identify prior turn • Difficultiesregarding Time • Segments differ (S2 S4) • Serial correlation (p8→ p9) •  Branches of notes

15. Statistical Discourse Analysis 1 2 4 6 3 5 7 8 • Strategies • Breakpoint analysis + Model • Multilevel analysis (MLn, HLM) • Test with I2 index of Q-statistics • Model with lag outcomes, KC (-1) • Store path: Identify prior turn • Vector Auto-Regression • Lag explanatory variables • e.g., Valid (-1), Girl (-1) • Valid (-2) • Difficultiesregarding Time • Segments differ (S2 S4) • Serial correlation (p8→ p9) Multiple topics •  Branches of notes (→→ )

16. Statistical Discourse Analysis Outcome Difficulties Ordered outcome (KC 1-5)  Infrequent outcomes (00010) Strategies Ordered Logit / Probit Logit bias estimator

17. Statistical Discourse Analysis • Explanatory model Difficulties • People, Groups & Topics differ • Mediation effects (X→M→Y) • False positives (+ + + +) • Strategies • Multilevel analysis •  Multilevel mediation tests • 2-stage linear step-up procedure 

18. Results – KC Phases

19. Results: Summaries as Pivotal Posts Each discussion averaged 1 pivotal post (2 time periods)

20. Results - KC Patterns Knowledge Construction Phase Segments Skipped KC phases No Regressive Segments No Regressive Segments Pivotal Posts → Distinct Segments Post Number

21. Predicting Pivotal Posts Role Current Post Synthesizer Extensive Summary Pivotal Post Wrapper

22. Predict Knowledge Construction After 1st pivotal post Time2 posts agoPrevious postRoleCurrent post Knowledge Construction Respond (-2) after 1st pivotal post New Idea (-1) after 1st pivotal post Synthesizer Summary Wrapper

23. Summary of Results KC pattern  KC phase 1 KC phase 3 or 5 (Share) (Negotiate Meaning or Agree/Apply)  Few KC phases 2 or 4 (Dissonance, Testing) Pivotal post  Extensive Summary often  By Synthesizer or Wrapper usually Extensive Summary  Showed higher KC  Elevated KC of subsequent posts

24. Implications Teacher / Designer  Assign Synthesizer Role - Increase midway summaries and elevate KC - Simple, effective intervention  Productive online discussions do not require all phases Researcher  Empirically test Gunawardena et al’s KC model  New method for analyzing online discussion - Statistically identifies pivotal posts & segments - Test hypotheses about relationships among posts - Examine variables at multiple levels - Examine differences over Time

25. Further Questions • With many choices of dimensions for the breakpoints, which one(s) should we use? • What do identification of same vs. different breakpoints across different dimensions tell us? • How can we do meta-analyses of multiple data sets with somewhat different codes? • Which analyses (qualitative and/or quantitative) might be fruitful on the same data set?