The Asynchronous Discussion Board as an Assessment Tool: A Critical Appraisal

1. 0 The Asynchronous Discussion Board as an Assessment Tool: A Critical Appraisal 8th CAA Conference Loughborough University 6-7 July 2004 Dr Jeremy Williams Universitas 21 Global

2. Overview 0 • The context • Learning through discussion • Criteria for assessing discussion • The hazards of assessing discussion • Authentic discussion board assignments • Preliminary findings

3. 0 1. The context

4. Universitas 21 Global 0

5. The global nature of U21G • Online graduate business school, headquartered in Singapore • Around 300 students enrolled from more than 20 countries, in many different time zones • Adjunct faculty from Europe, Australasia, Asia and North America

6. Pedagogy • Instructor-led, student centred • Problem-based learning • Extensive use of Harvard Business school cases • A commitment to authentic assessment • Modus operandi = asynchronous discussion => how discussion boards are used  critically important

7. 0 2. Learning through discussion

8. Communities of practice • ‘With the expansion of online learning … tremendous opportunities are becoming available to teachers and learners to foster peer relationships, team skills, collaboration, group problem solving and debate, as well as less formal or structured communications between students and their wider professional communities’ Morgan & O’Reilly (1999, p. 86)

9. Peer learning • ‘In the online classroom, it is the relationships and interactions among people through which knowledge is primarily generated’ Palloff and Pratt (1999, p. 15) • Collaboration between students from far-flung institutions around the globe can considerably enrich the experience and broaden the contextual perspectives of each participant Day (1998)

10. Dialogue and greater learner autonomy • Enhancing dialogue is a way of promoting independence and autonomy in learners, and challenges power relationships in teaching and learning Evans & Nation (1989) • ‘The digital learning environment will probably be the most efficacious “enabler” of independent and self-determined learning’ Peters (2000, p. 16)

11. Laurillard’s Conversational Framework • ‘… studies show that a collaborative discussion environment is highly valued by students …’ • ‘Students have access to an expert whom they can question to clarify the expert’s description • Students can articulate and re-articulate their descriptions of the topic in response to others’ ideas and comments • Students can reflect on the discussion to clarify their own understanding’ Laurillard (2002, p. 148)

12. Encouraging online discussion • Grades – the currency that students deal in Swan et al (2000) • Needs to be well-integrated into the subject, and will also be of greater purpose to students if it is assessed Day (1998) cited in Morgan & O’Reilly (1999, p. 86)

13. 0 3. Criteria for assessing discussion

14. MacKinnon (2000) • Four ‘categories of interaction’: • Challenging a point of view • Forwarding a new perspective • Relating the theory to one’s experience • Offering support for a position based on the literature.

15. Sabin et al (2000) • P is for participation • A is for additional commentary • C is for constructive criticism • E is for encouraging

16. Meyer (2004) • Proffers 4 different analytical frameworks: • Two were developmental models: • King and Kitchener’s Reflective Judgment Model • Perry’s model of intellectual and ethical development • Two captured levels of thinking: • Garrison’s four-stage critical-thinking model • Bloom’s taxonomy of educational objectives

17. 0 4. The hazards of assessing discussion

18. “You can take a horse to water, but …” • ‘Collaborative learning is undeniably important, and the communicative media are powerful enablers that match what is needed for discussion and collaboration, but to what extent do they succeed in enabling learning?’ • ‘… the properties of a medium do not determine the quality of learning that takes place…’ Diana Laurillard (2002) Rethinking University Teaching, p. 148

19. Assessing discussion: Counter-intuitive? • Harlen and Deakin Crick (2003) – the motivation for learning can be discouraged unwittingly by assessment and testing practices • Early experience at U21G suggests that when discussion contributions are assessed, discussion becomes a little contrived • A case of Goodhart’s Law applied to assessment?

20. 0 5. Authentic discussion board assignments

21. Discussion boards for discussion • Discrete small-scale problems abandoned in favour of discussion board assignments integrated with case study assignments. • Small assessment weighting for participation remains • The main incentive to contribute – the grounding it will provide for the submission of a case study assignment with a larger assessment weighting.

22. Assessment ‘in disguise’ • Trials now taking place: • All discussion board activity is evaluated with an overall mark awarded • Students nominate selected discussion board contributions for assessment (e.g. their 5 best) • Discussion boards will be considered by the instructor in validating peer assessments of student performance

23. 0 6. Preliminary findings

24. Student feedback This prompted actual threads of discussion rather than what I've found in other modules where one makes rather artificial comments on other student's postings as the class attempts to get marks for making exactly the same points in slightly different words.

25. Student feedback I thought the discussion boards were useful in preparing for the assignments because they focused on the content of those assignments so you were able to get tips from your fellow students and the tutor. In other modules, discussion boards were separate to the written assignment and so represented an additional rather than complementary task.

26. Summary • Appropriately structured, discussion boards can provide a useful vehicle for learner assessment • The act of assessing need not have a stultifying effect on the quality of debate and the depth of learning

27. Future work • A comparative analysis of the qualitative substance of discussion board postings before and after the change in discussion board format

28. Questions?