Identifying the Non-Functional Issues Affecting Student Participation In Discussion Boards

1. Identifying the Non-Functional Issues Affecting Student Participation In Discussion Boards Ismail Bhula Faculty of Engineering and Computing, Coventry University bhulai@coventry.ac.uk Peter Every (presenting) Faculty of Engineering and Computing, Coventry University peter.every@coventry.ac.uk

2. Impetus for research The use of two different forum tools over two years on the same module revealed a large discrepancy in student engagement and forum usage. Context 2003 – ongoing UK University Students co-located (not distance learning) Modules utilise lecture / seminar format with online tools as support Module populations 30 - 150

3. Engagement with forum discussions – comparing WebCT with WebWiz application Post quantity Web Wiz Forum @ 100 students: 4367 Posts in 842 Threads WebCT @ 80 Students 513 posts in 325 Threads Thread Activity Web Wiz Forum@ 100 students: 50 threads with 7 replies or over 6 threads with 25 or more replies WebCT @ 80 Students 2 threads with 7 replies Over 200 posts with no replies (!)

4. Research goals Evaluate the usability of the WebCT discussion forum based on a number of evaluation techniques. Determine the effect of the usability of the forums on the levels of participation and interaction. Produce a basic set of usability standards for the selection and utilisation of such tools in virtual learning environments.

5. Usability • ISO 9241 (part 11) defines usability as: • “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” • Main evaluation methods • Task based usability tests • Cognitive walkthrough • Heuristic inspection

6. WebCT: Task based usability tests Observation/recording of students carrying out tasks with WebCT – recording completion rates (effectiveness) and time (efficiency). User comments during or after the test were analysed to gauge satisfaction. Task types for forums: Make a post Reply to a post Search for a topic Find the most recent post Data can be used quantitatively (task completion / time taken) Or experientially –gaining immediate feedback of user issues.

8. Findings: Task based usability tests failed to find any significant problems with task completion within a reasonable time – users reported no significant difficulties in completing the tasks. Problems with task centred usability test approach: Not longitudinal - privileges novice users in first time interaction with the application. Privileges productivity over pleasure Ignores cultural context of use

9. Phase 2 – content analysis 2004 – 2006 method: The collection of student responses to open-ended questions on the efficacy of the ‘WebCT’ tool in particular, and online learning tools in general. 300 third-year undergraduate students using the WebCT discussion board over the course of three years. The study was conducted in a module on human-computer interaction. The issues identified are ‘reframed’ as ‘principles for improving engagement’ –to be validated and discussed with students in future modules.

10. Attend to usability - ensure each user has the potential to customise their experience of the forum. “WebCT can become very annoying to use. Even now I feel like not bothering to post anything just because WebCT discussion forums look ugly, they’re difficult to use and are just plain annoying!” VLE forums that ignore internet forum conventions can be viewed negatively by students, The most popular forum applications currently are PHPBB, vbulletin, UBB. These allow considerable customisation. There are no reasons why potential usability problems such as the order in which threads or messages are listed, or the style and colours of the board, or the ability to reply to more than one post at a time, should not be fully customisable to each individual’s preferences. Attending to usability should mean that manipulating the software is entirely incidental and ‘transparent’ as far as the learner is concerned.

11. Forum labelling: WebCT – no space to be descriptive about the function of each forum.

12. Webwiz – space for a descriptive sentence.

13. Notice also the system feedback about recent activity on each forum

14. Ensure each user has the potential to personalise his or her outward-facing ‘appearance’ on the forum “All the forums I know of and like enough to use are heavily community based. Without adding features to allow people to express who they are rather than simply being a rather anonymous automatically assigned user name, WebCT will never be a forum used by the majority of students.” “its like we are trying to create an online community but on … Webct you can’t really create this virtual community because it is hard to distinguish individuals.” Tools available to shape online ‘appearance’ include choice of user names, avatars, sig-files, links to blogs, myspace, facebook, MSN/chat numbers, online/offline status, tone/style of posts. All of these constitute a considerable investment of time and effort in the creation of an ‘online persona’. Question: Is there a correlation between investment in this persona and a willingness to engage in forum discussion?

16. Ensure adequate training for lecturers in discussion board facilitation and engendering ‘swift trust’. “The fact that we know the lecturers are sitting there reading our posts, but never replying is extremely disconcerting – and I believe this is a fundamental flaw.” “…part of the reason for not using them [boards] is a fear of coming across as an idiot.” Forums on the Internet may take many years to evolve an ‘ecology’ of community norms. Academic forums, however, are often ‘forced’. Short duration modules (< 1 year) require strategies to build trust and community norms quickly. Meyerson (1996) talks of “swift trust” - methods for groups of people who are forced to work together for a predefined short period of time around a clear purpose to interact efficiently and effectively. Positive feedback, follow on questions, recognition of student effort are the tools of swift trust in this context.

17. Ensure every discussion board has a designated forum for social interaction. “because the discussion board is purely for academic work, it stops people from being able to express personal things about themselves and share interests that might help to generate more of a sense of community” Several students in our study highlighted that, in their opinion, the tone of discussion was ‘too academic’ to engage a community of students who have interests beyond the university.

18. Where learners are co-located, promote the value of the discussion board as a persistent research tool. “we have each others mobile numbers and we chat on MSN do [yo]u really think we’re gonna use the discussion forum to show we[‘ve] been working together?” Tutor insistence on ‘seeing participation’ through the discussion board can be perceived as expecting students to ‘jump through hoops’ in a co-located context. Promoting the discussion forum as a ‘library’ or ‘repository for research links’ may be a more successful approach – providing students with an appreciation of the advantages of persistent, well-referenced and considered discussion.

19. Ensure privacy when groups are in the process of producing assessed work. “…the forums for each group… are not private… some of [x] group i have read have already started accusing each other about copying in that module.” This provision links into the concept of ‘lurking’ or ‘quiet participation’, which is very quickly apparent as a factor discouraging learners from engaging with online learning tools. When contribution to forums constitute part of the assessment for a course, the issue of lurking is often associated with plagiarism.

20. Prepare and invoke an online learning charter “Many of my emails to group members are quite chatting and informal – would this be okay to post on webCT? I don’t know, is the simple answer! Therefore, I don’t!” “i dont like [posting] on the webCT discussion forums because i find people ask the same questions even when they have been answered.” Web savvy students still need guidance to learn the norms and conventions of interaction in the academic context. Over time, inconsistencies in the ‘level of service’ provided by different tutors can lead to uncertainty. The purpose of the charter would be to specify, unequivocally, the uses for which the discussion board may or may not be employed, and the rights and obligations of both learners and lecturers towards each other.

21. Ensure learners know how much regular participation is essential, and are able to enjoy it just as much as their regular, non-assessed contributions. From our data it appears that many students are confused by the relevance of the discussion board to their overall learning experience and sometimes feel that participation without reward is pointless. “…the list of tasks never requires students to take part in a discussion forum as part of the assignment, so many think it is not part of the assignment, and therefore will not get them any marks, so why bother? why waste time over something which will not benefit me?”