Peer assessment of group work using WebPA

Peer assessment of group work using WebPA Symposium on the Benefits of eLearning TechnologiesUniversity of Manchester, in conjunction with the Higher Education Academy Subject Centres in Engineering, Physical Sciences, and Mathematics, Statistics and Operational Research Wednesday 13 April 2011 Neil Gordon

Introduction – why include group work? • It is perceived as a key skill • QAA expect it • Employers expect it • Accrediting bodies require ““working with others” • It is a useful skill for our students – one which they will need after their studies

Practical Issues • a key problem with group work is assessment. • Students may consider the process unfair if individual levels of contribution are not recognised. • But the tutor rarely has full knowledge of the team interactions to allow realistic allocation of marks. • Gathering data to do this from students is not viable with large cohorts. • WebPA provides a tool to allow this approach – providing a web interface to gather data, and implement algorithms to allow the allocation of individual marks based on a team’s overall mark.

Technology and Group Work • Use of technology to support team activities is standard in industry • In the context of teaching, use of these technologies in education helps develop “real world” skills, alongside its educational value • Many technologies now in common use (especially with Web 2.0) : VLEs, GroupWare and Intranet, Wikis, etc.

Peer (and self) Assessment • Many potential benefits – especially in ensuring that students become aware of marking criteria • It can reduce the marking burden • Different approaches e.g. • group agrees a weighting for each group member • Each student assesses each other student • Groups of students assess other groups • Can be problems with the volume of data gathered, as well as mediation issues

Transparency in marking criteria • By supplying clear marking criteria to students they can understand the assessment process • Particularly important when marking each other • Explaining this helps make students aware of assessment processes;

Example Peer marking criteria • Students asked to assess each other in terms of: • Contribute to meetings/online discussion • Offer constructive input to discussions • Contribution to production of report • Overall contribution to assignment • Marked in the range 0 (no input) to 10 (excellent input)

WebPA: a tool for peer assessment • WebPA provides a tool to automate much of the above with regards to peer (and self) assessment • Allows groups to be set up, and for students to assess their other team members • Uses a simple algorithm to allocate a weighted mark • Provides for students to enter marks in a faceless environment

Issues in peer and self assessment • Tools such as WebPA allow students to mark each other in a safe environment - through a web interface which protects the anonymity of individual students’ marks of their peers from other students; • WebPA also applies a suitable algorithm to apportion contribution based on the student marks; • One major issue when considering the use of peer assessment is the volume of data and its management • Arranging to collect and then use the data becomes a problem with large classes (e.g. >100): WebPA can manage this process in an automated way.

Case Study: Computer Science @ Hull • Regularly use team/group projects • I have used peer assessment across several modules, throughout the range of teaching (level 3 to level 6 and from cohort sizes of 6 up to 180). Nearly 1000 students • In small modules, originally did by paper, where it was fairly easy to manage – but was impractical for large modules

Student Views of group work • Some students would not contribute, and those who were active would end up doing all the work; • A desire to be able to choose their own groups, so they were confident that people would put in the effort; • Concerns about random groups, where members do not contribute fully and the workload is unbalanced; • Concerns about having to rely on others; • Acknowledgment that group work is essential in the software industry; • Concerns that criticising the work of a fellow student may mean that student gives them a low mark in the peer assessment; • Perceptions that it is more fun than working alone, but more complex

Student views of peer assessment prior to it’s use • Asked “do you like the idea of this form of assessment” - This question generated more negative responses, many from those who has not experienced it in practice • concerns were about biased marking • About other students not taking it seriously. • Several students felt that as long as the teacher was involved in the marking process, then it would be acceptable.

Solution: WebPA

Adding Data

Student’s views after module • Majority (70%) prefer to work alone (strongly agree or agree#). BUT • 65% would# use peer and self assessment in other modules given the choice and • 67% like# a mix of group and individual work • 80% comfortable# assessing own performance • 69% comfortable# assessing peers

Positive views of WebPA

More views on WebPA When asked “What changes (if any) to WebPA would you like to see? “: • 80% of students said no change • 9% asked for “Provide textual feedback to explain the scores you gave yourself“ • 1% for a “general feedback to add any other information”

My comments on using WebPA • Issues where a student had dropped out, but the marks had been applied before I updated that – so having to reverse engineer the mark to stop inappropriate marks of 100% • Would be helpful to be able to import groups (I usually manage these in Excel) • But generally – excellent

Conclusions • WebPA can allow the scaling up of group work with peer and self assessment • It assists the teacher/lecturer by reducing admin associated with the activity • It benefits students as it provides a safe environment in which to carry out the assessment • It can encourage self reflection and a focus on assessment criteria • It can be integrated fairly well into a VLE

Peer assessment of group work using WebPA