Peer Assessment

1. Peer Assessment COMP3120 – Managing Software Development

2. Contents • Value • Students • Academics • Purpose • How is peer assessment completed? • WebPA-x algorithm

3. Value to Student • Encourages reflection – learn to make judgements about the quality of one’s own and one’s peers’ work • Enables students to recognise individual contributions – quality, quantity, relevance • Helps to develop team work • Helps improve individual performance • http://cs.anu.edu.au/courses/COMP3120/pa.html

4. Value to Academics • Assess how successfully the group worked together and how well they managed their work • Assess the group product • Mandatory although it carries no marks and there are penalties for late submission • http://cs.anu.edu.au/courses/COMP3120/pa.html

5. Purpose • To assess the contribution made by each member of a team • Allows the identification of team members who have been active and collaborative and those who have not. • Used to calculate individual marks from the team mark awarded for the artefact

6. How is Peer Assessment completed? • WebPA-x • Base software developed by Loughborough University in the UK and released as free and open-source • Used by hundreds of universities world-wide • Software was amended by software engineering group project student team • Used by COMP3100/3500/4500 since 2010

7. How is Peer Assessment completed? • Evaluate team members and yourself against a set of categories • which when taken together should give a good overall indication of performance • Provide an overall, holistic rating of your and your peers' performance, including a written description which provides an overview of the behaviour, activities, events etc that explain your rating.

8. How is Peer Assessment completed? • Does not ask students to make a commentary of other students’ academic abilities • Asked to rate performance against a set of professional behaviour / performance categories • Sometime called Key Performance Indicators (KPI)

9. How is Peer Assessment completed? • Evaluation categories • Leadership • Professionalism • Respect • Excellence • Collaboration • Accuracy and attention to detail • Timeliness • Enthusiasm

10. How is Peer Assessment completed? • Holistic rating of performance, including a written description • Rating categories: • Excellent • Very Good • Expected • Ordinary • Unsatisfactory • Margin • Deficient • Superficial • No Show

11. WebPA-x algorithm • Each student in a team is given a pie which they divide up among their peers according to their assessment of the value of their peers' contributions. • The ratings that a student gives themself is not used when calculating the WebPA-x peer rating.

12. WebPA-x algorithm • Normalise ratings to overcome the issues related to one student awarding a maximum rating of 5 and another student awarding a maximum rating of 3, • Done prior to calculation of the WebPA-x peer rating. • When a team member fails to submit a peer assessment the algorithm acts as if they rated everyone as performing equally.

13. WebPA-x algorithm • Worked example • Collate submitted ratings • Normalise ratings awarded by each student • Calculate WebPA-x rating for each student • Rating against professional criteria worth 50% • Holistic evaluation worth 50% • Calculate peer assessed mark • Team mark multiplied by WebPA-x rating • Individuals rated highly by the team can get more than the team mark