1 / 6

Peer review with Aropa in Computing Science

Peer review with Aropa in Computing Science. Dr Simon Rogers Simon.Rogers@glasgow.ac.uk. The course: PSI. Professional Skills and Issues: L4 and Msc students (>100) Compulsory: Required for BCS accreditation Non-technical: Ethics, IPR, Data protection etc Not the most popular L4 course……

phuc
Download Presentation

Peer review with Aropa in Computing Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Peer review with Aropa in Computing Science Dr Simon Rogers Simon.Rogers@glasgow.ac.uk

  2. The course: PSI • Professional Skills and Issues: • L4 and Msc students (>100) • Compulsory: Required for BCS accreditation • Non-technical: Ethics, IPR, Data protection etc • Not the most popular L4 course…… • Assignment: • 1,500 word essay. • 2 stage peer review process. • Final mark (from academic) weighted by `engagement factor’

  3. Aropä • Peer review management software developed by John Hamer (Auckland & SCS) and Helen Purchase (SCS). • helen.purchase@glasgow.ac.uk • http://www.dcs.gla.ac.uk/~hcp/aropa • Handles everything: • Submission (word/pdf, txt) • Assignment to reviews • Review rubric (design your own) • Returning reviews to reviewers • Reviewing of reviews • Very easy to use. Local support available. • Willing to expand system to meet new requirements.

  4. Peer review process • Trial exercise • 500 word essay • 2 reviews • Not assessed (almost 100% response) • Assessed exercise • Students submit draft essay on one of 3 topics (0.7). • Assigned 3 reviewers from same topic. • Reviewers assess work (numerical ratings and text justification) (0.05 x 3). • Reviews returned to authors who can use them to improve essay. • Each student also reviews 3 other reviews (0.05 x 3). • Final marking done by academics, weighted by factors (X)

  5. Pros & Cons • Pros: • Aropa very easy to use – assigning reviews, handling deadlines, monitoring submissions etc. • High level of engagement (almost all students completed everything). • Massive quantity of feedback (not always high quality) • Cons: • Between cohort tension (L4 v MsC). • `Threat’ of plagiarism (by reviewers). • Lack of peer review experience – poor quality of feedback (first use of peer review for 85% of class). • Anonymity (.doc files).

  6. Other feedback • Only 20% of students would have been happy to be formally assessed by peers. • Of those who thought the exercise improved their essay (42%!), 71% felt that the most helpful bit was receiving feedback.

More Related