Promoting Psychology Student Learning through Assessment 27 March 2009

1. Helping psychology undergraduates improve examination writingKathy Harrington, Mercedes Freedman, Savita Bakhshi, Peter O’NeillLondon Metropolitan University Promoting Psychology Student Learning through Assessment 27 March 2009 London Metropolitan University HEA Psychology Network Write Now Centre for Excellence in Teaching & Learning

2. Overview of presentation • Aims and context of study • Pedagogical rationale • Description of the intervention • Method • Findings • Conclusions and implications

3. Study aims • Improve students’ examination writing and performance • In addition, we wanted to… • Build on previous research on use of assessment criteria to improve student learning • Develop and evaluate an intervention based on collaboration between psychology academics and writing specialists (Writing in the Disciplines model) • Create re-useable teaching & learning resources

4. Context of study • 2nd-year cognitive psychology module • Assessment 100% by examination • Traditionally lower than average pass rate • Identified by university as a “killer module” • Plan of response required • Write Now CETL works with discipline-based academics to improve student learning through curriculum and teaching development • Write Now CETL runs university Writing Centre staffed by trained student peer mentors in academic writing • Collaborative, non-directive, supportive • Enable students to take responsibility for own work

5. Pedagogical rationale • Students and tutors often interpret meanings of assessment criteria differently (Harrington et al., 2006; Lea & Street, 1998; Merry et al., 1998) • Providing clear and explicit criteria is a first step in helping students understand what tutors are looking for in written work • However, research has also shown that facilitating students' active engagement with the criteria is necessary if learning and performance are to be demonstrably enhanced (Price et al., 2001) • Structured interventions focussed on understanding and demonstrating assessment criteria have been shown to lead to improvements in student learning and performance (Norton et al., 2005; Rust et al., 2003) • Other research has found that students value being able to talk to peer tutors about their writing, and that psychology students prefer peer tutors from their own discipline (Bakhshi et al., in press)

6. The intervention • 4, hour-long compulsory workshops embedded as part of module teaching across the autumn semester 2008-09 • Immediately following two-hour lectures • Delivered in alternate weeks, with workshops run by lecturers in between (focussed more explicitly on lecture content) • Designed by team of academic writing specialists, psychology lecturer, psychology PhD student • with experience of pedagogical research in area of student writing and assessment • Delivered by academic writing specialists and peer writing mentors studying psychology (3rd-year and PhD students)

7. The intervention (continued) • Exam answers posted in VLE prior to workshops for students to read and give a grade • Set of accompanying materials developed using extracts from authentic examination answers annotated with comments in relation to assessment criteria • Specific focus on cognitive psychology • Use of departmental assessment criteria • In workshops, materials used to facilitate discussion about demonstrating assessment criteria at different levels of performance • Students guided in small groups to adopt role of examiner and apply criteria to whole past examination answers • Final class discussion drew out students’ insights and summarised main points

8. Data collection and analysis • Attendance registers taken at workshops 2, 3 and 4 (not at first workshop) • Questionnaire distributed at last workshop (n=63) • Likert scale: students’ perceptions of helpfulness of workshops in relation to • Examination writing • Meeting assessment criteria • Understanding subject matter of cognitive psychology • Examination grades • Analysis using SPSS to produce descriptive and inferential statistics

9. Study sample • N=205 students enrolled on the module who took the examination • 40 students enrolled on module did not take the examination

10. Findings: pass rate and mean grade

11. Pass rates by degree course

12. Attendance at workshops Figure 1: Percentage of enrolled students who attended workshops • Attendance not recorded for first workshop • Attendance highest at second workshop (37.1%)

13. Percentage who attended workshops • 100 (48.8%) students did not attend any workshops • 105 (51.2%) students attended at least one workshop

14. Attendance and pass/fail in examination

15. Attendance and final grade category Figure 2: Percentage of students who attended workshops by grade category

16. Attendance and course studied Figure 3: Percentage of students who attended workshops by course studied

17. Workshop attendance and grades: 1 Correlation between total number of workshops attended and final grade

18. Workshop attendance and grades: 2 Figure 4: Correlation between number of workshops attended and final grade • Significant positive relationship (r=.314, n= 63, p<0.05) • The more workshops attended, the higher the grade

19. Workshop attendance and grades: 3 Figure 5: Total number of workshops attended and mean final grade Mean final grade overall: 43.61% Conversion Diploma: 60.34% Single Honours: 38.8% Other: 58.2% Joint Honours: 27.44%

20. Similar findings in other research • Lusher (2007) • Small-group workshops focused on assessment criteria with 3rd-year health psychology students • Significant correlation between attendance and mean examination scores (r=0.254, N=111, p<0.01) • Multiple regression showed that performance did not independently predict attendance, so not just a matter of more able students attending workshops

21. Students’ perceptions of workshops • 7-point Likert scale: strongly disagree to strongly agree • Mean scores for all items were positive • Understanding what assessment criteria are • Understanding subject matter of cognitive psychology • Achieving a better grade • Producing better writing • Understanding how to demonstrate assessment criteria

22. Students’ perceptions (continued)

23. Students’ perceptions (continued)

24. Students’ perceptions (continued) • Highest mean scores for • “The workshops helped me understand what assessment criteria are” = 6.16 • “The workshops helped me understand what makes a good examination essay in Cognitive Psychology” = 5.92 • Lowest mean scores for • “The workshops motivated me to spend more time studying Cognitive Psychology” = 4.52 • “The workshops helped me understand the subject-matter of Cognitive Psychology” = 4.56

25. Conclusions • Module pass rate lower this year at 62%, compared to 71% in 2007-08 • However, a number of minor changes were made to content and delivery, so comparison across years problematic • In 2008-09, attendance at the workshops was significantly correlated with higher examination grades • Confounding factor is that more able students are more likely to be attending in first place • More tests needed, cf. Lusher (2007) • Students’ who attended perceived the workshops to be helpful for • Understanding what assessment criteria are • Understanding how to demonstrate the criteria in their own writing • Understanding the subject matter of cognitive psychology • Achieving a better grade • Producing better writing • Difficulty of addressing needs of weaker students, even with “embedded” teaching

26. Implications • Students benefit from explicit focus on writing within disciplinary teaching • Students benefit from talking about their own academic writing with trained peers in same discipline • Importance of embedding teaching of writing and “writing to learn” activities within modules, rather than viewing writing as an add-on skill separate from learning subject matter • Planned changes to module for next year • Less distinction between lectures and workshops by identifying 3-hour “teaching blocks” instead, with varied mix of lecture and workshop activities • Time to practice writing in teaching sessions • Peer review of own writing, facilitated by peer mentors

27. References Bakhshi, S., Harrington, K., and O'Neill, P. (in press). Psychology students’ experiences of academic peer mentoring at the London Metropolitan University Writing Centre, Psychology Teaching and Learning, Spring 2009. Harrington, K., Elander, J., Norton, L., Reddy, P., Aiyegbayo, O. & Pitt, E. (2006). A qualitative analysis of staff-student differences in understandings of assessment criteria, in C. Rust (Ed.), Improving Student Learning through Assessment. Oxford: Oxford Centre for Staff and Learning Development. Lea, M. R. & Street, B. (1998). Student writing in higher education: an academic literacies approach, Studies in Higher Education, 23, pp. 157-72. Lusher, J. (2007). How study groups can help examination performance, Health Psychology Update, 16, 1 & 2. Norton, L., Harrington, K., Elander, J., Sinfield, S., Lusher, J., Reddy, P., Aiyegbayo, O. & Pitt, E. (2005). Supporting students to improve their essay writing through assessment criteria focused workshops, in C. Rust (Ed.), Improving Student Learning: Inclusivity and Diversity. Oxford: Oxford Centre for Staff and Learning Development. Merry, S., Orsmond, P. & Reiling, K. (1998). Biology students’ and tutors’ understanding of a ‘good essay’, in C. Rust (Ed.), Improving Student Learning: Improving Students as Learners. Oxford: Oxford Centre for Staff and Learning Development. Price, M. and O'Donovan, B. and Rust, C. (2001). Strategies to develop students' understanding of assessment criteria and processes, in C. Rust (Ed.), Improving Student Learning - 8: Improving Student Learning Strategically. Oxford: Oxford Centre for Staff and Learning Development. Rust, C., Price, M. & O’Donovan, B. (2003). Improving students’ learning developing their understanding of assessment criteria and processes, Assessment and Evaluation in Higher Education, 35, pp. 453-472.

28. http://www.writenow.ac.uk