Retention Convention Leeds 3-4 March 2010

1. Retention ConventionLeeds3-4 March 2010 Identifying students ‘at risk’ of early withdrawal: issues and dilemmas Dr Elena Bedisti, Dr Kirsten Hall, Dr Maura O’Regan, Dr Sue Robbins

2. ‘What Works? Student Retention and Success National Audit Office Report (2007): Staying the Course. The retention of students in Higher Education: Strong performance of HEIs in retaining their students Wide range of advice on good practice in supporting and retaining students. Little evaluation of the impact and transferability of this practice Following the NAO report, HEFCE and the Paul Hamlyn Foundation made available a total of £1 million from 2008-2011 to support projects that identify, evaluate and disseminate institutional analysis and good practice relating to student retention. The ‘What Works? Student Success and Retention Programme’ aims to analyse and share good practice about the most effective strategies in order to ensure high continuation and completion rates within HE. Seven universities were selected and have been awarded grants to run the projects. All but one are collaborative projects. In total, 21 institutions are directly involved in the grants programme.

3. ‘What works? Student Retention and Success Programme’ – 2 key areas of work • providing academic support to students • promoting academic and social integration into the institution to promote a sense of belonging • Reading/Oxford Brookes Collaborative project: • Investigating the impacts on student retention of different approaches to Supporting students through study advice and personal development

4. Identifying ‘at risk’ students

5. ‘At risk’ HEFCE • Risk Factors: • Qualification on entry • Tariff Points • School Type • POLAR 2 – Postcodes which are identified as low HE participation quintiles (quintiles 1 and 2) • Socio-economic classification code • (risk factors between 4-8)

6. Reading ‘at risk’ *2008/09 cohort: N = 2,510 Total ‘at risk’ = 214 ‘At risk’ students withdrawn = 31 25 withdrawals 6 withdrawn as a result of academic failure 1 student was suspended * Expecting complete withdrawal data/retention rates from Planning *2009/10 cohort: N = 2,944 Total ‘at risk’ = 190 Total withdrawals from cohort: 33 ‘At risk’ students withdrawn = 9 * Preliminary data (March 1st 2010)

7. Reading ‘at risk’ Survey 2009-10 (2) – all Part 1 Undergraduate Students (158 responses): ‘I have thought about withdrawing from University’ 27 strongly agreed/agreed with the statement 2 were part of our ‘at risk’ students (‘I thought I might as well make it to the end of the academic year’, ‘stayed because of the benefits of a full degree’)

8. Survey 2009-10 (1_short) – all Part 1 Undergraduate Students (231responses): ‘I have thought about withdrawing from University’ No respondent belonged in the ‘at risk’ category

9. Short survey Survey 2009-10 (1_short) – all Part 1 Undergraduate Students (231responses): What the found helpful in their decision to stay:

10. Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method • The HEFCE method identifies students as being of High, Medium or Low risk dependent on their entry qualifications. • The Robbins method asks students to fill in a questionnaire that asks them a number of questions, including questions about their previous science qualifications. This enables the PASS team to ‘score’ for relevance the subject areas and grades that a student has obtained before entering the School of Life Sciences. Hence students are designated as: • At risk • Not at risk • Disengaged (if they do not respond to requests to fill in the questionnaire)

11. Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method Once the above profiling is established scores are assigned:

12. Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method A total A2 score of combined cohorts 2005-06 to 2008-09. For each A2 Robbins scoring band, percent of students in that band who passed/failed modules in year 1 Nearly half of students without A level entry qualifications fail at least one module in their first year. These students include those entering from ACCESS programmes, BTECs as well as baccalaureates. It appears that there is a problem if students do not have an academic background and lack academic skills

13. Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method vs HEFCE • Robbins method identifies 230 students as being ‘At risk’ whereas HEFCE identifies 88 students • Of those 88 students only 21 (24%) are identified as ‘At risk’ by the Robbins method • 93 (36%) of students identified as ‘At risk’ by the Robbins method are identified as ‘low risk’ by the HEFCE method • 25 (28.5%) students are identified as ‘high risk’ by HEFCE whereas using the Robbins method they are categorised as ‘disengaged’

14. Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method vs HEFCE ‘at risk’ criteria Academic outcomes for Robbins method of identifying ‘at risk’ students Of the 210 students who failed one or two modules in year 1, 68% were identified by the Robbins method as being ‘at risk’ or ‘disengaged’

15. Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method vs HEFCE ‘at risk’ criteria Academic outcomes for HEFCE risk criteria of identifying ‘at risk’ students Of the 210 students who failed one or two modules in year 1, 59.05% were identified by the HEFCE ‘at risk’ criteria as being of ‘High’ or ‘Medium’ risk of failure

16. Oxford Brookes ‘at risk’ (School of Life Sciences) – The Robbins Method vs HEFCE ‘at risk’ criteria • Conclusion: • The Robbins method identified 451 students as ‘at risk of failure’ or as ‘disengaged’. Of the 210 students that failed one or more year 1 modules, 68.1 were in this group of 451 students • The HEFCE categories identified 348 students as being at ‘high’ or ‘medium’ risk of failure. Of the 210 students that failed one or more Year 1 modules, 59.05% were in this group of 348 students • The Robbins method successfully identified nearly 10% more of the 210 students who went on to fail one or more year 1 modules

17. Self-reported academic confidence • Students were asked to rate their confidence (1-4) in a range of skills: speaking & listening, reading and researching, writing, time management, numeracy and IT. • Preliminary results (4 consecutive years were aggregated N=681): • The whole cohort is more confident about reading and researching than speaking and listening or writing. Time management confidence levels were somewhere between. Students’ confidence levels at numeracy approach those of reading and researching. • No correlation (positive or negative) was found in any skill area between self reported confidence at academic skills and academic results. This would indicate that students’ self assessment is not particularly accurate. However, some interesting features still emerged from the data. • One quarter of the students who failed one or more module had extremely high confidence in their reading and researching, and their numeracy skills. While Life Sciences modules depend to differing extents on students’ numeracy skills, all modules demand reading and researching skills. This would suggest that a high proportion of failing students have misplaced confidence in their reading and researching skills.

18. Self-reported academic confidence • Students were asked to rate their confidence (1-4) in a range of skills: speaking & listening, reading and researching, writing, time management, numeracy and IT. • These data were also broken down into age groups: 18-20, 21-29, 30+. This showed up certain features of the 30+ age group, which did not appear in other age groups: • Those who were confident at speaking and listening were more likely to be those who failed. Those less confident in this area were much less likely to fail. • This age group tend to be more confident than others at reading and researching, and at time management. As would be expected, they are less confident at IT skills. • The extremely confident at writing, at reading and researching, and at numeracy are more likely to fail. Their misplaced confidence may be through not knowing the requirements, having been out of education for the longest time. • One third of failing 30+ students have medium to high ‘life concerns’ that are affecting their studies. In fact, all of the most highly concerned students failed one or more module.

19. conclusions: • Identifying ‘at risk’ students using the HEFCE method seems to be ‘working’ to a certain extent • In the case of the School of Life Sciences, a more discipline specific method identifying students ‘at risk’ appears to be more successful (Robbins method) • Preliminary results show that there does not appear to be a correlation between self reported academic competence at academic skills and academic results • Some thoughts… • Identifying students ‘at risk’ – why? • Identifying students ‘at risk’ to target support or provide support for all • Identifying students ‘at risk’ – one method fits all? • Ethical implications about identifying students ‘at risk’