SSEUK Student Engagement Survey 2011

1. www.warwick.ac.uk/go/iatl SSEUK Student Engagement Survey 2011 • Jere Koskela • Gary Lee • Paul Taylor

2. www.warwick.ac.uk/go/iatl Aims • Explore practical issues with implementing Student Engagement Surveys in the UK • Carry out an in-depth study of the SSE carried out at Warwick in February 2011 SSEUK

3. www.warwick.ac.uk/go/iatl Objectives • Look for correlations between NSS and SSE data. • Compare SSE data from Reading and Warwick. • Assess whether SSE data can be used to benchmark against NSSE / AUSSE institutions • Triangulate the conclusions using focus group research. • Make recommendations on the use of SSE in the UK. SSEUK

4. www.warwick.ac.uk/go/iatl • http://www2.warwick.ac.uk/fac/cross_fac/iatl/engaged/ • Online questionnaire accessible to whole student body. • 29 questions and ~2000 responses, ~15% response rate • e.g. This year, how often have you worked with classmates outside of class to prepare assignments? • On scale of “Very little/not at all”; “Some”; “Quite a bit”; “Very much” • Interested in aggregate scores (0-3) across departments, faculties and benchmarks. Methodology

5. www.warwick.ac.uk/go/iatl • Two focus groups were run with ~10 participants in each from among survey respondents. • Participants selected to be approximately representative of university population: degree course and home department, gender, age, nationality etc. • Conversation on topics arising repeatedly in survey data • e.g. Group based learning, Acquiring broad education, Course structure Methodology

6. www.warwick.ac.uk/go/iatl • Reading survey uses 7 benchmarks • We divided our questions to cover 4 of these • Active and Collaborative Learning • Student Interaction with Academics • Social Inclusion and Internationalisation • Career Prospects and Employability • Average results within benchmarks compared across years of study and faculties. Replicating Reading University

7. www.warwick.ac.uk/go/iatl Replicating Reading University

8. www.warwick.ac.uk/go/iatl • Overall patterns are similar to Reading. • But clear differences in details. • Our results show greater variance. • Our question set is different, hence variation in details is to be expected. • It seems likely that directly comparable results could be obtained with standardised questions. Replicating Reading University

9. www.warwick.ac.uk/go/iatl NSS uses 7 indices • Overall • Teaching • Feedback • Support • Management • Resources • Development Our 4 benchmarks: • ACL • SIA • SII • CPE Correlation with the NSS → 28 natural comparisons

10. www.warwick.ac.uk/go/iatl Correlation with the NSS

11. www.warwick.ac.uk/go/iatl Correlation with the NSS

12. www.warwick.ac.uk/go/iatl • One in 28 plots shows relationship. All others contain no obvious pattern. • As we are using the same data for all 28 plots, one result among 28 is insufficient to conclude the results are correlated. • It appears there is no correlation between SSEUK and NSS results Correlation with the NSS

13. www.warwick.ac.uk/go/iatl • http://nsse.iub.edu/html/summary_tables.cfm • http://www.acer.edu.au/documents/aussereports/AUSSE_2010_AustralasiaUniversity_Report_Results.pdf • Comparing our aggregate results with 2010 NSSE and 2010 AUSSE across departments, faculties. • Comparisons done against Research University/Very High Carnegie classification • Comparisons restricted to first year and final year students NSSE/AUSSE

14. www.warwick.ac.uk/go/iatl NSSE/AUSSE by Department

15. www.warwick.ac.uk/go/iatl NSSE/AUSSE -Whole University

16. www.warwick.ac.uk/go/iatl • Our results look directly comparable with both NSSE and AUSSE. • Benchmarking against North American institutions allows us to highlight differences within Warwick departments. • Collecting results for first year students and finalists highlights variation across time. • Question by question comparison enables detailed analysis of the consequences of policies in education. NSSE/AUSSE

17. www.warwick.ac.uk/go/iatl • “We do a lot of group based work. I think it's good because it brings together a lot of people, it's quite easy to get perspectives from people from all around the world.” Focus Groups

18. www.warwick.ac.uk/go/iatl • “As an American, I remember applying to English universities and being impressed with the lecture seminar system. I like the system of 5-10 people in an office discussing about ideas and work.” Focus Groups

19. www.warwick.ac.uk/go/iatl • “We have absolutely no theory-based modules, mainly practical. [Theory] is almost non-existent.” • “It would be nice for lecturers to integrate real life situations with the theoretical concepts taught.” Focus Groups

20. www.warwick.ac.uk/go/iatl Conclusions • No correlation between SSE and NSS results. • SSE can be used to compare UK institutions. • SSE can be used to benchmark against NSSE (in particular) and AUSSE institutions. This benchmarking is robust enough to allow comparisons of individual departments. • Qualitative study supports quantitative data. Warwick achieves strongly compared to NSSE in critical thinking and communicating ideas, but less strongly in collaboration between students and staff. SSEUK

21. www.warwick.ac.uk/go/iatl Recommendations • SSE could be used to benchmark against NSSE and AUSSE institutions and identify strengths of UK higher education as well as opportunities for improvement. • The survey methodology could be improved, by adopting a standardised questionnaire across the UK. • A separate survey might be conducted for postgraduates and part-time students taking into account their particular circumstances, e.g. POSSE conducted by AUSSE. SSEUK

22. www.warwick.ac.uk/go/iatl • John Creighton • Amy Clarke • Susan Brock • All participating Warwick students With thanks to... The piloting of the AUSSE did provide the context for useful and enriching dialogue about learning and teaching; it remains unclear, however, whether it measures what it sets out to. Voerman, A. and Solomonides, I. 'Discourse and Dialogue in Student Engagement' From Solomonides, I. Reid, A. and Petocz, P. (eds)(forthcoming) Engaging with Learning in Higher Education