Elena Zaitseva , Clare Milsom, Martyn Stewart Academic Enhancement Unit

1. Keeping the students satisfied: a longitudinal, comparative institutional analysis of survey free-text comments Elena Zaitseva , Clare Milsom, Martyn Stewart Academic Enhancement Unit Liverpool John Moores University

2. Student voice and satisfaction • HEIs are concerned with meeting expectations of current and prospective students • Potential of the NSS for enhancement of the student learning experience has been considered by the HEA and HEFCE • LJMU introduced ‘Mirror’ surveys for Years 1 and 2 • Enhancement is only possible when you know what the issues are

3. NSS/Mirror surveys open text comments • An extensive source of authentic student voice • Analysis is often confined within departments/schools and not much cross-institutional sharing is taking place (various reasons for it) • Institutional level analysis is time and resource consuming, but is there a value to it?

4. An ‘alternative’ approach to analysing student comments • Using semantic analysis software Leximancer • Concept map shows a ‘landscape’ of student satisfaction/dissatisfaction • Sentiment analysis helps to establish an ‘institutional attitude’ to certain aspects of student experience • Longitudinal study undertaken – looking at the NSS and Mirror surveys results over two years (to identify changes and impact of interventions)

5. Year 1 comparative analysis 2011 2012

6. Year 1 comparative analysis • Feedback (fav) changed its location on the map: in 2012 it is closely positioned to helpful and tutor • Assignments: 2011 (unfav), positioned closely to lecture and communication(both unfav) ; in 2012 – favor concept, close proximity to tutor(s) and helpful • Learning is a new concept for 2012, as well as practical – both favourable and closely positioned on the concept map • Coursework changed sentiment to unfavourable in 2012 (group based coursework, coursework on joint degrees)

7. Year 2 comparative analysis 2011 2012

8. Year 2 comparative analysis • Feedback is perceived more positively by year 2 students in 2012 • Course becomes unfavourable concept for 2012 cohort (result of a major curriculum change – move to 24 credit modules) • Modules, being a favourable concept in 2011, change sentiment in 2012 (perceived lack of flexibility, limited optional module choice) • Practical – a new and favourably rated concept for 2nd years in 2012

9. NSS concept map 2011

10. NSS concept map 2012

11. NSS comparative analysis • Students are more concerned with employability and relevance of their degree to the world of work (course is connected to skills and knowledge in 2012) • Library changed position and became a favorable concept in 2012 (result of institutional actions taken) • Feedback still unfav but likelyhood is less then in 2012 (13 and 04 accordingly) • Degree changed sentiment to unfavorable in 2012 (students are more consented with relevance and practical side of the degree)

12. Conclusions • Student satisfaction is a dynamic process • Analysing comments longitudinally can help to identify changes in students’ attitude and see the impact of interventions at institutional level • Quick data turn around allows timely academic development activities • Presenting open text data as concept maps is a more engaging way of discussing some of the issues with academics

13. 'the practice of science happens at the border between the known and the unknown…..there is no certainty; there is simply the best description we have of the Universe....'. (Cox 2011)