Impact Assessments: Celebrating success ( or how to quantify the impact of learner support for inspection )

1. Impact Assessments:Celebrating success ( or how to quantify the impact of learner support for inspection ) NAMSS National Conference. March 2011.

2. Introduction Why impact assess How could you impact assess Use of KPI and Student Data Customer Journey Mapping Staff and Student Evaluations The Impact on services at Solihull College Celebrating Success Question and Answer Session Overview

3. Our approach has been developed in the context of: A General FE College Need to improve data management (2007) Need to improve performance management (2007) Student Services team structure (see delegate pack) Introduction

4. Our approach has been developed to measure: Equality and Diversity Profile of service users Presentation of Issues i.e. why they access the service (careers, counselling, welfare) Attendance, Retention and Success rates Student and Staff views of service provision Why Impact Assess?

5. Our approach has been developed to measure: Performance Monitor Service Level Agreements i) Response times ii) Waiting times Profile workload (peaks and troughs) Why Impact Assess?

6. Our approach has been developed to measure: Caseloads Profile workload i) Contact types (telephone, email, in person)ii) Contact choices (drop-in, one-to-one)iii) Turn up rate at appointments Engagement with the academic Faculties Why Impact Assess?

7. Tally Sheets Accommodation Drop-in Services How do we measure KPI Data?

8. Partnership Returns Sexual Health College Nurse Job Centre Connexions How do we measure KPI Data?

9. Student Surveys Settling in On Programme FFE NSS Bespokee.g. Link Users How do we measure KPI Data?

10. Database (Student Finance) How do we measure KPI Data?

11. Database (ECM) How do we measure KPI Data?

12. Database (Careers) How do we measure KPI Data?

13. Database (‘At Risk’Attendance and Disciplinary) How do we measure KPI Data?

14. KPI Reports (Every 6 weeks) How do we report KPI Data?

15. Periodic Impact Assessments i.e. OFTED Monitoring Visit, Peer Reviews. How do we report KPI Data?

16. End of Year Reports How do we report KPI Data?

17. ManagementRequestedReports(CommonRoom Users) How do we report KPI Data?

18. Business Planning(3 year Review) How do we report KPI Data?

19. Self Assessment Report (SAR) How do we report KPI Data?

20. 2009-2011 Pilot within Public Services School 261 in study with a 25% return rate 2010-2011 Pilot extended to include:Animal Care (169 to date, 18% return rate)A Levels (427 to date, 19% return rate)Hairdressing (103 to date, 23% return rate) Information Technology (134 to date, 33% return rate) Public Services (140 to date, 27% return rate) Customer Journey Mapping

21. Evaluates with an ‘emotional response’ for certain key stages in Admissions cycle including: Pre-Interview i.e. course and college selection Interview Welcome Days Enrolment Induction Customer Journey Mapping

22. Applicants are invited via open questions to make comments on three aspects of the stage (see delegate pack) In addition the applicant is asked the same single emotional question at each part of the journey. These scores are used to map the emotional response to each stage along the journey. The emotional question “How do you feel about you decision to come to Solihull College” would be scored on scale of Plus Ten to Minus Ten. Customer Journey Mapping

23. Management Reports on CJM Customer Journey Mapping

24. Student evaluations are undertaken for: All one-to-one sessions (careers, welfare, counselling) All tutorials delivered by Student Services staff All events organised by Student Services. All students accessing Financial Support. Student Evalutions

25. All evaluations are analysed and reports via meeting structure with separate QIP. Student Evalutions

26. Staff evaluations are undertaken for: All ‘core’ aspects of Student Services by College Management Team All one-to-one services (careers, welfare, counselling) by teaching staff. Staff Evaluations

27. Welfare End of Year Report 2007-08 KPI data showed the service had supported a range of issues such as homelessness, family difficulties, debt and finances. Good links with external agencies in place. Highlighted awareness and accommodation as areas for improvement. Examples of Impact on our Service

28. Impact of Service Delivery Effective use of resources The move from 3 FTE Counselling posts to 0.5 FTE with the additional appointment of a Welfare Officer was based on KPI data in 2006-07. This led to an increase in students receiving intensive one to one support.

29. Impact of Service Delivery Attendance and Success Rates

30. Impact of Service Delivery Student Evaluations of Provision ‘If I didn’t have support from Student Finance then I wouldn’t be able to afford it. If it wasn’t for help with travel, trips and exams then I would not be able to keep coming to college.’ Student Finance Case Study Review 2009-2010

31. Impact of Service Delivery

32. Impact of Service Delivery Student Evaluations of Provision Student evaluations of counselling completed this year have included the comments: ‘I found that I was able to talk about things more easily with the counsellor and I found that I understood what was happening to me’ Counselling Evaluation Form 2009-2010 Although I felt bad on some days – by talking to a counsellor I was able to carry on with my course’ Counselling Evaluation Form 2009-2010

33. Impact of Service Delivery Student Evaluations of Provision Student evaluations of welfare and counselling are completed at first meeting and at ‘last meeting. They score their personal well-being from very poor to very good.

34. Impact of Service Delivery Student Evaluations of Tutorial Provision

35. OFSTED Monitoring Visit (May 2009) What progress has the college made in making further improvements to student services? Significant Progress “Managers conduct a rigorous analysis of at-risk students and monitor the level of support to ensure their progress.” MONITORING VISIT: MAIN FINDINGS Impact of Service

36. Peer Review Visit (October 2009) Key Strength: Comprehensive performance management systems in place to monitor the impact of student services activities taking place in tutorials with feedback to curriculum areas on under utilisation. MONITORING VISIT: MAIN FINDINGS Impact of Service

37. Student Services Self-Assessment Report (October 2009) Department graded as ‘outstanding’. Key strengths included:Engagement with Student Services has demonstrable impact on attendance, retention and completion with high levels of engagement learners identified as Widening Participation and BME. Comprehensive performance management systems in place with Leadership and Management in the area delivering continuous improvement. Impact on Service

38. Full OFSTED Inspection (May 2010) “Attendance for learners who access many aspects of the support service is better than the college average.” “Staff monitor the effectiveness of the support it provides well and can demonstrate that it is effective.” “An increasing proportion takepart in the wide and developing range of sporting and other physical activitiesat the college….Learners also appreciate the information they receive on drug,alcohol and mental health awareness through tutorials” Full OFSTED Report Impact of Service

39. Evaluations are part of WSMT reporting. Quotations used in ECM posters (see delegate packs) SLA are posted on College intranet. Achievements against SLA are printed in Student Services literature. Celebrating Success

40. Staff Update Student Update Use of Case Studies Video and Pod Casts Celebrating Success

41. Any Questions before our activity Q&A Session