Collaboration on Impact Learning: Shared Measurement Approach in Northern Ireland Employability Programs

1. Collaboration on Impact Learning from a Shared Measurement Approach within post 16 Employability Programmes in Northern Ireland Gladys Swanton - CENI Lorraine Boyd - NEET Forum www.inspiringimpact.org

2. Context for Inspiring Impact • Inspiring Impact UK (2012): 10 year strategy – changing the way that Voluntary/Community sector and its funders think about impact: “putting impact practice at the heart of their work”. • Building Change Trust - Big Lottery £10 million investment focusing on the VCS in Northern Ireland. Funded Inspiring Impact NI Programme for an initial 2-years (2013) (with additional funding from the DSDNI) • IINI built upon existing work in the UK including: an Impact Practice Code; a Measuring Up assessment tool for organisations. IINI has been working across 3 main strands: • Support to the VCSE on Impact Practice • Engagement with funders to develop impact practice • Inspiring Impact Exchange • CENI (Community Evaluation NI) is the strategic partner for the delivery of IINI

3. …Journey towards EmploymenT The idea for JET Shared impact measurement approach • More joined up between funder and funded on impact • Better understanding of the desired impacts and what needs to be measured • Projects supported around a common framework • Potentially more comparable data to get assess achievements & learning

4. Voluntary and Community NEET Strategy Forum We are a group of organisations working with young people and have a shared vision that by: ‘working together so every young person in Northern Ireland is in meaningful education, training and employment’

5. Aim of the Voluntary and Community NEET Strategy Forum The NEET Strategy Forum works collaboratively to inform and showcase proactive work that respects the voice of, and focuses on the right of young people to access, maintain and progress to employment of their choice in Northern Ireland and beyond.

6. NEET Strategy Forum

7. Loose affiliation of organisations came together to : • to enable those working in the V&C Sector to share information and concerns experienced by young people who experienced difficulties access Education, Training, and Employment • to influence the development and implementation of a Strategy for young people who in need of education, employment and training Development of the V&C NEET Strategy Forum

8. Committee for Employment and Learning Inquiry into Young People not in Education, Employment or Training Volume One Together with the Minutes of Proceedings, Minutes of Evidence andWritten Submissions Relating to the Report Ordered by The Committee for Employment and Learning to be printed 15 December 2010Report: NIA 32/10/11R The Committee for Employment and Learning

9. Northern Ireland (NI) Executive NEET Strategy‘Pathways to Success’ in June 2012 Launch of Pathways to Success Our aspiration is that, by 2020, every young person will not only have an opportunity to access education, training or other preparation for employment but, to the extent that they are able, also avail of that opportunity.

10. NEETs Advisory Group (NAG) Interdepartmental Group – Chaired by Deputy Permanent Secretary Chaired by Deputy Permanent Secretary (DEL) 12 Departments 5 Voluntary and Community Sector NEET Strategy Forum NILGA – Local Government Employer representative

11. NEET Youth Forum • Core Young people who are in need of education, training and employment • Young people from across NI, Age ranging from 16 – 24 • Share experience and knowledge • Inform their choice of preferred best practice. • Have genuine constructive input into Northern Ireland NEET reduction agenda

12. Key components of work include: Collaboration Communication Best Practice Mapping provision and production of directory Youth Participation.

13. JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – Pilot in NI ……… • Initial presentation of JET to NEETs Forum • Building Change Trust support to pilot • Consultation to develop a draft NI framework • Testing of JET Framework & Process in NI • Expression of Interest from 14 C/V/SE groups • 6 pilots were identified for support

14. JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – LAUNCHED MAY 2013 A shared approach to measuring young people’s employability

15. Journey to EmploymenT (JET) framework Intrinsic factors Employability skills + + Qualifications education + training Attitudes + + Emotional capabilities Experience + involvement + + Personal circumstances Extrinsic factors Career management External factor: The labour market

16. JET OUTCOMES (NI draft) • PERSONAL CIRCUMSTANCES • Access to transport • Personal Safety • Access to the internet • Access to childcare • Access to support for young carers • Supported by family and friends • Access to support for disabled people • Reduced substance abuse • Resisting negative peer influences • Reduced offending/anti-social behaviour • QUALIFICATIONS, EDUCATION + TRAINING • Basic skills • Achieving qualifications • Attendance and behaviour • EXPERIENCES + INVOLVEMENT • Work experience • Perception of value of work experience • Networks • Volunteering • Inter-community engagement • CAREER MANAGEMENT • Career direction • Job search skills • Presentation to employers • Confidence in finding employment • Entrepreneurship • EMOTIONAL CAPABILITIES • Self-esteem • Autonomy and control • Grit and determination • Empathy • ATTITUDES • Aspirations for education • Attitudes to work • Aspirations for work • Aspirations for the future • EMPLOYABILITY SKILLS • Teamwork • Communication • Problem solving • Self-management • Leadership • EMPLOYMENT • Entry into employment • Sustaining employment • Quality of employment • Satisfaction with employment

17. Implementing JET in NI • Planning • Complete theory of change with each pilot group • Prioritising & matching to JET Framework • Measuring • Select measures & designing • Implementing/ sample/timeframe • Analysis & Use • Pre and post analysis • Using the learning

18. Six Participating GroupsSupport delivered May-Oct14 JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – Pilot in NI

19. Step1: THEORY OF CHANGE COMPONENTS Enabling factors: things that have to exist for the theory of change to happen Evidence or assumptions: why you think one outcome will lead to another and cases where it will not Activities: what the charities’ services are Outputs: what services are provided Outcomes: Short – Medium termchanges that the charity wants to happen Impact: Ultimate goal - what the charity is trying to achieve

20. Digital Media Example Project Outcomes – Short - Medium End Goal/Impact: Project activities Y/P are willing to engage with work/ learning outside of their local area YP achieve qualifications and experience required by employers in digital media YP have increased knowledge/skills in use of a range of digital media YP have greater choices whether to pursue further education/ training/ employment /self-employment ie entrepreneurship Enablers: Access to up-to-date quality training, equipment, resources, expertise and practical work experience Learner friendly approach and peer support Digital media training programme over 30 weeks/one day a week Y/P have increased confidence/ self-esteem/ achievement Work experience to contribute to digital media projects Y/P gain entrepreneurial skills / know what is involved in developing a business in the digital media field YP have positive role models Y/P secure jobs in digital media • Y/P have improved • employability skills: • Improved team-working • Improved communication • Improved problem solving • Improved self-management • Showing leadership Y/P are up-skilled and inspired

21. Priority Outcomes and JET Measures TableExample

22. STEP 4: USE THE JET FRAMEWORK TO SELECT MEASURES Example: Rosenberg Self-esteem scale 22

23. Step 5: CHOOSE A RESEARCH DESIGN After the course Post-questionnaire Before the course: Pre-questionnaire Long term follow up Intervention 23

24. Outcomes Prioritised by NI Pilot Projects

25. Final steps STEP 6 IMPLEMENTING MEASUREMENT TOOL STEP 7 ANALYSE THE DATA STEP 8 LEARN AND IMPROVE 25

26. “ The JET process supported the development of good impact practice within the organisation including: planning impact, using new tools to measure, building in analysis, learning and reporting • The use of externally validated measures strengthens the evaluation process and provides greater credibility when reporting about achievements Feedback on the Outcomes and measures… • The JET framework enabled a broader range of outcomes to be captured at a project level; outcomes which are not often articulated or measured • The JET pilot was part of the wider strategic level discussions on impact measurement which added value to the whole process

27. “Theory of Change was a useful process to think through – making the journey explicit “how to enable young people to move from hopelessness to aspiration” • The pre and post questionnaires were straightforward to use and advice in the JET pack on the practicalities of implementation useful • The pre-coded Excel analysis sheets produced a range of data useful for reporting – however user-friendly formats for presentation of data would be useful Views on use and implementation of JET resources • The visual presentation on the JET process and the Framework was seen as user friendly and eye catching • Looked at ‘Impact Practice’ in more systematic way. Breaking the process down into 8 distinctive inter-linking steps helped to make it manageable – all the steps were important • It was useful to have a menu of measures to draw from and be able to pick what suited – but there are some challenges with “off-the-shelf” measures

28. Taking it Forward • All the pilots groups indicated interest in using JET in their work • Pilot groups valued the CENI support in navigating their way through the 8 steps - training and support is needed • Feedback from the pilot support informed the final developmental phase – plans for user-friendly web-based version • Access to Resources • JET Framework NI on-line • Case studies made available on-line • Process Report on Piloting JET in NI

29. Thank You • Lorraine Boyd Regional Project Manager NEET Strategy Forum NI • E: lboyd@brysongroup.org • T: +44(0)2890325 835 • WWW.Neetforumni.org