IS2007 Aviemore, Scotland Vivienne Brown

1. Making a difference for individuals and the economyCareers Scotland’s research on the impact of career guidance and development services IS2007 Aviemore, Scotland Vivienne Brown

2. Why measure impact? • does careers guidance and development have an impact? How do we know? • can we articulate the impact of career guidance and development for individuals and specific projects and at higher policy or national levels? • can impact determine what contribution career guidance and development can make in key policy areas i.e. economic, social and education? • can impact evidence influence policy from an improved position of knowledge? • can we create a value for career guidance?

3. Measuring Impact – first steps towards a conceptual impact model • undertook desk research using the available literature on impact of career guidance and development to date, in the UK and beyond • created a set of hypotheses, with potential outcomes, based on the data review • identified the degree of availability of evidence using a traffic light system • available evidence was used to estimate the likely effects of career guidance for each inferential statement • we applied a GVA/GDP model of calculation to create an impact value • hypotheses also indicated potential topics for future research

4. Research Study Outcomes • increased knowledge of the impact and value of Careers Scotland based on a developing Conceptual Impact Model • advice on how better to articulate the impact and value of career guidance from a variety of viewpoints for stakeholders, partners, managers, staff and clients • identification of gaps in evidence provides opportunity to develop a Careers Scotland Research and Development strategy that can support future impact measurement as part of other research and evaluation

5. consideration of different approaches to widen the evidence base including longitudinal tracking, and other research with partners • build-in impact and value in all future research and evaluation • advice on developing our performance management systems, to improve our own data collection to measure impact

6. Hypotheses - the likely effects of career guidance on inferential statements – Learning Goals Outcomes • greater access to learning and training • greater participation in learning and training • higher retention rates in education and training • greater education and training attainment; and higher level skills • improved motivation and hence attainment in education and training Impacts • higher wage levels through gaining higher qualifications • increased entry rates through having a career focus

7. Hypotheses - the likely effects of career guidance on inferential statements – Economic Goals Outcomes • higher levels of participation in employment • lower levels of unemployment • improved job tenure through increased motivation at work • a more responsive and flexible workforce • improvements in the employability of individuals Impacts • higher wage levels - a long run uplift in wages attributed to career guidance • improved productivity

8. Hypotheses – the likely effects of career guidance on inferential statements - Social Goals Outcomes • increased confidence • increased well being which contributes to health benefits for society • reductions in crime and offending behaviour • greater levels of social inclusion Impacts • reductions in lost earnings and lower productivity through “lost” education and training • reductions in social security, NHS and other public costs

9. Traffic light system • satisfactory evidence: sufficient evidence to establish the link between career guidance and stated outcomes - information available across a number of sources with at least one element of robust empirical evidence • partial evidence: incomplete evidence to establish the link between career guidance and stated outcomes - info available across one or two sources and for some stated outcomes, sources lacked empirical evidence • little or no evidence - absence of empirical evidence with no source of information directly and adequately addressing the link between career guidance and stated outcomes

10. Applying a value – GDP/GVA calculations Examples • economic impact - increased workforce participation based on the calculation of an employment differential attributed to career guidance • learning impact - increased attainment levels associated with career guidance and the resulting improved earnings associated with higher levels of attainment • social impact - costs avoided including the costs of unemployment • GVA/GDP was calculated, using the evidence available, at more than 5 times Careers Scotland’s current annual budget - approximately £250m

11. Additional Findings – soft indicators The evidence used to develop the hypotheses also showed that career guidance and development: • raises self-confidence and self-belief • improves motivation, well-being and willingness to take calculated risks, based on improved understanding of work and learning goals • supports individuals to set career goals which in turn drives stronger educational ambition and career aspiration

12. often works best in the medium to longer term, and as part of a “package” of support, especially for those from lower socio-economic backgrounds • makes greatest impact on individuals with limited social ‘networks’, irrespective of academic ability • can demonstrate a longer term uplift in wages for those who have career goals and undertaken progressive career development actions, thereby contributing to a lifelong learning and career development culture

13. Key Messages • impact has been partially evidenced; and given an interim value • as additional evidence becomes available it is anticipated the value of career guidance and development will increase exponentially • need for investment in longitudinal tracking to elicit a clearer, bigger picture of impact; and ultimately value • most of the value has been calculated using the economic hypotheses – imbalance in the evidence available

14. as the gaps in impact evidence have been identified, particularly in relation to the learning and social policy areas, that may affect policy decisions on career guidance role in these policy goal areas • career guidance and development is most effective when used as part of a wider learning, training and personal development package. • partners involvement in impact measurement is required to more fully understand and articulate impact, which in turn may support better inter-agency working

15. Implications for Careers Scotland • consider new questions and samples for existing longitudinal tracking • find ways to improve attributability of positive client outcomes to career guidance activity, including learning and skill development • undertake an incremental research programme to strengthen evidence base • articulate better the impact and value of career guidance and development to stakeholders, partners staff and clients to create a more demand-led approach • work more with others to plan and undertake research