Career Development Proving our Value - Getting Much Closer

1. Career DevelopmentProving our Value - Getting Much Closer January, 2014

2. Evidence-Based Research in Canada National research agenda to better understand “what works” in career services 3 calls for proposals from HRSDC over 6 years Source of major international envy among Career Development Practitioners (CDPs) and Researchers

3. Why Evidence-Based Practice? • Ethics: Clients deserve interventions that have demonstrated success • Confidence: Practitioners should know that a given intervention will result in a predictable outcome • Security:Being able to provide evidence of success increases ability to obtain and keep funding for services and secure jobs for CDPs/Employment Counsellors

4. Why Evidence-Based Practice? • Outcome-focused intervention • Evidence of client change …contrast with… • Client flow • Ease of access • Practitioner time use It’s easy to measure variables that don’t address client change

6. Session Objectives Highlight results of 3 recent research projects: What have we learned? Explore how a select number of LMI and career development resources could be integrated into everyday evidence-based practice Explain where we are now and next steps Direct you to additional research studies

7. Before we start……. What evidence do you currently collect? What is done with the evidence currently collected?

8. Reflective Questions…. What did the research do? What was learned? What was most surprising? What are the implications for practice?

9. Research Project #1:Assessing the Impact of LMI on Career Decision Making and Work Search Research Team Canadian Research Working Group in Evidence-Based Practice (CRWG), Canadian Career Development Foundation (CCDF) In partnership with New Brunswick Post-Secondary Education & Labour, Saskatchewan Advanced Education, Employment & Labour

10. What is the problem? • There is very little literature about the direct contribution of LMI to employment/career decision-making (Savard& Michaud, The Impact of LMI on Career Decision-Making Process: Literature Review, FLMM, 2005) • Several questions remain unanswered: • How do people actually use LMI? • What (if any ) assistance would be helpful?

11. Research Questions • If client needs are assessed and clients are given LMI consistent with their needs, • To what extent does assistance by a service provider enhance their effective use of LMI? OR • To what extent is independent self-help a sufficient process for clients to use LMI effectively?

12. Method • We prepared “guided” LMI packages (take-home booklets and Resource Centre binders) on: • Career Decision Making: • Know yourself • Know the Labour Market • Put it all Together • Job Search: • Check for “Fit” • Get Ready • Search for Work • Get a Job

13. Intervention All participants in the study: • Received a needs assessment interview & completed an initial, pre-program survey • Received an LMI package specific to their identified employability need • Were randomly assigned to either a self- directed intervention delivery method or an assisted self-directed intervention delivery method. • Were given an orientation to the Resource Centre which they could freely use on their own • Completed a pre and post-pre survey • Received a cash honorarium and certificate of participation

14. Intervention (cont) The self-directed group: • worked independently for 3 weeks, making use of the materials and the Resource Centre • returned in Week 4 for their exit interview

15. Intervention (cont) The assisted self-help clients received: • two additional AIS (Advice and Information) interviews (20-30 minutes) in weeks 1 and 3 focused on helping them understand, interpret and apply the LMI to their own situations and /or access additional LMI • returned in Week 4 for their exit interview

16. Sample by Province

17. What did we Measure The Dependent Measures for the data analysis were: • General ability to use LMI • Knowledge • Clear vision of what I want in my career future • Knowledge of print and online resources • Skill • Have effective strategies for keeping myself motivated • Have a realistic action plan • Personal Attributes • Optimism about what lies ahead re meeting my career goals • Confidence in my ability to manage future career transitions

18. Examples of Questions used in pre and in post-pre surveys

19. Descriptive Results • Changes in competency self assessments: Pre program: • 30% and 58% of the respondents rated competency Not OK • 5% rated their competency as Exceptional Post program: • 1%– 9% rated competency Not OK • 39% rated their competency as Exceptional The amount of change was similar across all three dimensions of the survey, knowledge, skills, and personal attributes all demonstrated about the same amount of change.

20. Descriptive Results (Cont.) • Of particular note are items that suggest increased ability to self-manage their careers, such as: • A clear understanding of what I need to do to move forward in my career. • A clear vision of what I want in my career future. • Effective strategies for keeping myself motivated to achieve my career/employment goals. • Confidence in my ability to research career, employment, and training options that are available • Confidence in my ability to manage future career transitions.

21. Differential Results-Total Score • For group as a whole: • significant increase in overall ability to use LMI • neither intervention was more conducive to one manner of delivery compared to the other • Both CDM and JS groups had significant increases across time • Change in CDM group was significantly larger than in JS group • Participants in the JS group had higher scores than participants in the CDM group, likely indicating that JS participants were more familiar with using LMI before the project began. • Participants receiving assistance demonstrated greater change across time than did those in the independent modeSimilar pattern for all subscales

22. Attribution for Change To what extent would you say that any changes in the ratings on the previous pages are a result of your participation in this research project, and to what extent were they a function of other factors in your life? Program

23. Study repeated in francophone new brunswick • Overall,nosignificantdifferences in the results of the anglophone and francophone sampleswith the exception thatassisted francophone clients reportedincreased gains over the anglophone clients althoughboth groups reported more positive change in the assisted condition.

24. Follow up interviews (1 week and 4 monthsafter intervention) • “I was surprised at how much information there was” • “Initially I thought I could not do this, but I found out I could” • “There’s a lot to learn” • “I got more focused on my goal” • “I got clearer about what I wanted to do”

25. Client Criticisms • The program needed to be longer given the number of resourcesavailable. (222) • Working full time made itvery hard […]. With a little more help itwould have been mucheasier (242) • There wastoomuch information and that made making a careerdecision more difficult […] very hard to do thisalone (209)

26. What did practitioners say? • “The structure helped me be more focused and concrete” • “I never really thought about intake of clients as assisted or self-help—now it’s built into every intake I do” • “I used to be maternal and forget that it was the client’s decision and I got more involved than I needed to. Now I ask the clients to be more responsible” • “Now I give more homework tasks and I’m more specific. The clients felt more focused and so do I”

27. What did the Research do? • Isolated LMI as an intervention • Isolated giving information and advice from employment counselling • Organized LMI into a career development learning sequence • Tested self-help versus a combination of self-help, information giving and advice

28. What was learned? • Tailored LMI embedded in a learning process results in knowledge and skill acquisition as well as the capacity for self-management • LMI appropriate for a client’s specific need (opposed to general LMI) appears to support engagement and action • For many clients, a little (or no) professional LMI support is enough • Structure and timelines appear to motivate action and a sense of progress • Giving clients hands-on tools appears to motivate more than money

29. What was most surprising • The extent of positive client change in knowledge, skill and personal attributes in both conditions of self-help and supported self-help • The value clients placed on the LMI materials and the degree to which they used them • The positive impact of “organized” LMI and LMI “specific to client need” versus generalized LMI

30. What are the implications for practice? • To be discussed……..

31. Research Project #2:Assessing the Impact of Career Resource and Supports Across the Employability Dimensions Drs. Dave Redekopp and Bryan Hiebert Alberta Human Services Manitoba Entrepreneurship, Training and Trade Canadian Career Development Foundation (CCDF)

32. What is the Problem? There is very limited research about the direct contribution of career development learning resources on client progress toward or achievement of employability. There are many “assumptions” made about self-help resources but very little actual evidence of their impact. There is much “talk” about labour market attachment but on investigation it remains a very “wooly” construct.

33. This Study… Developed comprehensive career development coaching guides (way beyond LMI) covering all employability dimensions excepting career growth: • CAREER DECISION MAKING: Not yet clear about work/learning goal; needs to understand personal skills, interest, values and personality and more about different potential occupations in order to develop a vision for a preferred future • SKILLS ENHANCEMENT: Has career goal, but needs to explore skills and qualifications needed for the work they want and how to acquire them • JOB SEARCH: Has career goal and necessary skills/qualifications for desired work; needs skills, strategies and tools necessary to conduct a successful job search • JOB MAINTENANCE: Has difficulty keeping work; may need to address personal/life issues (e.g. health, behaviour, transportation, child care) or may need to work on the skills/attitudes required for success at work. The last part of this Guide is appropriate for someone already working, but wanting to improve their chances of keeping the job.

34. This Study… • Randomly assigned clients to one of two treatment conditions: • Practitioner launched (in which they had their needs assessed, were oriented to the appropriate Guide and then worked independently with their Resource Guide for 4 weeks) • Practitioner launched and supported (in which they also worked with their Career Consultant for 4 weeks of “regular” services (i.e. 2-4 sessions on average) using the Guide as appropriate but not restricted in their interventions  • Examined differential impact across treatment conditions on client knowledge, skills, personal attributes and labour market outcomes

35. Developed Two New Indices… Self Help Index(self-help-o-meter) to see if: The propensity for self-help can be measured The ability to self-help affects client outcomes Labour Market Attachment Index to see if: Specific variables related to the client’s work history and life situation can be measured These variables affect client outcomes

36. Research Question If clients are given a comprehensive needs assessment to determine their employability need(s), what is the differential effect of independent and consultant-supported career development resources on clients who are weakly attached to the labour market versus those who are strongly attached to the labour market? Will all clients demonstrate positive change? Will clients who are strongly attached to the LM show more positive change? Will clients who are supported by a consultant show more positive change? Will weakly attached clients struggle more in the independent condition?

37. Sample Composition

38. Key FindingsCareer Decision MakingTotal Score Summary: Main Effect for Delivery: F(1,70)=1.32; p=.26Main Effect for Time: F(1,70)=253.72; p<.01Delivery X Time Interaction: F(1,70)=.66; p=.42

39. Key FindingsSkill EnhancementTotal Score Summary: Main Effect for Delivery: F(1,35)=.48; p=.04Main Effect for Time: F(1,35)=69.33; p<.01Delivery X Time Interaction: F(1,35)=.51; p=.48

40. Key FindingsJob SearchTotal Score Summary: Main Effect for Delivery: F(1,72)=.71; p=.40Main Effect for Time: F(1,72)=260.49; p<.01Delivery X Time Interaction: F(1,72)=2.07; p=.16

41. Key FindingsJob MaintenanceTotal Score Summary: Main Effect for Delivery: F(1,15)=.18; p=.68Main Effect for Time: F(1,15)=28.03; p<.01Delivery X Time Interaction: F(1,15)=.19; p=.67

42. Attribution of Change 91% of clients in all interventions attributed their positive change partly (40%) or mostly (51%) to their participation in the study 92% had a clear plan for next steps after the 4 week intervention period 98% planned to use their Guide again and/or shared it with friends and family

43. In the Words of Clients… • The Guide was hugely beneficial. It was a profound and amazing experience. • My current job is not a good fit for me, but the job I have just been offered (after working with the CDM guide) is the PERFECT fit for me. • The guide really clarified for me what I want to do in my work – what is important to me. It helped me to realize I have the qualifications and [know]what I love to do. It re-connected me with myself. • The guide was a catalyst for action.  I knew in the back of my mind for a long time that I wasn’t happy in my work, but it was the guide that made me do something about it.  • I think the guide is pretty universal for Canada. Combined with the skills I got from my employment counselor, I feel more confident and feeling like I’m going in the right direction. It was only 4 weeks but sometimes that’s all you need! • Definitely not feeling the way I did before the guide. • This experience was phenomenal. I’m so glad I got to be part of it! I’m grinning ear to ear!! • I think high school students should get this.

44. Results Statistically and clinically significant positive impact of career interventions across ALL sub-scores and across ALL employability dimensions – compelling evidence of positive changes in clients as a result of career development coaching interventions

45. Results (cont’d) More positive trends for supported group but differences are not statistically significant (surprising and troubling result until overly caring practitioners admitted to “creaming” the sample with the best of intentions and researchers realized too late that practitioners really do care!!)

46. What Did the Research Do? Developed and tested impacts of career development coaching resources on changes in specific employability dimensions Simulated regular employment services to the extent possible Attempted to “unpack” two poorly understood constructs—capacity to self-help and labour market attachment

47. What We Learned • Many clients can benefit significantly from self-help resources when they are matched to their need and they are “launched” – more so than we anticipated! • This is especially noteworthy given the short intervention period of 4 weeks and the “real-life” setting in which the changes occurred.

48. What We Learned (cont’d) Self-help coaching guides provided after a thorough needs assessment and orientation are effective and can be used as a first line of intervention, saving valuable practitioner time for clients who really need it, or potentially opening a promising perspective on online career services. More research is needed to test the “Self-Help Index” and a “Labour Market Attachment Index”. These tools could not be effectively tested due to the unanticipated “creaming” of clients who, in the view of the practitioners, might have had difficulty with self-help materials (if assigned to the independent group) and/or might fall at the low extreme of labour market attachment.

49. What Was Most Surprising? The statistically and clinically significant amount of change experienced by clients – especially given the short intervention period The number of clients interviewed after the study who credited the guides for significant positive life changes The extent to which practitioners remained committed to the research despite significant additional demands on their time

50. What Are the Implications for Practice? To be discussed……..