TIP Model

1. TIP Model Use – Track - Evaluate Transition Facilitators

2. Now that you are trained in the TIP Model: Your agency may want to track the TIP Model Core Practices used by the Transition Facilitators. By doing this your agency will: • help improve the quality of service to youth- it helps the agency with rapid quality improvement activities. • track how are you using the Core Practices • improve transition age youth’s goal attainment. TIP Model: Transition to Independence Process Model Transition Facilitator: Staff, Service Providers Transition Age Youth (TAY): Youth between the age of 14-29 years old

3. TIP ModelUse – Track – Evaluate Overview Simcoe Muskoka Transition Age Youth System of Supports (TAY SoS) TIP Model Evaluation Working Group decided to focus on three main areas: CLIENT OUTCOMES PROGRAM FIDELITY STAFF SATISFACTION

4. TIP ModelData Collection & Reporting Overview * After staff are trained in the TIP Model and an implementation plan has been created, what data collection and reporting will happen within the agency (pending capacity). Program Fidelity Client Outcomes Program Fidelity Staff Satisfaction Try this for 3 months! CYRM-12: Child and Youth Resilience Measure- 12 items

5. Your agency may choose to track Core Practices manually with the following sheet and others available in Excel. TIP Core PracticesTracking Sheet (Tracking_Sheet_ 1_Client)

6. Some agencies will integrate the tracking process in their current data base. For example, CMHA has a roll down menu to track Core Practices. Others like Kinark has a Supervisor tally up the Core Practices when reviewing files. Fill in the Monthly Tracking SheetTracking_Sheet_2_Staff Any questions about Tracking contact Patrick.russell@camh.ca

7. Making The TIP Model Work • In some cases, a Core Practice like Futures Planning and SCORA might need more than one session to complete the practice. If the Transition Facilitator includes all the Fidelity Standards listed that he/she was planning to complete in the session, each session should be reported as a use of that Core Practice TOOLS: TIP Core Practices Fidelity Standards; ”When to Use”Guidelines; Guidelines and Engagers

8. What is “Attainment an end Goal” ? • Each agency defines what would be considered an end goal • Transition Facilitators will track the number of youth that achieved an end goal while receiving TIP-based interventions. This is done in the agency’s implementation plan Examples of “Attainment of an end Goal”: Kinark: Measure goal attainment based on the youth’s rating of the degree their treatment goal(s) were achieved using 7 out of 10 or higher as the cut off for the goal being attained. Elizabeth Fry: Measure goal attainment based on Youth Justice client’s having met or completed their diversion goals.

9. TIP Model Attainment of an End Goal • WHY measure the attainment of an end goal? • WHO administers and collects? • With WHAT resources ? • HOW often?

10. Let’s See How The TIP Model is Working for the Youth You Serve • Child and Youth Resilience Measure – CYRM 12 CYRM 12 Measures: TAY’s capacity for resilience by measuring their perceptions on how they define their situations and their everyday realities.

11. Child and Youth Resilience Measure (CYRM-12) • WHY survey for Resilience? Pre test : used as a screener focuses on where TAY are doing well and where they could use more help . Post test: measures the impact the TIP Model is having on youth outcomes. - Resilience or being resilient means : being able to safely find one’s way through life’s hard times. • WHO administers and collects data? Transition Facilitator or any person designated by the agency. An electronic format could be used to deliver the survey. • With WHAT resources? CYRM-12 Survey, Script, FAQ sheet and an Excel spread sheet • HOW often? Once when TAY enter a service program and once upon program exit or after 6 months

12. Let’s See How The TIP Model is Working for the Youth You Serve YOUTH SERVICE SURVEY • YOUTH SERVICE SURVEY • Measures: • The quality of services TAY received • How these services have impacted their lives

13. Youth Service Survey • WHY survey Youth on Agency’s Service? This is one way to find out how well youth are liking the service and the TIP Model. • WHO administers and collects? Transition Facilitator or any person designated by the agency. An online format could be used to deliver the survey. This is a confidential survey. - information on how the YSS will be delivered, collected, stored is determined by the agency and via the implementation plan • With WHAT resources ? YSS Survey, Script, and an Excel spread sheet template or computer version • HOW often? Deliver YSS Survey when TAY exit a service program or after 6 months or as defined by your agency.

14. Why SIMCOE and MUSKOKA chose to measure client outcomes with the CYRM-12 and the YSS CYRM-12 Child and Youth Resiliency Measure • Developed, field-tested and validated with diverse populations of TAY from 13 different countries (including Canada) • CYRM-12 and TIP Model both built on social ecological model of resilience • Both focus on individual, relational, and community factors that impact a youth’s capacity for resilience Youth Service Survey • Field-tested and validated with service-using youth across the world • YSS is relatable to the TIP Model as it contains items that measure how involved youth are in planning their goals/treatment YSS

15. TIP ModelTransition Facilitator Survey • Now that you are actively using the TIP Model: • What do you think about it? • Where can it be improved? • Your Supervisor, Manager or Evaluation person will send you a link to a fluid survey: 5-10 min. to complete Voluntary Confidential • Only CAMH’s Performance Measurement and Implementation Research Team will see the raw data.

16. TIP Model Transition Facilitator Survey (Staff Survey) • WHY survey the Transition Facilitators? This is one way to find out how Simcoe Muskoka staff think and feel about using the TIP Model to guide their work. • WHO administers and collects? A Supervisor, Manager or Evaluation person will send a link to staff to a fluid survey. This confidential survey will be completed online and submitted to kristen.morin@camh.ca Agency specific information will be shared, if possible, with agency’s Evaluation Lead to do their own analysis • With WHAT resources ? Online Fluid Survey • HOW often? Every 6 months

17. What’s Working For You and Where Can You Improve? • Self-Evaluation SELF-REFLECTION It is encouraged that you use the National Network of Youth Transition for Behavioral Health (NNYT) Evaluative tool regularly for self reflection For personal development It is not included in the Evaluation Framework

18. Fidelity is Adult Behaviour Fidelity matters!!! Higher fidelity is correlated with better outcomes! I didn’t have potatoes, so I substituted rice. Didn’t have paprika, so I used another spice. I didn’t have tomato sauce, so I used tomato paste. A whole can not a half can – I don’t believe in waste. My friend gave me the recipe and said you couldn’t beat it. There must be something with her, I couldn’t even eat it! Source: Senior Center Newsletter and Michelle Duda’s Implementation Science presentation Thunder Bay 2014.

19. What are “Fidelity Measures” ?How Well Do You Know The Youth You Are Serving? TIP Model Fidelity and Quality Improvement Tools • A Series of 5 Probes (series of questions) each one representing one of the life domains • They were created to review the Transition Facilitators knowledge of a selected young person’s goals, strengths, skills, needs and risk behaviours and to see if the model is being used as intended. Five Life Domains: Employment & Career Education Housing Personal Wellbeing Community-Life

20. TIP Model Fidelity & Quality Improvement Tools There are three different measures in the Practice Fidelity Probes Part A – Review of Transition Domain Part B – Application of TIP Model Core Practices Part C - Validation of TIP Practices in Service Documentation For Simcoe and Muskoka purposes, the reporting will consist of the results from Part A and Part B . Supervisors may complete all sections with their staff as QI tool

21. Practice Fidelity Probes • WHY Practice Fidelity Probes? • WHO administers? Team of 3 preferably; Supervisor plus 2 others • WHO will collect data? Supervisor or whoever is designated • With WHAT tools ? QI Fidelity Workbook, Template Spreadsheet data • HOW often? when TAY exit a program or after 6 months or as defined by your agency???

22. Fidelity Probes – Available In 5 Life Domains

23. Guidelines for TIP Fidelity & Quality improvement Tools Here’s the process in pictures for the Supervisor, Manager or Evaluation person leading the interviews.

24. Youth Focus Group – Another Fidelity Measure WHY agency with a Young Person Focus Group to see how the TIP Model is working in their personal transition.WHY

25. Young Person Focus Group • WHY Young Person Focus Group? • WHO administers? • WHO will collect data? • With WHAT tools ? Focus Group Tool – Young Person Focus Group on Quality and Effectiveness of a Transition Program • HOW often?

26. Reporting Information • Your Agency will complete its own report. • Templates have been provided from CAMH to help your agency analyze data in a set number of ways: • Excel Scoring 5 item, or 3 item CYRM • Excel Scoring YSS • QI Fidelity Workbook – Template Spreadsheet Data (NNYT’S ?) • Agencies can do additional analysis if they choose • According to your Agency’s reporting plan, a summary of results from these reports will be submitted to : • Kristen Morin • Kristen.Morin@camh.ca

27. We know who administers surveys and collects data. Who reports data and why?

28. Who Else Sees This Information?

29. CAMH reports data to TAY SoS Partnership- LHIN - Coalition • Combine responses from staff from agencies across Simcoe and Muskoka to provide a system wide understanding of: • How staff are tracking Core Practices and Goal Attainment • How staff are using the TIP model feel about it

30. Making Services And Systems Better For TAY in Simcoe and MuskokaThe Big PICTURE • By using, tracking and improving how we use the TIP Model, we are making a measurable system level change in helping support youth during their transition to adulthood as they work towards their goals related to school, work, housing, jobs, and personal well-being.