1 / 47

Relationship of Fidelity to Wraparound and Outcomes for Youth and Families

Relationship of Fidelity to Wraparound and Outcomes for Youth and Families. 6 th Annual CANS Conference Betty Walton, Vicki S. Effland & Eric Bruns San Francisco, CA April 20, 2010. Objectives. Effective implementation of research based practices Relationship of fidelity to Outcomes

giles
Download Presentation

Relationship of Fidelity to Wraparound and Outcomes for Youth and Families

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Relationship of Fidelity to Wraparound and Outcomes for Youth and Families 6th Annual CANS Conference Betty Walton, Vicki S. Effland & Eric Bruns San Francisco, CA April 20, 2010

  2. Objectives • Effective implementation of research based practices • Relationship of fidelity to Outcomes • Case Study: Implementation of Wraparound in Indiana • Next Steps

  3. What is Wraparound? • Wraparound is a family-driven, youth guided, team-based process for planning and implementing services and supports. • Through the wraparound process, teams create plans that are geared toward meeting the unique and holistic needs of children and youth with complex needs and their families. • The wraparound team members (e.g., the identified youth, his or her parents/caregivers, other family members and community members, mental health professionals, educators, and others) meet regularly to implement and monitor the plan to ensure its success.

  4. For which children and youth is the wraparound process intended? • Youth with needs that span home, school, and community • Youth with needs in multiple life domains • (e.g., school, employment, residential stability, safety, family relationships, basic needs) • Youth for whom there are many adults involved and they need to work together well for him or her to succeed

  5. For which youth in a system of care? More complex needs Most Intensive intervention level Full Wraparound Process 2% 3% Targeted Intervention Level 15% Targeted and Individualized Services Prevention and Universal Health Promotion Level 80% Less complex needs

  6. What is wraparound intended to achieve? • Basing plans on strengths, needs, and culture leads to more complete engagement of families • High-quality teamwork and flexible funds leads to better plans, better fit between family needs and supports, and greater integration of effort by helpers • = Greater relevance, less dropout, better follow-through • As family works with a team to solve its own problems, develops family members’ skills andself-efficacy • Process focuses on developing supportive relationships • Focus on setting goals and measuring outcomes leads to more frequent problem-solving, more effective plans, greater success

  7. What is the research base?Nine Published Controlled Studies of Wraparound *Included in 2009 meta-analysis (Suter & Bruns, 2009)

  8. Research to Date on Wraparound • There have been 9 controlled studies of wraparound published in peer review journals • Results consistently indicate superior outcomes for wraparound compared to “services as usual”* • Moderate (ES = .50) effects for living situation outcomes • Small – medium (ES = .25 - .40) effects for behavioral, functional, and community outcomes • These ESs are similar to studies of evidence based therapies and interventions (e.g., MST) as implemented in real world conditions against alternative treatment conditions *Suter, J.C. & Bruns, E.J. (2009). Effectiveness of the Wraparound Process for Children with Emotional and Behavioral Disorders: A Meta-Analysis. Clinical Child and Family Psychology Review, 12, 336-351

  9. We Have Issues. • 17 year gap between proving that an approach works and implementation in practice • Effective implementation involves multiple factors (Fixsen, Naoom, Blasé, Friedman & Wallace, 2005)

  10. Implementation with High Fidelity Requires… County Context and Readiness State Support Staff Selection Organizational Supports Program Evaluation Training Supervision and Coaching Performance Management National Implementation Research Network (NIRN)

  11. Fidelity is critical to outcomes • Higher levels of fidelity to organizational level assessment for ACT was associated with greater reductions in days spent in psychiatric hospitals (McGrew, Bond, Dietzen & Salyers, 1994) • Improved youth delinquency outcomes for higher fidelity Teaching Family model (Kirigin et. al. 1982) • Improved youth delinquency outcomes for higher fidelity MST (Henggler, Melton, Browndino, Scherer & Hanley, 1997) • Better overall outcomes for youth receiving model adherent FFT (Alexander, Pugh, Parsons and Sexton, 2000) • Better outcomes for school-wide behavioral management when implemented with fidelity (Felner et. al. 2001)

  12. Monitoring fidelity of implementation of child and family teams • Have facilitators and team members fill out activity checklists • Look at plans of care and meeting notes • Sit in on and observe team meetings • Ask the people who know– parents, youth, facilitators, program heads

  13. Wraparound Fidelity Assessment System TOM – Team Observation Measure WFI-4 – Wraparound Fidelity Index WFAS WFI-4 TOM Doc Review CSWI CSWI – Community Supports for Wraparound Index DRM - Document Review Measure

  14. The Wraparound Fidelity Index, version 4 • Assesses implementation of the wraparound process through brief interviews with multiple respondents • Caregivers • Youths • Wraparound Facilitators • Team Members • Found to possess good psychometric characteristics • Test-retest reliability • Inter-rater agreement • Internal consistency • Used in research on wraparound • Even more widely as a quality assurance mechanism by wrap programs

  15. Wraparound Fidelity Index, v.4 • Items on the principles and core activities, organized by the 4 phases of wraparound • Engagement: Did you select the people who would be on your youth and family team? • Principle = Team based • Planning: Does the plan include strategies for helping your child get involved with activities in the community? • Principle = Community based • Implementation: Does the team evaluate progress toward the goals of the plan at every team meeting? • Principle = Outcome based • Transition: Will some members of your team be there to support you when formal wraparound is complete? • Principle = Persistence

  16. Findings from fidelity research • Fidelity-outcomes associations are tenuous and inconsistent at the family/youth level • At the site/program level, there is a discernable pattern of WFI Fidelity scores across studies • Wraparound vs. non-wraparound programs • Wraparound programs with different levels of system support • Wraparound programs that achieve better outcomes • Beginning to be able to interpret the “meaning” of WFI scores • < 65% = Not wraparound • 65 – 75% = Low average / Borderline • 75% - 85% = High average / Acceptable • > 85% = “High fidelity”

  17. Wraparound in Indiana • Began in mid- 1990s…. • 3 CMHI grants • State legislation to replicate • 2 year development grants • Pilot 1915c Medicaid Waiver

  18. Lessons Learned Not Enough More Effective Sources inspiring, but … Legislation and state policy Targeted funding and contracts Training Persuasion Collaborating with stakeholders (youth & families, providers, systems) Gauging readiness & adjusting tactics Ongoing training, coaching and support Develop adequate infrastructure Monitor progress and use feedback

  19. CA-PRTF Grant Section 6063 of the Deficit Reduction Act of 2005 ( PL 109-171) authorized up to $217 million for a demonstration program for grantee states to use Medicaid funding for home and community-based services as an alternative to psychiatric residential treatment facilities (CA-PRTF). Research Questions. Does the provision of Medicaid-funded home and community based services to youth under this demonstration: • Result in the maintenance or improvement in child’s functional status? • On average, cost no more than anticipated aggregate PRTF expenditures in the absence of the demonstration?  • National & State Evaluation: IMPAQ & Westat, Project Director, Oswaldo Urdapilleta; Principal Investigator (PI), Garrett Morgan • Wraparound • 9 States

  20. Evaluation Minimum Data Set • Demographic Information • Functioning (CANS, CBCL or CAFAS) • Fidelity (WFI-4) • Services • Child Welfare and Juvenile Justice Involvement • Youth Services Survey (YSS & YSS-F) • Costs +Strengths Based Site Assessment (Effland, 2010)

  21. Indiana Youth Receiving Grant Services with Outcome & Fidelity Measures Jan 2008 - March 31, 2010 – 660 participants 372 youth had at least one CANS reassessment & a WFI survey Average age = 12.95 years old 71.3% Male 79.2% White 15.7% African American 4.4 % Hispanic

  22. Measuring Improvement Improvement in any Domain • Behavioral Health • Risks • Functioning • Strengths • Caregiver Reliable Change Index Calculated for Each CANS Domain, p < .05.

  23. Expected Reliable Change • 60-80% of youth are expected to improve in at least one of the dimensions • 20-40% of youth will improve in a specific dimension

  24. Improvement for Indiana Youth Ending an Episode of Public Mental Health ServicesJan – March 2010, n = 2218

  25. CA-PRTF GrantMonitoring Improvement (CANS), n = 484

  26. Variance in Improvement in 1 domain by Provider

  27. Variance in Improvement in Each CANS Domain by Provider Continued on next slide………

  28. Variance in Improvement in Each CANS Domain by Provider (continued)

  29. CA-PRTF Grant Fidelity, n = 481 teams *( Buchard& Bruns, 2007)

  30. Wraparound Fidelity (n = 481 teams, 3/2010)

  31. Using WFI Scores for Quality Improvement • Statewide Profile of WFI Elements, Phases and Total Fidelity • Personalized Reports for each Access Site (County) with 5 or more completed WFI surveys • Distributed at Quarterly SOC Meetings Indiana CA-PRTF: WFI Averages for XXXXXX County January 2008-March 2010 Indiana CA-PRTF Medicaid Demonstration Grant Wraparound Fidelity Index (WFI-4) Overall Wraparound Fidelity = .84 n = 34 child and family teams

  32. Comparison of Respondent’s Perspectives

  33. Variance in Caregiver’s WFI by Provider

  34. Strengths Based Site Assessment (Effland, 2010) • Completed by TA Center Coaches • Rating Guide Community characteristics at each stage of implementation 1. Collaboration & partnership 2. Capacity building & staffing 3. Acquiring services & Supports 4. Accountability 5. Family Support Overall Stage of Implementation = mean of ratings across the 5 community characteristics

  35. Stage of Implementation* Results • Stage 1 (Exploration & Adoption) 20.0% • Stage 2 (Program Installation) 34.5% • Stage 3 (Initial Implementation) 32.7% • Stage 4 (Full Operation) 12.7% *National Implementation Research Network (NIRN) (Fixsen, et. al, 2005)

  36. Wraparound Fidelity by Stage of Implementation • WFI interviews for at least one youth in 33 of 51 communities • 263 interviews in the 33 communities • Average of 8 interviews per community

  37. Wraparound Fidelity by Stage of Implementation

  38. Relationship of Fidelity and Outcomes(Effland, McIntyre & Walton, 2010)

  39. Logistic Regression Predicting Improvement in 1 CANS Domain __________________________________________________________ Predictor ßSEOdds ratio __________________________________________________________ Outcomes Based(WFI) 2.058 .782 7.833** Risks_Baseline(CANS) .145 .046 1.156** Functioning_Baseline .098 .043 1.103* Strengths_Baseline .076 .030 1.079* __________________________________________________________ *p< .05, **p< .01 Note: Total Fidelity Element Scores used in model.

  40. Logistic Regression Predicting Improvement in Behavioral Health _____________________________________________________ Predictor ßSEOdds ratio Age -.116 .049 .890 Psychosis_Baseline .295 .148 1.343* Anxiety_Baseline .589 .155 1.803*** Conduct_Baseline .646 .163 1.908*** Trauma_Baseline .387 .129 1.472** Substance Use-Baseline .674 .213 1.963** Team Based (WFI) -2.484 1.029 .083* Natural Supports (WFI) 1.690 .591 5.421** Collaborative (WFI) -3.217 1.327 .040* Persistent (WFI) -1.843 .830 .158* Outcomes Based (WFI) 2.299 .944 9.965** *p<.05, **p< .01, ***p<.001

  41. Logistic Regression Predicting Improvement in Risk Domain _____________________________________________________ Predictor ßSEOdds ratio ______________________________________________________ RiskDomain_Baseline .150 .048 1.162** Functioning_Baseline .087 .044 1.091* StrengthDomain_Baseline .085 .032 1.089** Outcomes Based (WFI) 1.887 .787 6.600* ______________________________________________________ *p<.05, **p< .01, ***p< .001 Total Fidelity Scores used in model.

  42. Logistic Regression Predicting Improvement in Functioning _____________________________________________________ Predictor ßSEOdds ratio _____________________________________________________ Race(White) .610 .357 1.840 Race (Black) 21.947 19472.230 3.400 Race(Native American) -.016 .698 .984 FunctioningDomain_B .339 .048 1.404*** FamilyVoice (WFI) -2.582 1.298 .076* Natural Supports (WFI) 1.523 .563 4.588** Outcomes Based (WFI) 2.815 .955 16.700** Age -.119 .046 .888** *p< .05, **p < .01, p<.001

  43. Logistic Regression Predicting Improvement in Strengths _____________________________________________________ Predictor ßSEOdds ratio _____________________________________________________ StrengthsDomain_B .315 .040 1.371*** Total WFI 4.379 1.500 79.734** Age -.122 .047 .885** _____________________________________________________ *p<.05, **p<.01, ***p<.001 .

  44. Logistic Regression Predicting Improvement in Caregiver Strengths & Needs _____________________________________________________ Predictor ßSEOdds ratio _____________________________________________________ Age_Baseline -.100 .044 .905* Anxiety_Baseline .126 .152 1.201 CGDomain_Baseline .126 .029 1.134** ______________________________________________________ *p< .05, **p<.01, ***p < .001

  45. Conclusions • Evidence of association between level of system of care development, wraparound fidelity and outcomes • Considering elements of wraparound, domains and key item CANS scores enrich understanding • Outcomes for “No Wraparound” similar to usual public services • Emerging evidence of the importance of outcomes based practice & overall fidelity link to building strengths • Further research indicated

  46. Next Steps • Update analysis every 6 months to monitor trends, & continue using WFI information to improve services • Compare Group(s) of Youth with Similar Needs who received different services • Consider refining Fidelity categories • Explore what factors predict high fidelity and positive outcomes • Explore who benefits from intensive community based services under what circumstances

  47. Betty A. Walton: betty.walton@fssa.in.gov http://dmha.fssa.in.gov/darmha Vicki S. Effland: veffland@choicesteam.org Eric J. Bruns: ebruns@uw.edu www.wrapinfo.org; www.nwi.pdx.edu For more information, contact:

More Related