1 / 61

Improving Practice to Wraparound Through Use of Fidelity Measures

Improving Practice to Wraparound Through Use of Fidelity Measures. Enrica Bertoldo, Quality Support Manager Eleanor Castillo, Director of Outcomes & Quality Assurance Mary Ann Wong, Research Specialist Veronica Padilla, Manager of Outcomes & Evaluations

vidar
Download Presentation

Improving Practice to Wraparound Through Use of Fidelity Measures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Practice to Wraparound Through Use of Fidelity Measures Enrica Bertoldo, Quality Support Manager Eleanor Castillo, Director of Outcomes & Quality Assurance Mary Ann Wong, Research Specialist Veronica Padilla, Manager of Outcomes & Evaluations 26th Annual California Mental Health Advocates for Children & Youth Conference, Asilomar, California May 4, 2006

  2. Our Mission To work with children and their families to transform their lives, build emotional, social and familial well-being, and to transform the systems that serve them. Our Philosophy Family Voice  Team Based Collaborative/Integrative  Community-Based Culturally Competent  Individualized Strength-Based  Natural Supports Persistence  Outcomes-Based EMQ Children & Family Services

  3. Purpose Given the complexity of Wraparound, existing fidelity measures are utilized to improve practice. • This workshop will share successes and challenges in the implementation of fidelity measures (WFI-3.0 & WOF-2.0) that affect practice and supervision to Wraparound. • A pilot in utilizing a Wraparound supervisory adherence measure (W-SAM) will also be discussed.

  4. What is Wraparound? • Team-based, individualized service planning process • Principles of community-based, strength-based, family-centered, and culturally competent • Wraparound is described as a promising practice (Burns, Hoagwood, & Maultsby, 1998)

  5. Voice and Choice Youth/Family Team Community-Based Services/Supports Cultural Competence Individualized Services/Supports Strengths-Based Services/Supports Natural Supports 8. Continuation of Services/Supports Collaboration Flexible Resources/ Funding Outcome-Based Services/Supports 11 Core Elements of Wraparound

  6. Essential Elements of Wraparound • Element 1: Voice and Choice Families must be full and active partners at every level of the wraparound process. If the team cannot reach consensus, the final decision should be up to the caregiver. • Element 2: Youth and Family Team Wraparound is a team-driven process involving caregivers, youth, natural supports, and community services working together to develop, implement, and evaluate the individualized plan. • Element 3: Community-based Services and Supports Services and supports that the youth and family receive should be based in their community. The family should not have to leave their community if more restrictive services are necessary. • Element 4: Cultural Competence The team should not only be respectful of the family’s beliefs and traditions, but also actively seek to under-stand the family’s unique perspectives and convey them to others.

  7. Element 5: Individualized Services This means that services and supports are tailored to the unique situation, strengths, and needs of teach individual, and may involve existing categorical services and informal sup-ports; modifying existing services and supports; and or creating new services and supports. Further, the team should create a specific plan to meet the family’s goals and crisis/ safety plan to manage potential emergencies. • Element 6: Strengths-Based Services The focus of the team should be on what is working and going well for the family. While goals may be drawn up based on the family’s needs, the plan should capitalize on the family’s positive abilities and characteristics. • Element 7: Natural Supports Services and supports should reflect a balance of formal and informal community and family supports rather than a reliance on formal professional services. • Element 8: Continuation of Care Services and supports must be provided unconditionally. In a crisis, services and supports should be added rather than placing the youth with a new provider.

  8. Element 9: Collaboration The team should coordinate services and supports so they seem seamless to the family rather than disjointed. • Element 10: Flexible Funding and Resources Successful wraparound teams are creative in their approach to service delivery and have access to flexible funds and resources to implement their ideas. • Element 11: Outcome-Based Services Specific, measurable out-comes should be monitored to assess the youth and family’s progress toward goals. The description of each element is taken directly from Wraparound Fidelity Index 3.0 (2002), produced by the Wraparound Evaluation and Research Team at the University of Vermont.

  9. What is Fidelity to Wraparound? • Treatment Fidelity = The degree to which a program is implemented as intended (Rast & Bruns, 2003; Moncher & Prinz, 1991) • Adherence to the 11 Core Wraparound Elements (e.g., WFI-3.0; Suter et al., 2002) • Adherence to the Child and Family Team Process (e.g., WOF-2.0; Epstein et al., 2002)

  10. Why Measure Fidelity? • Measuring fidelity is essential to families, providers, policy makers and researchers • Without measuring fidelity, how do you ensure the Wraparound process is occurring? • Without outcomes, the Wraparound process is just one more fad • High fidelity has been associated with positive outcomes in some studies • It improves quality assurance • It helps agencies secure more funding by proving outcomes • It can even help create legislation on how families and kids receive services

  11. More Why’s….. All nationally recognized Wraparound programs have extensive fidelity measurements in place. Why? Without fidelity measurement standards firmly in place, Wraparound programs and team members have difficulty knowing what they are doing well and what they need to improve in order to achieve the ideal as presented in the Wraparound vision.

  12. Fidelity Implementation Measures • Wraparound Fidelity Index-3.0 (WFI-3.0) • Measures fidelity to Wraparound principles through structured interviews • Administered to Youth, Caregiver, & Resource Facilitator • Higher Score = Greater Fidelity • Wraparound Observation Form-2.0 (WOF-2.0) • Measures fidelity to Wraparound through observation of the Child and Family Team meetings • Higher Score = Greater Fidelity

  13. Fidelity to Wraparound • EMQ Sacramento has collected the Wraparound Fidelity Index (WFI) since 2001 • Data from WFI-3.0: January 2003 – September 2005

  14. A Comparison of EMQ Data & National Data

  15. WFI Total Scores by Respondent: EMQ & National Data *National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the 18th Annual Research Conference on Systems of Care and Children's Mental Health in Tampa on 3/7/05 by Eric J. Bruns, Ph.D. for "Is it Wraparound Yet?" - Bootstrapping wraparound fidelity standards using the WFI .

  16. WFI Elements: EMQ & National Data *National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the Technical Assistance Partnership Webinar on 4/5/04 by Eric J. Bruns, Ph.D. for "Ensuring High-Quality Wraparound"

  17. WFI Elements: EMQ & National Data *National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the Technical Assistance Partnership Webinar on 4/5/04 by Eric J. Bruns, Ph.D. for "Ensuring High-Quality Wraparound"

  18. The Relationship Between Fidelity & Outcomes

  19. The Relationship Between Fidelity & Outcomes • Previous research has found an association between greater Wraparound fidelity and better child and family outcomes (Bruns, 2004), but the relationship has not been clearly understood • EMQ (2005) completed a study to further understand the relationship between fidelity to the 11 core elements of Wraparound and treatment outcomes

  20. Demographics • Average Age at Admission: 14 years • 63% Male; 37% Female • 62% Caucasian; 26% African-American; 8% Latino; 2% Asian/Pacific Islander; 1% Native-American; 1% Other • Average Length of Stay: 15 months

  21. Fidelity to Wraparound • Implementation Measure: • Wraparound Fidelity Index – 3.0 (WFI-3.0) • Collection since 2003 • Sample in study included the following: • 146 WFI’s collected from Youth • 124 WFI’s collected from Caregivers • 183 WFI’s collected from Resource Facilitators

  22. Outcome Measures at Discharge • Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 2000) • Measures youth’s level of functioning; completed by Resource Facilitator • Lower Score = Greater Functioning • Exit Total CAFAS Score • Child Behavior Checklist (CBCL; Achenbach, 2001) • Measures youth’s level of behavior problems; administered to Caregiver • Lower Score = Lower Behavior Problems • Exit Total, Externalizing, & Internalizing CBCL Scores • Living Arrangement at Discharge: Community or Facility

  23. Correlations Among WFI Scores and Outcomes at Discharge Note: *p<.05; **p<.01

  24. Significant Results Note: *p<.05; **p<.01 • Greater community-based services is related to lower impairment on the CBCL and CAFAS at discharge • Greater community-based services is related to community living arrangement at discharge

  25. Results • Greater fidelity to the provision of Community Services/Supports in Wraparound is related to positive outcomes • No other elements had as significant correlations to outcomes as Community Services/Supports • No significant relationship found between Total Fidelity Scores and outcomes at discharge • Correlations were consistently low even when significant

  26. Implications • Implications for teams to support a youth’s school attendance and involvement in work, training, and other community activities (e.g., church, sports, art, etc.) as well as support community-based living situations for youth (as an alternative to residential or institutional care) in order to improve outcomes.

  27. Challenges of the WFI • Direct feedback to teams was not provided • The utility of the WFI in its relationship to outcomes was limited based on EMQ’s analysis of the data • Time intensive and limited resources available to conduct 3 structured interviews per youth

  28. EMQ Practice Standards Initiative

  29. Initiative to Improve Practice • Quality Improvement Project launched in January 2004 to implement practice standards and feedback tools across positions • Family Facilitator • Family Specialist • Family Partner • Goal was to standardize job positions, provide role clarity, and improve consistency in job performance and supervision

  30. Example of EMQ Feedback Tool:Family Facilitator Scoring: Opportunity for Improvement (1) Acceptable (2) Exemplary (3) Not Observed (*) 3 Dimensions: A. Families receive services that are consistent with wraparound values B. Families receive services that are collaborative, integrated and adhere to best practices C. Children and Families served are safe and stable at home and in the community

  31. Facilitator Field Feedback Tool A. Families receive services that are consistent with wraparound values 1. Arrived on time and was prepared for the CFT 2. Started the CFT with “what’s working” to maintain strength based focus and future orientation 3. Facilitated the CFT to validate and incorporate ideas, requests, and concerns of all members 4. Evidence that needs and strategies were reviewed within the CFT to assure the family voice and preference guide all activities 5. Evidence that plans were reviewed for progress on current measurable, observable goals and updated as needed to address new and changing needs 6. Evidence that the next CFT and staff visits were scheduled according to family preference

  32. Facilitator Field Feedback Tool B. Families receive services that are collaborative, integrated and adhere to best practices 1. For children open six months or longer, evidence that the CFT was comprised of at least 50% non-paid informal members 2. Facilitated the CFT to allow pertinent agenda items to be covered according to family preference and time availability 3. Demonstrated engagement and rapport with all team members including system partners 4. Therapy, medications and adjunctive services were evaluated relative to plan goals as needed • Specific tasks were assigned C. Children and Families served are safe and stable at home and in the community 1. Evidence that team has a plan B to back up plan A 2. If the CFT was for a child in out of home placement, this was addressed as a priority with specific plan, tasks, and target date 3. Evidence that strategies and interventions developedemphasize the use of typical resources in the child’s community

  33. EMQ Feedback Tools • Data analysis showed that the tools were not measuring what was intended; therefore, tools were discarded • Led to an examination of the national standards for Wraparound • EMQ Sacramento decided to collect data on the Child and Family Team process using the Wraparound Observation Form – Version 2.0 (WOF-2.0)

  34. Wraparound Observation Form – Second Version (WOF-2) • Measure to assess the implementation of the Wraparound approach by direct observation of youth and family team meetings • WOF has demonstrated good inter-rater reliability (kappa=.886; Nordess & Epstein, 2003) • 48 items that measures 8 characteristics: • Community-Based  Unconditional Care • Individualized  Measurable Outcomes • Family-Driven  Management of Team Meeting • Interagency Collaboration  Care Coordinator

  35. Implementation of the WOF • WOF data collection began July 2005 • Feedback provided in individual supervision if completed by direct supervisor • Verbal feedback provided to facilitators after the child and family team meeting if completed by neutral observer. Copy of tools also provided to supervisors

  36. Wraparound Observation Form – Second Version (WOF-2) • Community-based resources (5 items) 1. Information about resources / interventions in the area is offered to the team. 2. Plan of care includes at least one public and/or private community service/resource. 3. Plan of care includes at least one informal resource. 4. When residential placement is discussed, team chooses community placements for child(ren) rather than out-of-community placements, whenever possible. 5. Individuals (non-professionals) important to the family are present at the meeting.

  37. Individualized services for the family (9 items) 6. If an initial plan of care meeting, the parent is asked what treatments or interventions he/she felt worked/didn’t work prior to WRAP. 7. Facilitator advocates for services and resources for the family (e.g., identifies and argues for necessary services). 8. All services needed by family are included in plan (e.g., no needed services were not offered). 9. Barriers to services or resources / interventions are identified and solutions discussed. 10. The steps needed to implement the plan of care are clearly specified by the team. 11. Strengths of family members are identified and discusses at the meeting. 12. Plan of care that includes life domain(s) goals, objectives, and resources/interventions is discussed (or written). 13. Plan of care goals, objectives, or interventions are based on family/child strengths. 14. Safety plan/Crisis plan developed/reviewed.

  38. Family-driven services (10 items) 15. Convenient arrangements for family’s presence at meeting are made (e.g., location, time, transportation, day care arrangements). 16. The parent/child is seated or invited to sit where he/she can be included in the discussion. 17. Family members are treated in a courteous fashion at all times. 18. The family’s perspective is presented to professionals from other agencies. 19. The family is asked what goals they would like to work on. 20. The parent is asked about the types of services or resources/interventions he/she would prefer for his/her family. 21. Family members are involved in designing the plan of care. 22. In the plan of care, the family and team members are assigned (or asked) tasks and responsibilities that promote the family’s independence (e.g., accessing resources on own, budgeting, maintaining housing). 23. The team plans to keep the family intact or to reunite the family. 24. Family Members voice agreement/disagreement with plan of care.

  39. Interagency collaboration (7 items) 25. Staff from other agencies who care about or provide resources/interventions to the family are present at the meeting. 26. Staff from other facilities or agencies (if present) have an opportunity to provide input 27. Informal supports (if present) have an opportunity to provide input. 28. Problems that can develop in an interagency team (e.g., turf problems, challenges to authority) are not evident or are resolved. 29. Staff from other agencies describe support resources interventions available in the community. 30. Statement(s) made by a staff member or an informal support indicate that contact/ communication with another team member occurred between meetings. 31. Availability of alternative funding sources is discussed before flexible funds are committed.

  40. Unconditional Care (3 items) 32. Termination of Wrap services is discussed because of the multiplicity or severity of the child’s/family’s behaviors /problems. 33. Termination of other services (not Wrap) is discussed because of the multiplicity or severity of the child’s/family’s behaviors /problems. 34. For severe behavior challenges (e.g., gangs, drugs), discussion focuses on safety plans/crisis plans (e.g., service and staff to be provided) rather than termination. • Measurable Outcomes (3 items) 35. The plan of care goals are discussed in objective, measurable terms. 36. The criteria for ending Wrap involvement are discussed. 37. Objective or verifiable information on child and parent functioning is used as outcome data.

  41. Management of team meeting (5 items) 38. Key participants are invited to the meeting (e.g., family members, CPS worker, teacher, therapist, others identified by family). 39. Current information about the family (e.g., social history, behavioral and emotional status) is gathered prior to the meeting and shared at meeting (or beforehand). 40. All meeting participants introduce themselves (if applicable) or are introduced. 41. The family is informed that they may be observed during the meeting. 42. Plan of care is agreed on by all present at the meeting.

  42. Facilitator (6 items) 43. Facilitator makes the agenda of meeting clear to participants. 44. Facilitator reviews goals, objectives, interventions and/or progress of plan of care. 45. Facilitator directs (or reflects) team to discuss family/child strengths. 46. Facilitator directs (or reflects) team to revise/update plan of care. 47. Facilitator summarizes content of the meeting at the conclusion of the meeting. 48. Facilitator sets next meeting date/time.

  43. EMQ WOF-2 Results:Total Element Scores (N=30)

  44. Results of WOF • EMQ Sacramento shows higher scores on elements of Family-driven, Collaboration, Unconditional Care & Management of CFT Meetings • EMQ Sacramento continues to be challenged by lower scores on Community-based Resources, inclusive of natural supports • In contrast to the results of the WFI for Outcomes (high scores), the WOF showed low scores on this element • Slight increase in Total WFI scores since the implementation of the WOF

  45. Back to the WFI • Could the use of the WOF impact WFI scores? • A look at WFI scores across time

  46. WFI Total Scores Across Time WOF Implementation Begins 7/05

  47. WFI Respondent Scores Across Time High Scores on Voice and Choice, Cultural Competence, Individualized Services, Strengths, Outcome-Based Services Low Scores on Natural Supports, Community-Based Services, and CFT Element

  48. Challenges of the WOF • Limited resources in staff to become neutral observers • Managers wanted to use the WOF as a supervision tool • Willingness and comfort level of staff to participate in direct observation • Staff perception of fear of families being judged by observers • Sponsorship by management • Competing priorities in data collection

  49. Next Steps • Supervisors will continue to utilize the WOF to provide feedback • Operational realities limit the use of neutral observers • EMQ will no longer collect the WFI due to limitations in resources, the utility of the tool for feedback, and its relationship to outcomes • Continue to develop practice standards

  50. References • Achenbach, T. M., & Rescorla, L.A. (2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families. • Epstein, M.H., Nordess, P.D., & Hertzog, M. (2002). Wraparound Observation Form-Second Version. • Epstein, M.H., Nordess, P.D., et al. (2003). Assessing the Wraparound Process During Family Planning Meetings. • Hodges, K. (2000). Child and Adolescent Functional Assessment Scale (CAFAS). CAFAS Self-Training Manual and Blank Scoring Forms (2003). Ann Arbor, MI • Nordness, P.D., & Epstein, M.H.(2003). Reliability of the Wraparound Observation Form Second Version. • Suter, J., Force, M., Bruns, E., Leverentz-Brady, K., & Burchard, J. (2002). Manual for training, administration, and scoring of the WFI 3.0. Burlington: University of Vermont.

More Related