Tools That Work: Improving Child Welfare Services Through Research, Performance Measurement, and Information Technology

1. Tools That Work: Improving Child Welfare Services Through Research, Performance Measurement, and Information TechnologyNovember 12, 2003 Best Practice Outcomes in Behavior Support and Intervention: A Multi-site Evaluation Lloyd Bullard Katie Johnson

2. Background • Physical Risks • Hartford Courant series documents 142 deaths related to the use of restraint and seclusion • Harvard Center for Risk Analysis estimates 50-150 deaths per year are related to the use of restraint and seclusion

3. Background • Psychological Risks • Consumers describe restraint and seclusion as dehumanizing and humiliating (Binder and McCoy, 1983) • Children and adolescents who have been restrained in psychiatric hospitals report nightmares, intrusive thoughts, avoidance responses, and marked startle responses (General Accounting Office, 1999). • Restraint and seclusion have been identified as having the negative neuro-biological effects associated with trauma and re-traumatization

4. Background • Charles Curie and the Pennsylvania Initiative • Children’s Health Act of 2000 • SAMHSA develops RFP • Three-year grant beginning October, 2001 • Selects five demonstration sites and one coordinating center

5. The Coordinating Center Partnership: CWLA and FFCMH • Child Welfare League of America • Establish an Advisory Committee • Provide Technical Assistance • Conduct Evaluation • Disseminate Findings • Develop Tools and Resources

6. The Coordinating Center Partnership: CWLA and FFCMH • Federation of Families for Children’s Mental Health • Provide Technical assistance to sites • Participate on the National Advisory Committee • Conduct key informant groups and provide comprehensive report of focus group activities • Establish Family Advocate National Advisory Committee

7. Demonstration Sites • The Methodist Home for Children and Youth: Macon, GA • Girls and Boys Town National Resource and Training Center: Boys Town, NE • A.B. and Jessie Polinsky Children’s Center: San Diego, CA • Lakeside Treatment and Learning Center: Kalamazoo, MI • University of Alabama Brewer Porch Children's Center: Tuscaloosa, AL • Connecticut Collaboration for Training Excellence • Klingberg Family Centers: New Britain, CT • Devereux Glenholme School: Washington, CT • Riverview Hospital: Middletown, CT

8. Project Goals • Year One: Identify best practices for reducing restraint and seclusion use • Year Two: Reinforce staff training with organizational support • Year Three: Disseminate findings

9. Evaluation Methodology • Individual Incident Data • Individual variables (e.g. race, gender, time of day, incident precipitant, etc.) recorded for each incident of emergency physical intervention • Standardized Measures • Measures (e.g. Organizational Climate, Family Centeredness Assessments, etc.) evaluated at specific time periods • Qualitative Measures • Reports to SAMHSA, significant events timelines & process meetings

10. Demonstration Sites Data collection worthiness - for the purposes of SAMHSA research Incident tracking Isolating intervention effects Over exposure Risk management Administratively burdensome Program rigidity Coordinating Center Uniformity Common definitions Common data points Common measures Complex design Multiple sites Multiple baselines Multiple interventions Multiple crises training modules (e.g., TCI, CPI, etc.) Isolating intervention effects Provide context to outcomes Challenges to Evaluation

11. Evaluation Tools • Key Informant Focus Groups • Organizational Climate Assessment • 14 dimensions that identify organizational factors affecting outcomes • Assessment of Policy and Procedure • Based upon Best Practice Guidelines For Behavior Management • Family Centeredness Assessment • Family-Professional Partnership Scale developed by University of Kansas • Trieschman Carolinas Project Instrument (TCPI) • Process Assessment and Feedback • Data Collection

12. Current Evaluation Outcomes • Key Informant Focus Group Findings • The seclusion and restraint process needs to be explained more thoroughly. • The training of staff to handle seclusion and restraint incidents is critical to avoiding injury. • Family members need to be a part of the service planning process. • Family members need to be notified in a timely way of any incidents. • Youth in facilities need to be able to file grievances.

13. Current Evaluation Outcomes • Organizational Climate Assessment • Staff from two sites successfully reducing restraint and seclusion were rated highly in the following categories: • Ability to get things done, meet the needs of their clientele, and use funds efficiently. • Ability to respond to change, flow of communication, compensation, competency, morale, personnel evaluation process, training, organizational values, and mission to serve youth. • Quality of relationship with supervisor and perceived standards of services rendered.

14. Current Evaluation Outcomes • Assessment of Policy and Procedure • Overall Policy Findings • The findings revealed that there was little difference between the site’s staff and the external site consultant’s assessment ratings. • What Sites are Doing Best • Consultants and site representatives rated the sites the highest on policies governing Legal and Ethical concerns and Professional Development. • What needs Improving • Representatives and consultants indicated complete agreement that the sites were relatively weakest on their written policies governing Medical Issues.

15. Current Evaluation Outcomes • Family Centeredness Assessment • Families indicate that it is “very important” to them that they have good working relationships with their service provider. • Families indicate that they are “satisfied” with their relationship with their child’s primary service provider.

16. Current Evaluation Outcomes • Process Assessment and Feedback • Sites attributed trends in physical intervention data to the following: • Temporal Factors • Policy and Practice Factors • Staff Factors • Child Factors • Training Factors

17. Current Evaluation Outcomes

18. Current Evaluation Outcomes

19. Current Evaluation Outcomes • Overall Practice Findings • The frequency of incidents largely coincides with seasonal events. • The use of seclusion and mechanical restraint has decreased. • The number of physical restraint incidents has increased sharply over the past six months. • Overall, injuries to children and staff have increased. • What Sites are Doing Best • Multiple preemptive behavior management techniques are attempted with the youngest children. • There has been no increase in medication management as reported by most sites. • The duration of seclusion and mechanical incidents is decreasing. • What Needs Improving • Property damage accounts for 20% of all behavior management incidents. • Injuries to staff and child occur more often in restraints that require moving the child. • Ethnic disparities exist in the use of seclusion and physical restraint.

20. Program Resources • CWLA Best Practice in Behavior Support and Intervention Assessment Instrument • Trieschman Carolinas Project Instrument: A Questionnaire For Family Centered Group Care Practice • Effective Supervisory Practice: Behavior Support and Intervention for Children and Youth • National Task Force on Behavior Support and Intervention Training Guidelines • Reducing the Use of Restraint and Seclusion: Promising Practices and Successful Strategies • Other Resources

21. Project Resources • CWLA Best Practice in Behavior Support and Intervention Assessment Instrument • Aims to help agencies improve their behavior support and intervention policies, procedures and practices through careful self-assessment • Example: • The agency has a philosophy governing the use of restrictive procedures. • Any behavior management intervention is first based on the premise of “first, do no harm.” • Nonphysical interventions are always the first choice, unless safety issues demand an immediate physical response. • Caregivers are involved in developing and/or selecting a behavior intervention model. • Caregivers support the philosophy governing the use of restrictive procedures as indicated by their knowledge of that philosophy, by their adherence to the philosophy in daily practice, and by their completion of all relevant documentation. • A written philosophy governing the use of restrictive procedures is present, and it is consistently reflected in all areas of agency practice. • A written philosophy governing the use of restrictive procedures is present, but only partially reflected in agency practice. • A written philosophy governing the use of restrictive procedures is present, but poorly reflected in agency practice. • A written philosophy governing the use of restrictive procedures is absent or clearly inadequate. • Not applicable.

22. Program Resources • Trieschman Carolinas Project Instrument: A Questionnaire For Family Centered Group Care Practice • A staff survey to assess an agency’s commitment to family-centered group care • Example: • To protect the interests of children they must remain connected to their parents regardless of circumstances. • Strongly agree Strongly disagree • (1) (2) (3) (4) (5)

23. Program Resources • Effective Supervisory Practice: Behavior Support and Intervention for Children and Youth • Synthesizes the best practices that supervisors can employ in order to reduce the use of restraint and seclusion • Competencies: • Supervisors understand that positive organizational climate is necessary to an effective behavior management program. • Supervisors know how to support staff in developing the skills and attitudes that are essential to successful implementation of an effective behavior management program. • Supervisors understand their role and essential skills and attitudes in successful implementation of an effective behavior management program.

24. Program Resources • National Task Force on Behavior Support and Intervention Training Guidelines • Creation of guidelines that will assess behavior support and intervention training programs. • Proposed Chapters: • Organizational Culture and Leadership • Behavior Support Training Programs • Risk Factors • Emergency Physical Interventions • Training Process

25. Program Resources • Reducing the Use of Restraint and Seclusion: Promising Practices and Successful Strategies • An issue brief that annotates policies and practices that successfully reduce the use of restraint and seclusion • Chapters • Leadership • Organizational Culture • Agencies’ policies, procedures and practices • Staff Training and Professional Development • Treatment Milieu • Continuous Quality Improvement

26. Program Resources • Website • Annotated Bibliography • Standards/Definitions for Restraint and Seclusion • Report Card Vols. 1 to Vol. 5 • CWLA Best Practice Guidelines for Behavior Management

27. Recommendations and Next Steps • Creation of a national resource center that • Acts as a clearinghouse that creates/disseminates evidenced-based practices • Compiles research and data on restraint and seclusion issues • Conducts trainings and provides technical assistance to states, providers, and consumers, and • Provides national database of restraint and seclusion incidents, program outcomes, etc.

28. Lloyd Bullard Project Director (202) 942-0280 Darren Fulmore Evaluation Coordinator (202) 639-4904 For Further Information Katie Johnson Research Assistant (202)639-4910 Coordinating Center for Behavior Support and Intervention behaviormanagementtraining@cwla.org or http://www.cwla.org/programs/behavior/