National PBIS Leadership Forum October 3-4, 2019 Chicago, Illinois

1. National PBIS Leadership Forum October 3-4, 2019 Chicago, Illinois Session B – 9Tier 2 Groups for Students with Internalizing Mental health NeedsLucille Eber, Midwest PBIS NetworkJenniferMautone & RicardoEiraldi, Children’s Hospital of Philadelphia, University of PennsylvaniaBarry McCurdy, Devereux Center for Effective SchoolsKey Words: mental health, Tier 2, Applied Evaluation,

2. Advancing Education Effectiveness: Interconnecting School Mental Health (ISF) and School-Wide Positive Behavior Support (PBIS) Editors: Susan Barrett, Lucille Eber and Mark Weist pbis.org csmh.umaryland IDEA Partnership NASDSE

3. Where We’ve Been: • 2002-2007: Site Development with PBIS Expansion (informal and independent) • 2005: Community of Practice focus on integration of Positive Behavioral Interventions and Supports (PBIS) and School Mental Health (SMH) • 2008: ISF White Paper: formal partnership between PBIS and SMH • 2009- 2013 Monthly calls with implementation sites, national presentations (from sessions to strands) • 2009-2011 Grant Submissions • June 2012- September 2013 ISF Monograph & Monograph Advisory group • 2015: ISF Learning Community, SOC Webinar Series • 2016: Randomized Control Trial (RCT) Grant awarded • 2016: Targeted Work Group Webinars (8) • 2017: Targeted Work Group Webinars continuing, knowledge development sites across country • 2018: Continued webinars and knowledge development sites • 2018: Expanded Training/TA curriculum and workbook available online • 2019: Targeted Work Group (PLC) webinars

4. Where We* are Headed • National PBIS Center: Development of ISF Demonstration sites • Expanding on-line curriculum, resources, tools, • Volume 2 of ISF monograph: An Implementation Guide * OSEP’s National PBIS TA Center (PBIS V - 2018- 2024)

5. ISF Volume 2: An Implementation Guide Chapter 1: Context and Structure for Volume Chapter 2: Defining ISF: Origins, Critical Features, and Key Messages Chapter 3: Exploration and Adoption Chapter 4*: Installing ISF at the District and Community Level Chapter 5*: Installing ISF at the Building Level Chapter 6: Full Implementation and Sustainability * Includes Installation Guide

6. ISF District/Community Installation GuideISF V2 Chapter 4 (in press) Purpose:  This guide is intended to be used by facilitators and coaches to support District/Community Leadership Teams to install structures/systems needed to support an Interconnected System Framework (ISF). The goal is for teams to examine current system using installation activities and generate actions to move toward a more efficient and effective service delivery model.

7. ISF School Installation GuideISF V2 Chapter 5 (in press) Purpose: This guide is intended to be used by facilitators and coaches to support District/Community Leadership Teams to install structures/systems needed to support an Interconnected System Framework (ISF). The goal is for teams to examine current system using installation activities and generate actions to move toward a more efficient and effective service delivery model at the building level.

8. Partnerships are needed: • One in 5 youth have a MH “condition” • At least 50%, perhaps 80%, of those get no treatment • School is “defacto” MH provider • Juvenile Justice system is next level of system default • Suicide is 2nd leading cause of death among young adults • Factors that impact mental health occur‘round the clock’ • It is challenging for educators to address the factors beyond school • It is challenging for community providers to address the factors in school

9. Current National Data: Principals’ Concerns

10. ISF Defined • Structure and process for education and mental health systems to interact in most effective and efficient way. • Guided by key stakeholders in education and mental health/community systems • Who have the authority to reallocate resources, change role and function of staff, and change policy.

11. ISF Enhances MTSS Core Features • Effective teams that include community mental health providers • Data-based decision making that include school data beyond ODRs and community data • Formal processes for the selection & implementation of evidence-based practices (EBP) across tiers with team decision making • Early access through use of comprehensive screening, which includes internalizing and externalizing needs • Rigorous progress-monitoring for both fidelity & effectiveness of all interventions regardless of who delivers • Ongoing coaching at both the systems & practices level for both school and community employed professionals

12. Moving from co-located to a single system…….

15. TA Brief for AlignmentRDQ Brief as example with Mental Health Integration Technical Guide for Alignment of Initiatives, Programs and Practices in School Districts (OSEP Technical Assistance Center on PBIS, 2017) http://www.pbis.org/blueprintbriefstools Roundtable Dialogue: Aligning and Integrating Mental Health and PBIS to Build Priority for Wellness http://www.pbis.org/presentations/chicago-forum-17 Recorded webinar

16. ISF District/Community Installation GuideISF V2 Chapter 4 (in press) Purpose:  This guide is intended to be used by facilitators and coaches to support District/Community Leadership Teams to install structures/systems needed to support an Interconnected System Framework (ISF). The goal is for teams to examine current system using installation activities and generate actions to move toward a more efficient and effective service delivery model.

17. LOOK FOR LOTS OF OTHER SESSIONS ON ISF LISTED AS MENTAL HEALTH STRAND

18. ADDITIONAL RESOURCES www.midwestpbis.org **WATCH FOR ALL ISF RESOURCES TO ALSO BE AT THE NEWLY REFORMATTED PBIS.ORG

19. Effectiveness of Two Consultation strategies PBIS Implementation with Mental Health Support Jennifer Mautone, Ph.D., ABPP / Ricardo Eiraldi, Ph.D. Children’s Hospital of Philadelphia, University of Pennsylvania Barry McCurdy, Ph.D., NCSP, BCBA-D Devereux Center for Effective Schools 2019 PBIS Leadership Forum October 3rd & 4th, 2019

20. Project ACCESS Years 1 - 3

21. Project ACCESS – Study overview • Hybrid Implementation-effectiveness trial • Implementation of Tier 1 plus Tier 2 mental health interventions • Schools randomized to conditions • Consultation • Consultation Plus • Partnership between CHOP, Penn, Devereux CES, and the school district • Funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD)

22. Study Aims & Hypotheses • Examine differences in implementation outcomes • Schools in the C+ condition will have higher levels of implementation fidelity for Tier 1 and Tier 2 than those in the C condition • Examine differences in school and student level outcomes • Schools in the C+ condition will have lower levels of ODRs and better school climate than those in the C condition • Children attending schools in the C+ condition will have better outcomes than those in the C condition

23. Demographic Information • Schools • 6 schools (4 K-8, 1 K-4, 1 K-5) • 4,024 Students • 17.45% English Learners (range 12%-24%) • 15.06% Receiving Special Education (range 9%-22%) • 100% Free or reduced-price lunch • Racial/Ethnic Breakdown • 54.5% Latino (range 25%-77%) • 33.17% African American (range 14%-52%) • 10.17% Other ethnic minority (range 7%-14%) • 1.8% Caucasian

24. Study Design and Timeline Eiraldi, McCurdy, Khanna, Mautone, et al. (2014). Implementation Science, 9:12

25. Tier I Supports and outcomes Years 1 - 3

26. Training: Universal support • Product Development • Teaching matrix • Behavioral lesson plans • Recognition matrix • Flowchart for addressing problem behavior Year 1 • Day 1 • Context for problem behavior • History and components of PBIS • Behavioral expectations, setting specific rules • Day 2 • Teaching and encouraging behavior • Discouraging rule violations • Day 3 • Data-based decision-making • Team meetings • Roll out • Progress monitoring

27. Brief trainings Roll out • PBIS roll out checklist and action plan • Preparing faculty for roll out • Implementing roll out for students and families • Planning for post-roll out Standard Meeting Agenda • Adapted from Team Initiated Problem Solving • Standard structure for leadership team meetings • Concludes with team evaluation of meeting School-wide Information System • Trained administrator, LT data analyst, office clerk • Data entry and producing graphic displays

28. Training: Year 2 and After Team-Initiated Problem Solving • Meeting Foundations • Standard meeting agenda • Team roles (facilitator, minute taker, time keeper, data analyst) • Projected minutes • Problem Solving • Developing “precision” problem statements • “Seventh and eighth grade students are engaging in physical aggression in the hallway between 11:30 and 12:30 for peer attention at a rate of .25 incidents per day” • Identifying goals for change • Identifying solution actions • Implementing plans with contextual fit School-wide Interventions • 15 prevention, teaching and reward strategies • Precorrection, modeling, peer tutoring, group contingencies, tootling

29. Consultation • Consultation Plus • Coach attends leadership team meeting • Provides prompts • Makes recommendations • Shapes problem-solving skills • Coach provides performance feedback after meeting ends • Pre-meeting phone consultation (30 min – 1 hr.) • Masters-level consultant/coach • Coach responsibilities • Facilitator responsibilities • Leadership Team agenda • Prepared in advance and forwarded to coach • Includes section for problem-solving • Based on data analyst report • Leadership team meets following the conference call • Minutes forwarded to consultant/coach

30. Pre-meeting phone consultation Facilitator Activities Coach Activities • Forward agenda to coach • Items to be addressed • “To-do” items from last meeting • Previously defined problems in “precise” statement format • New problems based on SWIS data • Reviews agenda • Meeting logistics • Confirms scheduled meeting • Confirms minutes have been disseminated • Address questions/concerns for upcoming meeting • Reminds facilitator to forward minutes after meeting

31. Content and Process fidelity Content Fidelity • Extent to which meeting foundations were implemented • Roles of facilitator, minute taker, data analyst • Extent to which effective problem-solving occurred • Precise problem-statement format • Action solutions identified with implementation plan • Items scored Yes/No • Total score = percentage of items completed Process Fidelity • Capacity of team to effectively identify and solve problems • Ability of team to meet consensus • Scale of 1 to 5 (1 = Not at all; 5 = Very Often)

32. Tier I implementation outcomes Years 1-3

33. Fidelity: Leadership team meetings

34. Fidelity: set Teaching Expectations

35. SET: Mean Score

36. Tier I child & system outcomes Years 1-3

37. OFFICE DISCIPLINE REFERRALS: Average odrs per student

38. School Climate

39. Summary of Tier 1 Outcomes • Facilitators in Consultation Plus schools are less reliant on the pre-meeting conference call • Fidelity to the protocol for leadership team meetings is similar in Consultation-Only and Consultation Plus schools • SWPBIS fidelity, as measured with the SET, is higher for Consultation Plus schools • School climate, as measured with the CREST, improved slightly for Consultation-Only schools • ODRs per student in both conditions are similar to other district schools implementing PBIS

40. Tier 2 Supports and outcomes Years 1 - 3

41. Intervention Selection Tier 2 Interventions: Depression: PASCET Anxiety: FRIENDS for Life Disruptive Behavior: Coping Power Program • Pilot study funded by CDC • Needs assessment • Parent rating scales • Parent, teacher, student interviews/focus groups • Review of literature and databases • http://nrepp.samhsa.gov/landing.aspx • http://www.ebbp.org • Iterative process of adaptation • Focus groups, acceptability ratings • Pilot implementation to assess feasibility

42. Group Cognitive-Behavioral skills training Coping power Friends for life • Teach coping skills • Self-control • Perspective-taking • Problem-solving • 3-6 children per group • Fourteen 40-minute sessions • Education and skill building • Recognize signs of anxiety • Change negative – positive thoughts • Relax in tough situations • Exposure • Problem-solving • 5-6 children per group • Fourteen 40-minute sessions

43. Initial training • Two-day workshop • Screening / referral • Externalizing & internalizing disorders • Child group management • Behavioral and cognitive principles of behavior change • CPP & FRIENDS treatment theory & treatment manuals • Active learning strategies (e.g., role-play, modeling) • Training to criterion – Knowledge of Evidence-Based Services Questionnaire (KEBSQ) • Measuring fidelity

44. Annual Re-Training • Individual meetings with school staff • Tailored to staff member’s needs • More in-depth presentation for new staff members • Abbreviated for returning staff • Completion of knowledge assessment after training to verify readiness for implementation

45. consultation • Consultation Plus • Self-reflection • Goal setting • Video-based performance feedback • Well-executed activities • Areas for improvement • Discussion of referrals • Dealing with logistical problems • Session preparation • Foundations of CBT • Manual review • Materials Review

46. School Partner Participants N = 24 (87.5% female) 96% Masters degree (counseling, SW, education) Years of Experience M = 16.3 (1-41) Roles: Counselor/SW = 14 Teacher = 3 School Climate = 4 Administrator = 2 Missing = 1 Racial/Ethnic Background Hispanic – 41.7% Non-Hispanic Black – 45.8% Non-Hispanic White – 33.3% Mixed Race – 8.3% 20 School Partners conducted groups

47. Child participants Coping power friends • N = 193 children (179 received intervention) • Grades 3 – 8 • Race/Ethnicity • Latino – 54% • African American – 48% • White – 35% • Multi-Racial – 10% • Other – 7% • N = 101 children (94 received intervention) • Grades 3 – 8 • Race/Ethnicity • Latino – 50% • African American – 31% • Multi-Racial – 18% • Other – 2%

48. Content and Process fidelity Content Fidelity • Extent to which school partners covered content as prescribed in the manual • Items scored Yes/No • Total score = percentage of items completed Process Fidelity • Adapted from Lochman et al. (2009) • Extent to which intervention was delivered in an orderly fashion, using active learning strategies and relevant examples • 10 items; 2 factors (Active Engagement and Organized Teaching) • Scale of 1 to 5 (1 = Not at all; 5 = Very Often)

49. Results: Content Fidelity N=11 N=9

50. Results: Process Fidelity