Carole Upshur, EdD., Professor Department of Family Medicine and Community Health

1. What Works? Effective Early Childhood Mental Health ConsultationGeorgetown Center for Child and Human Development WebinarJuly 28, 2010 Carole Upshur, EdD., Professor Department of Family Medicine and Community Health Carole.upshur@umassmed.edu Melodie Wenz Gross, PhD, Assistant Research Professor Department of Psychiatry University of Massachusetts Medical School Melodie.wenzgross@umassmed.edu

2. Addressing Behavior Problems in Preschool Settings National Institute of Mental Health R34 MH073830

3. Study Contributors • Together for Kids Coalition • Health Foundation of Central Mass • United Way of Central Mass • University of Massachusetts Zuckerberg Leadership Fund • Lynn Hennigan, Director of Services for the Young Child, Community Healthlink • Rainbow Child Development Center • Worcester Comprehensive Child Care Services-Great Brook Valley • YMCA of Central Mass, Worcester Commons • YWCA of Central Mass, Salem Square

4. Study Staff • Carole Upshur, EdD, Professor, Department of Family Medicine and Community Health, UMMS, PI • Melodie Wenz Gross, PhD, Assistant Research Professor, Department of Psychiatry, UMMS, Co-Investigator • George Reed, PhD, Associate Professor, Biostatistics Unit, Division of Preventive and Behavioral Medicine, UMMS • Judy Davis, MPH, and Gail Sawosik, Project Coordinators • Rachel D’Angelo, Data Entry • Katelyn Allen, Katie Bateman, Courtney Carpenter, Alicia Derry, Rachel Friendly, Jenny Hazelton, Karen Nideur, and Laura Seekamp, Observational staff • Esteban Cardemil, PhD, Clark University, Stephanie M Jones, PhD, Harvard University, Daniel Shaw, PhD, University of Pittsburgh, Scientific Advisory Committee

5. Background • 2001-current: work of TFK coalition to develop and provide mental health consultation services to child care centers • This work found: • Great interest in child care sites in receiving help to work with young children’s disruptive behavior • Ability of centers to conduct regular screening for behavior problems and work collaboratively with TFK to deliver teacher mental health consultation and parent and child services • Successful outcomes in reducing behavior problems and increasing developmental and school readiness skills among children and families provided services

7. Kindergarten Follow-up 2009- Services

8. Child Behavior Problems Rated by Preschool and Kindergarten Teachers-2 Waves of Kindergarten Follow-Up (Sutter Eyberg score)

9. Challenges of Mental Health Consultation Model • Many teachers want to leave the problem to the ‘expert’; we found some teachers did not embrace new learning about behavioral management but thought consultant would ‘fix’ the child • Over several years of mental health consultation we did not find significant changes in teacher self-reported confidence in knowing how to handle challenging behaviors • Some teachers with children with challenging behaviors did not refer children because parents were reluctant to participate in MH consultation • Consultants could not provide individual services to all children identified in the course of the year

10. Rates of children/1000 screening above cut off for problem behavior; and rates of suspension/termination between 2003 and 2008

11. Goal of Introducing & Piloting a Classroom Curriculum • Review and select/adapt from several already developed models (3P, Incredible Years) a primary prevention approach • Goal: supplement mental health consultation to: • Teach all children social skills and emotional regulation • Serve as early intervention and prevention of high rates of problem behavior • Reduce the numbers of children that require referral for mental health consultation • Improve teacher skills in managing behavior • Not require stigmatization of child/family for children to receive some needed intervention

12. Second Step Curriculum • Group social skills curriculum for preK-1st grade • Taught by teachers (not counseling or outside staff) • Picture cards, stories, puppets, CD with songs, supplemental story books • Teaches recognizing feelings in self and others and labeling them • Learning to calm down when upset • Learning how to problem solve: • Share toys, join a group, wait politely, avoid distraction etc.

13. Why Second Step? • Feasible cost to implement • Kits cost $289 per classroom • Training of trainers 2-day session per person costs $525 • Level of overall training required modest compared to some other programs • Accessible train the trainer workshops around the country • Only a 2-4 hours required to get teachers started • Teacher requirements and time modest • Only a few minutes needed to prepare a lesson • Lessons can be broken down and delivered in short periods of time that can fit with other curriculum requirements • Pictures in the materials were very representative of the multicultural background of urban preschool children

14. Adaptations • Curriculum adapted from 1x per week and 45 minute lessons to 4x per week and 15 minute lessons=89 lesson days • Sessions repeated as needed • Meet children where they are (e.g., give out small paper plates to draw happy and sad faces on, create personalized mats for children to sit on for the lesson • Reward children frequently during the session for paying attention; added durable wooden heart rewards to be used all day • 4-6 parent group sessions were offered each year, adapted from Second Step parent training; parent take home letters were adapted/simplified; additional parent handouts provided

15. Methods • 4 programs with 15 classrooms randomly assigned to intervention (2) or control group (2) • All sites (intervention and control) continued to receive Mental Health Consultation through TFK • Teachers were trained in an initial 2 hour dinner meeting and then monthly 90 minute dinner meetings the first year; alternate months the second year • Mental health consultants received individual training on the curriculum, attended some teacher sessions, and periodically consulted with the research staff about matching curriculum topics to individual treatment plans for identified/referred children • Mental health consultants co-led the parent sessions in Year 2

16. Implementation Challenges • Activities and language in curriculum too advanced for younger children/those with developmental delay/ESL • Children could not sit quietly for any length of time and had never been asked to raise their hand in a group circle activity • During implementation, teacher turn over was 25% and 37.5% in the two centers in Year 1 and 75% and 25% in Year 2 • Children coming and going all year; less than 50% received most of the curriculum • All centers had new state curriculum requirements and accreditation visits during Year 1 • Few parents attended parent sessions designed to link classroom activities to home activities

17. How Successful Was the Implementation? • Of 25 total lessons (broken into the 89 delivery days), teachers completed: • a mean of 22 lessons in Year 1 (range=17 to 25) and • a mean of 23 lessons in Year 2 (range=11 to 25); (only 1 classroom completed less than 24) • Teachers reported: • 85% completion of main curriculum points in Year 1 (range = 62% to 99%) and • 87% (range = 49% to 99%) in Year 2 • Mean observed fidelity ratings on a five point scale (1= ‘not at all’ to 5 = ‘very much’) were: • 3.6 in Year 1 (range 3.11 to 4.25) and • 3.5 in Year 2 (range 2.90 to 3.93)

18. Teacher Reports of How Hard it Was to Implement

21. Percent of Parents Reporting Participation

22. What was Related to Greater Implementation Success? Teacher Characteristics: • Higher baseline teacher personal accomplishment self-rating • Less teaching experience; Greater teacher education • Greater baseline teacher observed skills • Greater attendance at Second Step trainings • Higher self-efficacy in delivering the lessons Classroom characteristics: • Having more children with behavioral or other problems in the classroom related to completing fewer lessons, but with greater fidelity • Lower child and teacher turnover • Higher quality baseline classroom environment and interactions

23. Did It Work? • Was Second Step effective in changing the classroom environment? • Did it help to improve… • teacher skills? • teacher and child interactions? • child behavior? • child social skills?

24. Outcome Measures • Observed quality of classroom interactions (ECERS Interaction scale: teacher discipline, supervision, teacher-child interactions, child-child interactions) • Observed individual teacher skills (Caregiver Interaction Scale summary score) • Observed child behavior (classroom level behavioral counts of disruptive behavior, verbal aggression and physical aggression) • Teacher ratings of child behavior problems(Sutter-Eyberg) • Teacher ratings of child adaptive/prosocial skills(Adaptive Behavior Scale)

25. Preliminary findings • Teacher skills • ECERS Interaction Quality Discipline General supervision Staff-child interaction • Observed ratings of classroom disruptive behavior by children

26. Baseline Classroom Characteristics

27. Differences Between Intervention and Control Centers in Years 1 & 2 • No significant differences between intervention and control in any outcomes in Year 1 • In Year 2, Intervention Centers improved whereas Control Centers got worse in: • Observed teacher skills • Observed ECERS Interaction scale (including discipline, general supervision and staff-child interaction) • Observed overall child behavior problems

28. CIS: Observed Teacher Skills Year 1 Year 2

29. ECERS Interaction Scale: Observed Classroom Interaction Summary Score Year 1 Year 2

30. ECERS Interaction Scale: Observed Discipline Year 1 Year 2

31. ECERS Interaction Scale: Observed General Supervision Year 1 Year 2

32. ECERS Interaction Scale: Observed Staff-Child Interactions Year 1 Year 2

33. Observed Classroom Behavior Problems: Total Behavior Intervals Year 1 Year 2

34. Percent of Parents Reporting Child Talks About Feelings

35. Percent of Parents Reporting Change in Their Child’s Ability to Cooperate After Second Step

36. Conclusions: Implementation • High implementation: 88% lessons in Year 1& 92% in Year 2 • No teachers rated that it “Takes too much time” or “Was too hard” • Positive teacher ratings on helpfulness to children, themselves and classroom • Evidence for sustainability: 62% said they would “definitely” and another23% would “probably” use the curriculum after the project ended • Teacher experience, skills, attitudes, classroom quality, teacher and child turnover, and number of children with behavioral problems and delays affect implementation • Mental health consultants felt the curriculum was helpful to reinforce individual child work • The majority of parents reported that they received letters about Second Step and tried the suggested activities at home

37. Conclusions: Outcomes • Teacher observed skills, and classroom interactions seems to improve in intervention versus control classrooms, more strongly so in the second year of implementation • Mental health consultants felt the curriculum helped their work with individual children • In Year 2 mental health consultants were able to open more consultation only case (as opposed to individual therapy) and caught up with the number of referred children; some indication that the curriculum allowed them to be more effective with cases and handle more cases • Observed child behavior got worse in all classrooms in Year 1, but seemed to show improvement for the intervention sites in Year 2, while child behavior in control classrooms got worse • The majority of parents reported that their child talked about feelings frequently and in Year 2, a majority of parents reported that their child’s ability to cooperate had improved

38. Resources • www.cfchildren.org for Second Step materials and training • http://www.americanwoodcrafterssupply.com For wooden hearts-1 ¾” • http://www.hfcm.org/?id=288 • Reports of the results of the TFK mental health consultation development project • http://www.sciencedirect.com/science For copies of the article: Upshur, Wenz Gross & Reed (2009). A pilot study of early childhood mental health consultation for children with behavior problems in preschool. Early Childhood Research Quarterly, 24: 29-45.