Appendix A. The Incredible Years: Parent, Child and Teacher Training Series

1. Appendix A.The Incredible Years: Parent, Child and Teacher Training Series

2. The Incredible Years • Award winning “exemplary best practices” program with demonstrated effectiveness. [Office of Juvenile Justice and Delinquency Prevention, Substance Abuse and Mental Health Services Administration] • Comprehensive—targets home, school and child. • Designed to prevent, reduce and treat conduct problems and to increase children’s social competence. • Parent skills training uses group discussion, videotape modeling, and homework exercises . • Child series (Dina Dinosaur) uses puppets and videotapes to teach social skills, conflict management, and successful classroom behavior.

3. The Incredible Years Outcomes Research indicates that children’s emotional, social and behavioral adjustment is as important for school success as cognitive and academic preparation. • Child program promotes children’s cognitive problem-solving strategies, increases use of pro-social skills and reduces conduct problems at home and school. • Parent program helps strengthen parenting skills, reduces parenting stress and increases parent’s involvement in school activities. • Teacher program strengthens classroom management skills, reduces classroom aggression and improves teachers’ ability to focus on student’s social, emotional and academic competence.

4. Incredible YearsAdherence to Model Fidelity CMHRC commitment to excellence, demonstrated through: • Training and certification of facilitators, dedicated FT coordinator for Incredible Years programming. • Purchase of all required curriculum and program materials. • Use of treatment fidelity groups, peer review, videotape feedback. • Continued use of ongoing training and technical support from the Incredible Years developers • Ongoing evaluation of parent and child groups.

5. CMHRC Goals for Incredible Years • Implement a comprehensive model that employs evidence-based multiple strategies for providing supportive environments and that accommodates ethnic, cultural and economic diversity. • Expand the target population to universal and earlier prevention—reaching children with certain family characteristics that put them at particular risk for developing conduct problems, delinquency and substance abuse. • Strengthen partnerships with local agencies that serve at-risk populations and, with the community at large, promoting aholistic view of health that includes physical and mental well-being.

6. The Incredible Years and Ben Franklin School • Partnership sponsored through a two-year grant from the Rochester Area Foundation First Steps Initiative • Implements the comprehensive model-teacher, child and parent. • First year, Franklin Kindergarten teachers and para-professionals receive extensive training and mentoring from certified Incredible Years trainers. • Second year, Franklin Kindergarten teachers implement independently with guidance and support from CMHRC IY staff. • Second year, Franklin 1st Grade teachers will receive comprehensive training and mentoring from CMHRC IY staff (like Kindergarten teachers this year).

7. Incredible YearsDina Dinosaur Prevention Why offer Dina Dinosaur in kindergarten classes at Ben Franklin school? • Social and emotional competencies are important predictors of school readiness and academic success. • Prevalence of aggressive behavior in early childhood 10% and may be as high as 25% for socio-economically deprived children. • Dina program cost effectively reaches high number of at-risk youth early in life when behavior is most malleable and before severe behavior problems arise.

8. Incredible YearsDina Dinosaur Prevention Goals for program at Ben Franklin [Kindergarten]: • Prevent early behavior problems from escalating. • Integrate children into peer group –classroom as “family”. • Improve social competence for entire class, provide common vocabulary and problem solving steps to use in everyday conflict. • Magnify dosage of IY intervention through teacher reinforcement of key concepts throughout the school day.

9. Incredible YearsDina Dinosaur Prevention - Children Implementation at Ben Franklin during School Year 2007-2008: • Offered 2 times a 6-day cycle in 5 kindergarten classrooms • Lessons over entire school year, seven units in sequential order [rules/behavior, feelings, problem solving, anger management, peer relations etc.] • Led by Children’s Mental Health Incredible Years staff with Ben Franklin teachers participating in planning and delivery of lessons

10. Incredible YearsTeacher Training at Ben Franklin • Teacher training at Ben Franklin during School Year 2007-2008 included: • Five full day interactive classroom management workshops offered throughout the school year, and 3 days of training from Seattle IY trainer on Dina Dinosaur curriculum. • Sessions on promoting pro-social behavior and school readiness, reducing non-cooperation and aggressive behaviors, and ways to collaborate with parents to support school involvement and school-home consistency. • Teachers implementing a classroom action plan with ongoing support from CMH IY staff.

11. Incredible YearsParent Training at Ben Franklin • Parent training at Ben Franklin during School Year 2007-2008 included: • A 12-week parent group with two CMH IY facilitators meeting once a week. Child care, supper and transportation offered to program participants. • Used a collaborative process to promote positive parent-child relationships through play, praise and attention. • Worked on effective limit-setting, use of incentives, building social competence and handling misbehavior. • All Kindergarten parents received detailed handouts on the Dina Dinosaur curriculum throughout the year and received tips on how to use the Dina techniques at home.

12. Evaluation of Incredible Years at Ben Franklin • Evaluation on all three components of the model, data collection includes: • Classroom Environment Scale [pre/post] • Social Competence Scale on Student Behavior [pre/post] • Teacher Classroom Strategies [pre/post] • Teacher Workshop Satisfaction Surveys • Parent Dina Dinosaur Satisfaction Surveys • Parent Stress Index [parent group, pre/post] • Eyberg Child Behavior Inventory [parent group, pre/post] • Parent Group Satisfaction Surveys

13. Classroom Environment Scale Teachers filled out the scale in October 2007 and again in May 2008. Instructions were to: Think about the general atmosphere of your classroom during the last month. Use the scale below where ‘10’ stands for always, ‘0’ stands for never and ‘5’ stands for half the time. 0.…1 .... 2….3.…4….5.…6….7.…8.…9….10 Rarely Sometimes Almost always

14. Classroom Environment ScaleOctober 2007 versus May 2008 Teachers rate their classrooms on scale of 1 through 10 for behaviors exhibited in past month (N=8 Classrooms) Q1. Students follow classroom rules Q2. Students work cooperatively (helping sharing, teamwork). Q3. Students attempt to problem solve when conflicts arise (talk about problem, discuss feelings, suggest solutions). Q4. Students manage anger (uses words not fists, takes deep breaths, remains calm). Almost Always Some- times Rarely Statistically significant = p≤.002 p≤.001 p≤.000 p≤001 Source: Classroom Environment Scale database

15. Classroom Environment ScaleOctober 2007 versus May 2008 Teachers rate their classrooms on scale of 1 through 10 for behaviors exhibited in past month (N=8 Classrooms) Q5. Students express feelings appropriately (positive or negative) Q6. Students handle transitions well. Q7. Students stay on task during class activities. Q8. Students show awareness and concern for individual differences Almost Always Some- times Rarely Statistically significant = p≤.000 p≤.002 p≤.003 p≤000 Source: Classroom Environment Scale database

16. Classroom Environment ScaleOctober 2007 versus May 2008 Teachers rate their confidence on six point scale: very unconfident, unconfident, somewhat unconfident, somewhat confident, confident, very confident (N=8 Classrooms) How confident are you in managing current behavior problems in your classroom? How confident are you in managing future behavior problems in your classroom? Source: Classroom Environment Scale database

17. Classroom Environment Scale By Franklin Program (AM, PM, Extended Day)October 2007 versus May 2008 Average TOTAL score on Classroom Environment Scale by Program (maximum =80 points) Source: Classroom Environment Scale database

18. Social Competence Scale – Teacher* • Teachers rate individual student behavior on 25 questions using scale: ‘not at all’, ‘a little’, ‘moderately well’, ‘well’ ‘very well’ . • Ratings occurred in October 2007 and again in May 2008. • Questions look at: • Emotional regulation • Problem solving skills • Attentiveness and initiation • N= 89 students • (17 students moved before post data was collected and are not included in the analysis.) * Tool used by Incredible Years Programs – Denver Colorado School District

19. Social Competence Scale TeacherEmotional Regulation October 2007 versus May 2008 Percentage who do “well” or “very well” N=89 students Source: Social Competence Scale Teacher database

20. Social Competence Scale TeacherSocial Skills/Peer Interaction October 2007 versus May 2008 Percentage who do “well” or “very well” N=89 students Source: Social Competence Scale Teacher database

21. Social Competence Scale TeacherAttentiveness/ Initiative October 2007 versus May 2008 Percentage who do “well” or “very well” N=89 students Source: Social Competence Scale Teacher database

22. Social Competence Total Score by Franklin Program (AM, PM, Extended Day)October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) N=89 students Source: Social Competence Scale Teacher database All statistically significant at p≤000

23. Social Competence Total Score by EthnicityOctober 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) N=42 N=8 N=6 N=10 N=20 1 Native American pre 75, post 125 All statistically significant at p≤01 Source: Social Competence Scale Teacher database

24. Social Competence Total Score by Gender / Ethnicity/ Disability: October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) N=47 N=51 N=6 N=42 N=83 N=38 All statistically significant at p≤01 Source: Social Competence Scale Teacher database

25. Social Competence Total Score by TeacherOctober 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) N=20 N=33 N=18 N=18 Source: Social Competence Scale Teacher database

26. Social Competence Total Score by Gender / Ethnicity/ Disability: October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) N=47 N=51 N=6 N=42 N=83 N=38 All statistically significant at p≤01 Source: Social Competence Scale Teacher database

27. Social Competence Total Score by EthnicityOctober 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) N=42 N=8 N=6 N=10 N=20 1 Native American pre 75, post 125 All statistically significant at p≤01 Source: Social Competence Scale Teacher database

28. Social Competence Total Score by Franklin Program (AM, PM, Extended Day)October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) N=89 students Source: Social Competence Scale Teacher database All statistically significant at p≤000

29. Social Competence Total Score by TeacherOctober 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) N=20 N=33 N=18 N=18 Source: Social Competence Scale Teacher database

30. Teacher Strategies Scale* • Teachers rate how often they use various teaching strategies on 34 questions using scale: ‘rarely/never’, ‘sometimes’, ‘half the time’, ‘often’ ‘very often’ . • Ratings occurred in October 2007 and again in May 2008. • Questions divided into five scales**: • Positive Approaches with Parents • Praise and Incentives • Proactive Strategies • Limit-Setting Strategies • Inappropriate Strategies (lower score better) • N= 5 teachers, parent scale =4 teachers *Tool used by Incredible Years Programs – Denver Colorado School District ** Incredible Years in process of completing psychometric analyses, currently using these scales

31. Teacher Strategies ScaleFall versus Spring Average total score on each subscale, n=5 (parent n=4) Increase in proactive strategies and decrease in inappropriate strategies significant at p≤05, increase in praise marginally significant p≤06 Source: Teacher Strategies database

32. Teacher Training: Satisfaction with Workshops 1-5 Ben Franklin Staff Only There were no “not helpful” answers Source: Teacher workshop satisfaction database

33. Teacher Training: Satisfaction with Workshops 1-5 Ben Franklin Staff Only Source: Teacher workshop satisfaction database There were no “neutral” or “not helpful” answers

34. Teacher Training: Satisfaction by Workshop Ben Franklin Staff Only Average score Scale: 1=not helpful, 2=neutral, 3=helpful, 4=very helpful Source: Teacher workshop satisfaction database

35. Parent Feedback Survey Dinosaur School Parents were asked to read a description of the Dinosaur School (puppets named Dina and Wally and the lessons being taught) They were then asked to think about the last three months and rate the following questions on a 4-point scale “very often”, “somewhat often”, “once in awhile”, “not at all”. • How often does your child talk about the things he or she is learning in Dinosaur School? • 2. How often have you noticed your child doing the following: • Talks about his/her feelings • Identifies / recognizes when there is a problem • Tries to solve problems when they come up (uses solutions/ideas from Dinosaur School) • Shares or helps friends/siblings

36. Tools Used to Measure Outcome: Pre and Post Parenting Stress Index [PSI]Paper/pencil assessment parent fills out to assess parent-child relationship (three subscales and total stress score) Pre and Post Eyberg Child Behavior Inventory [ECBI] Paper/pencil behavioral rating scale parent fills out to measure their perception of child conduct problems and the extent to which it is a problem for the parent. Parent BASIC Group Weekly parent group using Incredible Years curriculum to strengthen parenting skills and reduce stress. Nine parents completed [3 fathers, 4 mothers, 1 stepparent, 1 grandparent]

37. Parenting Stress Index Pre to Post Program Significant Drop in Percentage of Parents at a “Clinical Level” of Parental Stress (TSS*) Average stress level for the Ben Franklin parent group [beginning and end] is less than for the parent “treatment groups” held at the CMHRC. Average Entry Score – 81.2 (SD=8) Average Exit Score - 47.7 (SD=13) t{5}=6.5, significant at p<.001 N=6 with pre and post PSI* *Total Stress Score (TSS) is an indication of the stress level experienced within the role as parent. Clinical level is at or above the 90th percentile on PSI scale. Source: PSI / ECBI Database

38. Eyberg Child Behavior Inventory [ECBI] Pre to Post Program Significant Improvement in Behavior Intensity and Child Behavior Problems (as perceived by parents) Average ECBI levels for the Ben Franklin parent group [beginning and end] is lower than for the parent “treatment groups” held at the CMHRC. N=9 with pre and post ECBI INTENSITY Average Entry Score - 132 (SD=19) Average Exit Score - 89 (SD=8) t{8}=8, significant at p<.001 PROBLEM Average Entry Score – 12 (SD=10) Average Exit Score - 1 (SD=3) T{8}=3 , significant at p<.01 Source: PSI / ECBI Database

39. Parent Satisfaction with BASIC IY Parent Program, n=9 parents Problem(s) that originally prompted me to take this program for my child…. Use of methods to try and change child’s problems… My feelings about my child’s progress… Has helped with other personal/family problems… Source: Tally sheet for parent BASIC satisfaction surveys

40. Parent Satisfaction with BASIC Parent Program • Of 9 respondents: • All would “strongly recommend” the program to a friend or relative. • All felt “positive” [3] or “very positive” [6] about achieving their goal in the program for child and family. • All were “confident” [5] or “very confident” [4] they could manage current behavior problems. • All were “optimistic” [1] or “very optimistic” [8] about good results from the program. • All felt the parenting approach used to change child behavior was “appropriate” [1] or “greatly appropriate” [8]. Source: Tally sheet for parent BASIC satisfaction surveys

41. Parent Satisfaction with BASIC Parent Program Response to Facilitators Response to Group • All participants felt “very supported” [6] or “supported” [3] by the group. • Eight of nine program participants would like to keep meeting as a group. (The one participant who did not want to keep meeting was a grandparent). • Seven felt it was “likely”or “somewhat likely” they would continue meeting. The other two were “neutral” • Two facilitators: • I feel the leader’s teaching was… • The leader’s preparation was… N=9 responses Source: Tally sheet for parent BASIC satisfaction surveys

42. Parent Satisfaction with BASIC Parent Program • What did you see as the main benefit of the Incredible Years program? • I have become so much better with my children in all the ways that a father should be. I feel this will help me stay that #1 dad that I’ve always wanted. Thank you very much. • Positive talk was the most beneficial for me. Also the skills taught – how and when to ignore, using calming down techniques. • How easy it is to make small changes and have such wonderful results. When the program is broken down the way it was, everything makes so much sense. • Learning skills to improve communication, bringing parents into the children’s setting really helped my kids. They have been so proud and excited to have me here every Monday. • Consistency and reinforcing positive/expected behaviors.