‘ POSITIVE CLASSROOMS, POSITIVE CHILDREN’ Evaluating the Incredible Years Teacher Classroom Management programme in I

1. ‘POSITIVE CLASSROOMS, POSITIVE CHILDREN’ Evaluating the Incredible Years Teacher Classroom Management programme in Ireland Dr Sinead McGilloway Principal Investigator Mental Health and Social Research Unit Department of Psychology NUI Maynooth

2. Overview • The Incredible Years Ireland Study: our role and who we are • Study background: - EBD in the classroom - research objectives/questions - study design • Outline of key findings • Conclusion

3. The research evaluation: Our role • To undertake a 4-year national community-based evaluation of the Incredible Years programme in Ireland: - Parent (BASIC) training - Teacher Classroom Management training** - Combined parent and small group (Dina) training • To work collaboratively with Archways and other community-based organisations and schools in executing the research • To generate high quality findings that will impact on policy and practice in Ireland (and elsewhere) -> improve the lives of children with EBD and their families

4. NUI Maynooth Dr Sinead McGilloway Lynda Hyland Dr Grainne NiMhaille Yvonne Leckey Paul Kelly Prof Donal O’Neill Mairead Furlong Church of Ireland College of Education Dr Anne Lodge Bangor University, Wales Dr Tracey Bywater Trinity College Dublin Prof Catherine Comiskey Queen’s University Belfast Dr Michael Donnelly The Research Team

5. EBD in the classroom • Child conduct problems in school academic under-achievement, poor attendance and early school leaving poor mental health, juvenile delinquency • Impact on the teacher’s ability to teach the class effectively • Negative impact on classroom environment and the teacher-child relationship • Can affect teacher job satisfaction, stress and likelihood of burnout • DES survey (2006): 32% of trainee teachers had fair or ineffective classroom management skills

6. The Incredible Years TCM programme • Social and behavioural learning principles • Help teachers promote pro-social behaviour • Reduce undesirable and aggressive behaviour • Establish more positive relationships with pupils and facilitate peer-to-peer bonding • Encourage teachers to collaborate with parents and promote parent involvement in school (Webster-Stratton, 2005) • The intervention • A brief group-based programme • Five one-day sessions (each month) • Videotape modelling, role plays and discussions

7. Research questions • Does the IY TCM programme improve classroom management competencies and provide skills to better manage challenging behaviour? • To what extent does the TCM programme improve child behaviour in the classroom? • How do outcomes for teachers and children change over time? • What are the experiences of teachers and which factors facilitate or inhibit the effective implementation of the programme? • How cost-effective is the programme?

8. Research Design • Three separate studies: • Main study: A clustered Randomised Control Trial (RCT) design (intervention and control group) • A smaller qualitative study assessing experiences and views of teachers who took part in the RCT • A cost analysis to assess the cost-effectiveness of the programme

9. Method 1. RCT • 22 teachers from 11 schools in Limerick area (445 children aged 4-7yrs from Junior and Senior Infant classes; 53% female) - Child behaviour (teacher & researcher-completed) - Teacher behaviour and self-report • Initial screening - low’, ‘medium’ and ‘high’ EBD (n=217) • Assessed at baseline (pre-intervention) and 6 months later 2. Qualitative sub-study • 1:1 interviewswith 11 teachers from 6 schools (baseline and 6-mth follow-up only) • Thematic analysis 3. Cost analysis • Detailed cost data on: (1) non-recurrent initial training costs of facilitators; (2) pre-delivery cost of programme; (3) group training costs; and (4) teacher substitution costs outcome data

10. Schools (n=12) in the greater Limerick area approached by Archways to participate in the research Nine schools provided informed consent Three schools were unable to participate (reluctant to participate [n=2] and time concerns [n=1]) Names of 2 other schools provided to the research team by Archways. Both consented to participate Total number of participating schools (n=11), teachers (n=22), children (n=445) All teachers completed and returned questionnaire measures T-SDQ child scores computed and index children identified. Parental informed consent obtained 234* index children were selected for observation from the initial sample of 445 on the basis of their total SDQ scores Randomisation process conducted; 11 intervention group teachers and 110 index children; 11 control group teachers and 107 children Baseline (T1) observations carried out in all classrooms (n=22) with all teachers (n=22) and index children (217) Intervention group teachers (n=11) receive TCM training (110 Index children) Follow-up (T2) assessments conducted immediately after TCM training completed (ie. 6 months after baseline assessment). * 17 children were lost to follow-up. RCT Flow Diagram

11. Mean SDQ ‘Total Difficulties’ scores across the sample range

12. Key findings:1. RCT • Baseline: 26% of children showed significant social, emotional and behavioural difficulties at baseline (SDQ) • Several statistically significant improvements in pre-post intervention teacher behaviour: - use of fewer negative classroom management strategies - give children more time to comply with questions/instructions - positive upward trend in positive classrm management strategies • Increase in the intervention group in self-reported frequency of use and perceived usefulness of positive classroom management strategies • All teachers reporting feeling more confident in their ability to manage conduct problems

13. Teacher ‘no opportunity’ for Intervention and Control Groups

14. Child outcomes • Children in intervention group: - showed significant reductions in emotional symptoms - positive trends with respect to overall behaviour • Those most ‘at risk’ showed most significant improvements: - overall behaviour - peer problems - general distress and social impairment

15. SDQ Total Difficulties score for ‘high risk’ children

16. 2. Qualitative study (Lynda Hyland) Theme 1: Reality of the classroom environment • All teachersreported significant behavioural challenges in their classrooms; some also reported physical and verbal aggression: “…he has thrown things behind his head and hit children with them. And he has even been violent towards adults as well too, children and adults… numerous teachers I’d say at this stage have gotten kicked and punched when he lashes out.” (T1) • Teacher stress: “Some days I went home and just cried, because I was just so exhausted and frustrated with the situation. Because, you know, I could see it in the other children, and I could see it in the other parents as well.” (T4) • Reluctance to seek support

17. Theme 2: TCM learning experience • Increased awareness: Course helped crystallise theoretical knowledge into conscious practical applications in the classroom • Experiential learning: A very positive aspect of the programme “ I’m delighted that I went on the training. As I said overall, you’re a way more confident teacher dealing with behaviour. You just have better and more resources and different things to draw upon.” (T11) • Role of the Group leaders: Building relationships and facilitating learning “… (The group leaders) were very, very good. They were very open and… they’d ask and they would try their best - even give you ideas to try this or that or try the other.” (T1) • ‘I am not alone’: Social support and the learning process

18. Theme 4: Impact on child behaviour - A ‘therapeutic milieu?’ A number of reported improvements in pupil behaviour: • overall conduct • staying more ‘on task’ - less distracted • fewer attention-seeking behaviours • better peer relations “They would be encouraging each other. They have started to get on better with each other and they’re co-operating amongst themselves even.” (T1) “…for the children…,you know who were doing well and who might often get overlooked, they were encouraged and you know, they were boosted and their behaviour improved as a result.” (T2)

19. Theme 5: The ‘positive classroom’ • Teachers reported having much more positive classroom experiences, and seeing the classroom as a happy place to work and learn • They reported feeling much less stressed and better equipped to deal with situations when they arise: “We’ve been having more and more really, really good days where they’ve been enjoying it and I’ve been enjoying it. And you know, they have been saying things like, ‘oh you should get a sticker too teacher’ …you were very good today’ and all this ‘kind of thing’. So I think it helps me personally feel like I can cope with anything now.” (T7) “ I think it’s good. I think it’s very good…I definitely would recommend people to do it” (T3) • All interviewees would highly recommend programme to another teaacher/school

20. 3. Cost Analysis (Prof.Donal O’Neill) • Total cost of delivery: €22,012.92 • Average cost/teacher: €2012.92 • Average cost/child: €100.65 (average class size=20) • Approx 1% of total expenditure per primary school child (Eurostat, 2007) • Combined with outcome (SDQ) data -> €52.97 per 1 point change in SDQ scores (ICER); e.g. it would cost approx €158.91 to bring the average highest scoring child in sample to within ‘normal’ range of behaviour • Much lower than other education-based interventions ($4863-$20,972 per child/youth)

21. Conclusion heooThe good nTeh ews… • Findings support those from elsewhere • Significant improvements in key areas: - Teacher skills/competencies in managing behavs - Child behaviour - Classroom atmosphere/ positive classroom envt • Programme was regarded as being very worthwhile • Other benefits: teacher stress, process of self-reflection, better home-school collaboration • Very modest costs • TCM programme is effective, cost-effective, highly acceptable to teachers, well suited to an Irish context and consistent with policy and practice guidance in Ireland

22. Thanks to… • All participating schools, their principals, teacher participants • The staff of Archways • The Atlantic Philanthropies & Dormant Accounts fund • The Expert Advisory Committee and Peer Reviewers • All members of the research team and especially the MHSRU team at NUI Maynooth www.iyirelandstudy.ieSinead.McGilloway@nuim.ie