Natasha Mitchell, Research Fellow Hannah Buckley, Statistician

1. A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah Buckley, Statistician York Trials Unit, University of York

2. Co-authors: Ainsworth, H., Heaps, C., Hewitt, C., Jefferson, L., Torgerson, C., & Torgerson, D. Funded by: Education Endowment Foundation Intervention delivery partner: Calderdale Excellence Partnership

3. Protocol available at: http://educationendowmentfoundation.org.uk/uploads/pdf/Transitions_-_Calderdale.pdf • Full report available at: http://educationendowmentfoundation.org.uk/uploads/pdf/EEF_Evaluation_Report_-_Improving_Writing_Quality_-_May_2014_v2.pdf

4. Overview • EEF Transitions Round: • Evaluate 3 different writing interventions with aim to improve children’s writing skills during transition from primary school to secondary school • Individually Randomised (Discover) • Split Plot Design (Exeter) • Cluster Randomised (Calderdale)

5. Background • Many children leave primary school without achieving a Level 4 or above in writing (Dept of Education, 2013) • The intervention was aimed particularly at children: • from disadvantaged backgrounds • not achieving a ‘secure Level 4’ in year 6 • Not reaching Level 4 at the end of KS2 linked to progress in secondary school (Dept of Education, 2014)

6. Background • Using memorable experiences and the Self-Regulated Strategy Development (SRSD) programme. • SRSD strategy developed in America • Writing process model involves: • Plan • Draft • Edit • Revise • Key aspects include self monitoring & goal setting to provide pupils with ownership over improving their writing.

7. Research Question What is the effectiveness of the Improving Writing Quality programme compared with “business as usual” on the writing skills of participating children?

8. Design • Pragmatic cluster randomised controlled trial. • Approximately 24 schools (feeding into 3 secondary schools) randomised on a 1:1 ratio to intervention and control groups

9. Design Intervention Group • The intervention was offered to all pupils in Year 6 in 2013. • Pupils predicted to achieve Level 3 or an insecure Level 4 in English (based on teacher assessment) continued to receive intervention in Year 7 (Autumn term). • Memorable experiences for pupils in Year 6. • Pupils used SRSD programme to embed these experiences into their writing. • Professional development for primary and secondary school teachers in key elements of the writing intervention SRSD programme (including discussion, modelling and planning). Control Group • Business as Usual in 2013. • Primary schools trained in SRSD in 2014.

10. Design • Hosted a information meeting to explain intervention and evaluation, hosted by delivery partner and evaluators in February 2013. • Recruited: 23 primary schools; 3 secondary schools. • Randomised primary schools in March 2013. • Post test conducted in December 2013 after transition to secondary school.

11. Outcomes Outcome measure • Progress in English (PiE) 11: Second Edition Long Form (LF) Test, GL Assessment Primary outcome • Combined score on extended writing tasks Secondary outcome • Combined score on reading tasks and combined score on the spelling and grammar tasks

12. Analysis Methods • Cluster trial • Pupils from same primary school transitioning to different secondary schools • Cross-classified multilevel model used to account for this • Adjustment was made for baseline predicted KS2 writing level alongside other predictors of attainment(gender, FSM status, EAL status, month of birth) • ITT

13. Results • 209 pupils meeting inclusion criteria randomised to control and 223 to intervention • Following transition 130 control and 154 intervention pupils • Primary analysis conducted on 119 control and 142 intervention pupils

14. Results • A significant increase of 2.53 marks in writing score (p=0.002, ES=0.74) was found for struggling writers in the intervention group compared with the control group. • Although this increase was larger when analysis was conducted on the FSM subgroup, there was no significant difference between FSM and non-FSM children. • There was no evidence of a difference between the randomised groups in reading or spelling and grammar score.

15. Strengths & Limitations Strengths • Study conducted to CONSORT standards. • Used blind marking of test papers. • Good relationship between CEP & schools therefore had high completion. • Analysis plan produced a priori. • Analysis accounted for clustering. • Followed principles of intention to treat.

16. Strengths & Limitations Limitations • During the trial there was ‘intervention development’. • Anglicising the intervention. • Refresher training to secondary schools incorporated ‘lessons learnt’ and adaptations. • Imperfect implementation in secondary schools. • Based in single geographical area.

17. Conclusions • This intervention appears to be highly effective at improving children’s writing skills despite being used for a relatively short period of time. • No statistically significant difference found between FSM and non-FSM children.

18. What next? • Larger-scale effectiveness RCT of the intervention to confirm findings of this trial. • Multiple geographical areas to be included. • Developing an ‘Anglicised’ version for use in primary schools.

19. Thank you Questions?