The York READ ing for ME aning Project

1. The York READing for MEaning Project A randomised controlled trial of three interventions designed to improve poor comprehenders reading comprehension skills Paula Clarke, Emma Truelove, Maggie Snowling, Charles Hulme Centre for Reading and Language, Department of Psychology

2. Poor comprehenders • Average word readers but poor at reading comprehension • 10% of primary school aged children (Nation & Snowling,1997) • Unnoticed in the classroom • Persistent difficulties • (Ehlich, Remond & Tardieu, 1999; Cain & Oakhill, 2006) - Listening Comprehension + - Decoding + The Simple Model of Reading (after Gough & Tunmer, 1986) Is there a core cognitive deficit in poor comprehenders? Are there a number of different underlying cognitive deficits which might cause a poor comprehender profile?

3. Poor comprehenders

4. Poor comprehenders However a key issue is that poor comprehender samples being examined across studies are not equivalent. Different selection criteria and selection measures have been used.

5. Why is an intervention study needed? • There is no consensus concerning the core cognitive deficit in poor comprehenders. • The majority of studies have focused on highly selected samples of children and have been small in scale. • Studies have used largely cross sectional, control group comparison or correlational designs so have not yet been able to demonstrate causal influences. • A large scale intervention study will allow us to: • Examine different poor comprehender profiles and subgroups. • Investigate which factors drive response to intervention and improvements in reading comprehension. • Create resources and training programmes for schools (if the interventions evaluated can be shown to be effective).

6. Previous research • Yuill and Oakhill (1988) - Inference Training • Oakhill & Patel (1991) - Mental Imagery Training • Yuill and Joscelyne (1988) - Story Structure & Inference Training • Johnson-Glenberg (2000) - Verbal vs. Visual RT Training • Interventions teaching specific components of reading comprehension have generated some impressive results in poor comprehenders. • Promising findings have not yet been replicated. • Studies have focused largely on text level and metacognitive skills. • No studies have pulled these components together to create a comprehensive reading comprehension training programme. • The only study to include an untreated control group did not use a RCT design.

7. Project aims Oral Language (OL) Text Comprehension (TC) Combined (COM) • To investigate three approaches to improving reading comprehension skills in poor comprehenders. • To compare these approaches to existing classroom practice by monitoring the performance of an untreated waiting control group. • To address the objectives of the primary framework (NLS) and equip teaching assistants with a wide range of skills and materials, useful in supporting children with reading comprehension and oral language difficulties.

8. The intervention programmes Oral Language Programme Vocabulary Listening Comprehension Figurative Language Spoken Narrative Text Comprehension Programme Metacognitive Strategies Reading Comprehension Inferencing from Text Written Narrative Combined Programme All eight components connecting oral language and text-based activities in an integrated and naturalistic approach. All sessions contained both reading and listening comprehension to support complementary components. Opportunities for children to encounter new vocabulary/idioms/inferences in both written and spoken language.

9. Session structure

10. TC programme contents • 1. Metacognitive Strategies (Cain, 1999) • Re-read (Garner, et al., 1984) • Look-back (Garner, 1982) • Think aloud (Farr & Connor, 2004) • Mental imagery (Oakhill & Patel, 1991) • Explain & reflect (McNamara, 2004) • 2. Reading Comprehension (Reciprocal Teaching - Palinscar & Brown,1984,1985, Palinscar,1986) • Clarification • Summarisation • Prediction • Question generation • 3. Inferencing from Text (Yuill & Oakhill, 1988) • Lexical inferencing • Bridging inferencing • Elaborative inferencing • Guessing missing information (Yuill & Joselyne, 1988) • Evaluative inferencing • 4. Written Narrative • Story structure (Beck & McKeown, 1981; Pearson, 1982; Idol & Croll, 1987) • Sequencing • Story production

11. OL programme contents • 1. Vocabulary (Multiple Context Learning, Beck, McKeown & Kucan, 2002) • Graphic Organisers (Nash & Snowling, 2006) • Verbal Reasoning • Mnemonic Strategies (Levin, 1993; Peters & Levin, 1986; Graves & Levin, 1989) • Illustrations • 2. Listening Comprehension (Reciprocal Teaching - Palinscar & Brown,1985; Palinscar,1986) • Clarification • Summarisation • Prediction • Question generation • 3. Figurative Language • Idioms (Legler, 1991) • Riddles (Yuill, 1988) • Jokes (Yuill, in press) • Similes and metaphors • 4. Spoken Narrative • Story structure (Beck & McKeown, 1981; Pearson, 1982; Idol & Croll, 1987) • Sequencing • Story production

12. Phases of the project Screening Intervention development T1 assessment T2 assessment T3 assessment T4 assessment Control Block 1 Control Block 2 Block 1 Block 2 Oct 2006 8-9 years Oct 2007 9-10 years Oct 2008 10-11 years Oct 2009

13. Participant flow Not meeting inclusion criteria (n = 634) Excluded (n = 45) 2 schools excluded due to low numbers of children showing poor comprehender profile Absent for key measures (n=13) Refused to be assessed (n = 2) Group Screening (Oct-Dec 2006) Y4 children in 23 schools in York & N.Yorks; eligible for assessment (n = 1120) Group assessments: Listening Comp (n = 1042); Ravens (n = 1054); Spelling (n = 1045); Numerical ops (n = 1050) Complete data on key measures (n= 977) Individual Screening (Jan-Feb 2007) 21 schools; eligible for assessment (n = 296) Individual assessments: NARA reading comp (n = 284); TOWRE (n = 282); WASI Verbal IQ (n = 277) Complete data on key measures (n= 282) Not meeting inclusion criteria (n = 114) Excluded on behavioural grounds (n =1) Declined to participate in intervention (n = 7) Selection & Pre-test (Feb-Apr 2007) 21 schools Selected for intervention (n =168)

14. Eligibility criteria Eligibility criteria • Discrepancy in standard score points between NARA II reading comprehension and TOWRE real word reading efficiency. • NARA II reading accuracy standard scores of 85 and above. • NARA II reading comprehension scores of 105 and below. • Age appropriate spelling ability. • Non-Verbal IQ of 85 or above. • Of these children, we selected eight children within each school with the greatest discrepancies. Sample size Power calculations for this study were carried out using estimates of effect sizes from previous studies of a similar nature.  A sample size of 40 children in each arm of the intervention provides 90% power to detect a difference of 0.6 standard deviations between the control group and the three intervention groups combined (with alpha = 0.05 two-tailed).

15. Participant flow Random allocation to intervention groups Selection & Pre-test (Feb-Apr 2007) 21 schools Selected for intervention (n =168) (Apr-Jul 2007) 20 schools Block 1 of intervention programmes (n = 157) Mid-Intervention Assessments (n = 159) OL programme Took part in block 1 of intervention (n = 39) TC programme Took part in block 1 of intervention (n = 40) COM programme Took part in block 1 of intervention (n = 39) Waiting Control Took part in block 1 of intervention (n = 39) Intervention: Excluded (n = 8) 1 school withdrawn from project Declined to participate in intervention (n = 1) Moved school (n = 2) OL programme Took part in block 2 of intervention (n = 38) Post-intervention assessments (n = 38) TC programme Took part in block 2 of intervention (n = 40) Post-intervention assessments (n = 40) COM programme Took part in block 2 of intervention (n = 38) Post-intervention assessments (n = 38) Waiting Control Took part in block 2 of intervention (n = 39) Post-intervention assessments (n = 39) (Sept-Dec 2007) Block 2 of intervention programmes (n = 155) Post-Intervention Assessment (n = 159) Intervention: Moved school (n = 2) Post-intervention Assessments (wave 1) (n=155) In new schools (n = 4)

16. Baseline characteristics of each group *Statistically significant between groups difference (p<0.05)

17. Intervention delivery • Two 10-week blocks of intensive teaching in individual and pair sessions. • Each session is 30 mins. • Children receive 2 pair sessions and 1 individual session per week (1½ hours per week). • Teaching took place in designated areas within school (small classrooms/meeting rooms etc.). • Teaching times varied depending upon existing timetabled commitments.

18. Treatment fidelity • Manuals Detailed, prescriptive manual and pre-prepared worksheets, readers and resources • Training Training took place over 3.5 days. Delivered by the research team. • Fortnightly tutorials Opportunity to monitor delivery of programmes by discussing experiences, ideas and observations. Some sessions took the form of top up training in which we focused on particular components of the programmes. • Observations Each TA was observed by a member of the research team at least twice in each intervention block. Careful records were kept and onsite feedback and support was given. • Filmed sessions Five TAs gave us permission to film teaching sessions.

19. Primary outcome measures • WIAT II Reading Comprehension • Children read (aloud or silently) a range of passages and sentences (narrative, adverts, non-fiction information etc.) Includes literal, inference and vocabulary dependent question types. • NARA II Reading Comprehension (Form 2 at pretest, Form 1 at post test) • Children read aloud short passages then answer a range of literal and inferential open ended questions. • TORCH Reading Comprehension • Silent reading. Comprehension assessed using a cloze procedure. Responses require a range of skills including inferencing and vocabulary knowledge.

20. Secondary measures

21. Data collection & analysis • T1 data was collected in school by the research team (blind) and the trained teaching assistants (blind at this point). • T2 & T3 data was collected by the research team (blind). • Testing conditions varied across schools. • Some assessments were individually administered, others were group administered. • All score sheets analysed blind. • 10% double marking for reliability where necessary. • Missing data estimated using data imputation. • Regression based approach used, controlling for performance at T1. • Report 95% robust confidence intervals.

22. Reading comprehension p<0.001 p<0.05 p<0.01 NARA II WIAT II

23. Listening comprehension CELF Listening to paragraphs

24. Vocabulary p<0.001 p<0.01 p<0.001 Taught Non taught

25. Verbal IQ p<0.05 WASI Vocabulary WASI Verbal Similarities p<0.05 WASI Verbal IQ

26. Points to consider • The groups do not significantly differ in terms of changes in performance on the control task (Numerical Operations). • Additional bespoke and standardised measures are currently being analysed (e.g. narrative, comprehension monitoring). • How do we interpret the improvements in verbal IQ? • Might certain core skills be better suited to different teaching approaches? • Further exploration of how the different standardised measures of comprehension correspond to the content of the programmes is required.

27. Conclusions • 20-week intervention programmes can produce significant gains in both expressive vocabulary and reading comprehension in poor comprehenders. • Importantly these gains are relative to an untreated waiting control group. • What’s next? • Further data collection at T4 to examine maintenance effects. • Statistical analyses to examine subgroups response to intervention. • Investigate whether expressive vocabulary improvement is a mediating factor in reading comprehension gains. • Development of waiting control group intervention.

28. Research Team Liaison Group Teaching Assistants Schools Children pjc118@york.ac.uk Contact: Many thanks to: