Supporting Effective Instruction: Technology and Professional Development

1. Supporting Effective Instruction: Technology and Professional Development Carol McDonald Connor Frederick J. Morrison Christopher Schatschneider Barry Fishman

2. Coming to PD through the back door • Individualizing Student Instruction Study • RFT to test the impact of child characteristics-by-instruction interactions on student outcomes • Aptitude by treatment interactions

3. Designing the PD Protocol • Used the research base to develop a “state of the art” professional development protocol, • delivered to all treatment group teachers • We assumed that, with our coaching, the teachers would be able to implement ISI in the first experimental year with enough fidelity so that we could test our hypotheses • most PD research suggests that it takes 2 to 3 years for teachers to fully implement a new practice

4. What was the Intervention? • Created algorithms based on the HLM results in our Beyond the Reading Wars paper • Assessment to Instruction (A2i) software created to compute recommended amounts and types of instruction for each child • Embedded in planning software design to scaffold research-based reading practices

6. Multiple Dimensions of Instruction

8. Algorithm Results TM-CF TMCFa = ((End of Year Target - (.2* LW GE))/(.05 + (.05 * LW)))+13. TMCF_Recommended = (TMCFa - (.82 * Month )).

9. Algorithm Results for CM-MF CMMFa = ((3.76 – End of year target + (1.4 * Vocabulary AE))/(.30)) - 14. CMMFsl = 10-(.24*CMMFa). CMMF_Recommended= CMMFa + .5*(CMMFsl * Month).

14. PD Protocol • Focused on using A2i to plan instruction and then on implementing the recommended amounts for each child in the classroom • Grouping • small group rather than whole class • using A2i recommended groupings (homogeneous) • Classroom organization • Effective child-managed activities • Individualizing time, content, and delivery • no one way to individualize instruction • Research-based practice

15. Mentor or Coaching Model • “Research Partners” • classroom based – 2 hours bi-weekly • School level meetings • Individual meetings • Individualized • Same amount of time • Content and focus varied

16. Did the PD work? • Evidence from child outcomes • Evidence from A2i software tracking • Evidence from changes in teacher knowledge • Evidence from classroom observations

17. Evidence from Child OutcomesHLM - Treatment versus Control Student Reading Comprehension Outcomes Mean scores controlling for fall vocabulary, passage comprehension, letter-word reading, curriculum, FARL, and Reading First status. 464 = GE 1.8, 468 = GE 2.0, n = 616 students

18. Evidence from Software TrackingA2i Use and Reading Comprehension AE = 8.2 years AE = 6.0 years HLM fitted growth curves controlling for fall vocabulary, letter-word reading, curriculum, FARL, and Reading First status. 464 = GE 1.8, 468 = GE 2.0,

19. Evidence from changes in Teacher Knowledge • Teacher Knowledge Test Descriptives and Reliability • Assesses teachers’ understanding of English phonology, orthography, and morphology, and concepts of literacy acquisition and instruction • 34 multiple choice and 11 short answer items • Administered fall and spring • Cronbach’s alpha = .87. • Teachers in the treatment group had significantly higher spring TKS scores compared to control teachers • Controlling for school SES status, other PD opportunities, teacher credentials and fall TKS scores • standardize beta = .40 • A2i and TKS correlations • Fall TKS and A2i total use did not correlate • Spring TKS and A2i correlation = .58

20. Importance of Teacher Knowledge Scores on the TKS ranged from 9 to 36 (M = 23.45, SD = 7.27). First Graders – end of year grade equivalent score of 1.9 = 428

21. Evidence from Classroom Observations Child-managed Pair 4.1. Literacy Codes: 4.1.2. Phoneme Awareness 4.1.3. Syllable Awareness 4.1.4. Morpheme Awareness 4.1.5. Onset/Rime Awareness 4.1.6. Word ID/Decoding 4.1.7. Word ID/Encoding 4.1.8. Fluency 4.1.9. Print Concepts 4.1.10. Oral Language 4.1.11. Print Vocabulary 4.1.12. Reading Comprehension 4.1.13. Text Reading 4.1.14. Writing 4.1.15. Library 4.1.16. Assessment 4.1.2. Phoneme Awareness 4.1.2. Phoneme Awareness 4.1.2.2. Blending 4.1.2.3. Elision/Initial 4.1.2.4. Elision/Final 4.1.2.5. Elision/Vowel 4.1.2.6. Elision/Medial 4.1.2.7. Substitution/Initial 4.1.2.8. Substitution/Final 4.1.2.9. Substitution/Vowel 4.1.2.10 Substitution/Medial 4.1.2.11 Segmenting/Counting

22. TCM Small-group Code-focused

23. Teacher-Managed Instruction Small Group Whole Class

24. Child Managed Instruction

25. Winter Observed – A2i recommended amounts Distance From Recommendation Absolute Values * Simple Differences

26. Effect of Distance from Recommendations Winter TM-CF DFR Winter CM-MF DFR 450 = 1.9, 458 = 2.5 .6 GE or about a 5 and a half month difference in GE 464 = 1.8, 470 = 2.1 or about a 3 month difference in GE

27. HLM - DFR predicting student outcomes • Cumulative fall, winter and spring DFR for TM-CF and CM-MF • DFR Treatment teachers < DFR Control Teachers • Total Amounts of instruction did not predict student outcome growth (residualized change) • Cumulative TM-CF and CM-MF DFR negatively predicted both Passage Comprehension and Letter-Word Identification residualized change • TM-CF DFR amount and change fall to spring • CM-MF DFR amount

28. Discussion • Finding effective ways to change teacher practice quickly are critical • For Random Field Trials • Improving student learning • Why did our PD work? • Practice-based PD versus General Knowledge • Focus on A2i, which was designed to scaffold the kinds of instruction research suggests are more effective in improving students’ reading skills • Did the technology really make a difference? • We don’t know – and will need to conduct a RFT to find out