Literacy Coaching in Georgia Reading First

1. Literacy Coachingin Georgia Reading First Michael C. McKenna University of Virginia Sharon Walpole University of Delaware

2. Today’s Goals: • Examine evidence that coaching in Georgia Reading First has had an impact on student achievement. • Contrast the GARF coaching model with other models. • Consider how the GARF model might be modified in the years following RF funding to make coaching sustainable.

3. Does coaching really work?

4. That question is hard to answer because so many factors affect student achievement. However, you have helped us take the first steps toward answering it for GARF!

5. If coaching does work, then this is the way it works. • We know that some instructional methods are more effective than others. • We know that not all teachers employ these methods consistently and well. • We hypothesize that coaching can move teachers toward more research-based practice. • This change in the way teachers teach should result in higher achievement.

6. The coach’s impact on achievement is indirect. Coaching Altered Practice Increased Achievement

7. You helped us by completing 2 surveys: • A Coaching Observation Survey, which was completed by the regional specialists. This was our measure of coaching. • A Grade-Level Team Observation Survey, which was completed by coaches and regional specialists working together. This was our measure of instruction.

8. Our goal was to see if the survey results correlated with each other and with student achievement. To accomplish this goal, we first had to make sure the surveys were valid and reliable. We next had to make sure that the achievement data were accurate.

9. We investigated these questions: • What evidence exists to substantiate the validity of our coaching observation protocol? • What evidence exists to substantiate the validity of our teaching observation protocol? • Are specific aspects of coaching support associated with specific aspects of instruction? • Are specific aspects of coaching support associated with changes in oral reading fluency, listening vocabulary and listening comprehension? • Are specific aspects of instruction associated with changes in oral reading fluency, listening vocabulary and listening comprehension?

10. We narrowed our focus: • We looked only at 2nd grade for 2007-08. • We included 64 out of 117 schools, excluding those with mandates concerning differentiation and read-alouds. • We looked at DIBELS ORF from fall to spring. • We found 6,008 students with both scores. • We looked at CRCT from spring grade 1 to spring grade 2. • We found 6,180 students with both scores.

11. Research Question 1: What evidence exists to substantiate the validity of our coaching observation protocol?

12. We examined your responses to the survey to find out which of the items clustered together statistically. We could only guess at this in advance, but the results allowed us to form four clusters of related items. (Some of the items didn’t work and we had to get rid of them.) Each cluster is associated with a “factor.” We computed relationships between these factors and the other measures in our study.

13. The Coaching Observation Survey • Collaboration • Differentiation • Principal Support • District Support These portions of the final survey represent the four coaching factors.

14. The Coaching Observation Survey • Collaboration • Differentiation • Principal Support • District Support These portions of the final survey represent the four coaching factors.

16. The Coaching Observation Survey • Collaboration • Differentiation • Principal Support • District Support These portions of the final survey represent the four coaching factors.

18. The Coaching Observation Survey • Collaboration • Differentiation • Principal Support • District Support These portions of the final survey represent the four coaching factors.

20. The Coaching Observation Survey • Collaboration • Differentiation • Principal Support • District Support These portions of the final survey represent the four coaching factors.

22. Research Question 2: What evidence exists to substantiate the validity of our teaching observation protocol?

23. We did a similar analysis of the teaching survey and discovered three factors related to instruction.

24. The Grade-Level Team Survey • Effective Instruction • Read-Alouds • Use of Formative Assessment These portions of the final survey represent the three instructional factors.

25. The Grade-Level Team Survey • Effective Instruction • Read-Alouds • Use of Formative Assessment These portions of the final survey represent the three instructional factors.

27. The Grade-Level Team Survey • Effective Instruction • Read-Alouds • Use of Formative Assessment These portions of the final survey represent the three instructional factors.

29. The Grade-Level Team Survey • Effective Instruction • Read-Alouds • Use of Formative Assessment These portions of the final survey represent the three instructional factors.

31. Summary of Survey Validation • We used factor analysis to keep some items and get rid of others. • Several logical clusters of items were found. • These became “subscales” (portions of the overall surveys) that we used to test the last three questions. • Reliability of all subscales of both surveys was good.

32. Research Question 3: Are specific aspects of coaching support associated with specific aspects of instruction?

33. Here we examined the first link. Coaching Altered Practice Increased Achievement ?

34. Here’s what we found: • Principal support for the coach was related to Read-Alouds and Formative Assessment. Coaching Collaboration Differentiation Principal Support District Support Instruction Effective Instruction Read-Alouds Formative Assessment

35. Here’s what we found: • Coaching for Differentiation was associated with Formative Assessment. Coaching Collaboration Differentiation Principal Support District Support Instruction Effective Instruction Read-Alouds Formative Assessment

36. Here’s what we found: • Coaching for Collaboration was associated with all three dimensions of instruction. Coaching Collaboration Differentiation Principal Support District Support Instruction Effective Instruction Read-Alouds Formative Assessment

37. Here’s what we found: • District support for the coach was unrelated to instruction. Coaching Collaboration Differentiation Principal Support District Support Instruction Effective Instruction Read-Alouds Formative Assessment

38. Research Question 4: Are specific aspects of coaching support associated with changes in oral reading fluency, listening vocabulary and listening comprehension?

39. Here we examined the long-range link. Coaching Increased Achievement ?

40. Here’s what we found: • Coaching for Differentiation was associated with growth in vocabulary and listening comprehension, as measured by the CRCT. Coaching Collaboration Differentiation Principal Support District Support Achievement CRCT DIBELS ORF

41. Here’s what we found: • Coaching for Collaboration was associated with fluency growth. Coaching Collaboration Differentiation Principal Support District Support Achievement CRCT DIBELS ORF

42. Here’s what we found: • Principal Support was associated with fluency growth. Coaching Collaboration Differentiation Principal Support District Support Achievement CRCT DIBELS ORF

43. Research Question 5: Are specific aspects of instruction associated with changes in oral reading fluency, listening vocabulary and listening comprehension?

44. Here we examined the last link. Coaching Altered Practice Increased Achievement ?

45. Here’s what we found: • All three dimensions of instruction were associated with vocabulary and comprehension growth on the CRCT. Instruction Effective Instruction Read-Alouds Formative Assessment Achievement CRCT DIBELS ORF

46. Here’s what we found: • The use of Formative Assessments was associated with the number of students scoring at the Advanced Level on the CRCT. Instruction Effective Instruction Read-Alouds Formative Assessment Achievement CRCT DIBELS ORF

47. Here’s what we found: • Effective Instruction and Formative Assessments were associated with fluency growth, but Read-Alouds were not. Instruction Effective Instruction Read-Alouds Formative Assessment Achievement CRCT DIBELS ORF

48. What does it all mean? • These results are very encouraging! • They indicate that coaching, instruction, and achievement growth are related in ways that make sense. • They help to validate the hard work you have done. • Although these correlations do not prove causal relationships (only a true experiment can do that), they are a critical first step.