Evaluation of South Carolina’s Reading First (SCRF) Initiative

1. Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational Research Association’s Annual Meeting March 27, 2008

2. Tammiee S. Dickenson, PhD Robert L. Johnson, PhD Heather L. Bennett, MSW Katie A. Sesso-Dahlke, MSW Brett Ermer Mugdha Galande Joanna Gilmore Jessalyn Smith Diane M. Monrad, PhD Sarah J. Gareau, MEd Jennifer Gay Diana Mindrila Anita Rawls Pam Wills Patricia Branham Becca Driggers Diane Dunham, EdS

3. Overview • SCRF evaluation • Data sources • Achievement data • Participant group survey results • Conclusions

4. SCRF Evaluation

5. Participants • 2004-2005: served 52 schools in 24 districts (approximately 12,000 students) • 2005-2006: served 51 schools in 23 districts (approximately 11,000 students) • 2006-2007: served 49 schools in 23 districts (approximately 10,500 students)

6. Data Sources • Stanford Reading First assessment • Palmetto Achievement Challenge Test (PACT) • Participant group surveys • School Leadership Team (SLT) surveys • Professional development workshop surveys

7. Annual SLT Presentation • Results presented from previous year • Stanford Reading First assessment results • Cohort analysis of all students in previous year • Matched analysis of students in all years • Individual school reports provided • Participant group survey results • Formative feedback for program improvement

8. Stanford Reading First Assessment • Measures achievement in grades 1 through 3 • Sections: • Multiple choice (components – phonemic awareness; phonics; vocabulary development; reading fluency; and reading comprehension strategies) • Oral fluency (components – speaking vocabulary and oral reading fluency)

9. Results: At Grade Level (AGL) Grade 1 Percentage of SCRF Grade 1 Students Scoring At Grade Level (AGL) on Stanford Reading First and Annual Gains

10. Results: At Grade Level (AGL) Grade 2 Percentage of SCRF Grade 2 Students Scoring At Grade Level (AGL) on Stanford Reading First and Annual Gains

11. Results: At Grade Level (AGL) Grade 3 Percentage of SCRF Grade 3 Students Scoring At Grade Level (AGL) on Stanford Reading First and Annual Gains

12. Average NCE Scores on Stanford Reading First for SCRF Students with AGL Reference Line

13. Average Scale Scores on Stanford Reading First for SCRF Students and the Norm Group

14. Participant Group Surveys • Respondent groups: interventionists, literacy coaches, principals, and teachers • Sections: implementation, level of support, roles and responsibilities, professional development, and program effectiveness • Comparison across years (2005-2006 compared to 2006-2007)

15. Results: Participant Group Surveys • Better understanding of program goals • Reported a high rate of involvement in professional development activities and found those activities helpful • Decrease in the need for professional development about the program and use of assessments • Increased need for professional development on effective instructional strategies to use for students below grade level

16. Implementation Survey Items

17. Implementation Survey Items (Cont.)

18. Describe One Benefit of the SCRF Initiative • Commonly cited benefits across groups Focus on assessment • Increased awareness of students’ needs/strengths/weaknesses • Progress monitoring; Dominie • Improved use of assessments; use of assessment data to guide instruction Resources • Additional books and materials • Extra funding/money Professional development; study groups Increased collaboration Learned new strategies

19. Results: Participant Recommendations

20. Results: Effectiveness Ratings

21. Conclusions: Achievement • Students performed better on the Stanford Reading First assessment in the 2006-2007 school year as compared with the 2005-2006 school year. • Students’ scores on the Stanford Reading First assessment improved from fall to spring in all three years for all grade levels. • The largest gains were made by students in grade 1 and the lowest gains occurred in grade 2 in 2006-2007, which is also consistent with the last two years.

22. Conclusions: Participant Groups • Program participants have a better understanding of program goals. • Participant groups find professional development useful. • Over 80% of each participant group rated the initiative as either effective or very effective in Years 2 and 3.

23. Recommendations • More time for sustained reading for students in grade 2 • Additional strategies for working with students below grade level • Participant groups’ suggestions regarding study groups

24. Contact Information South Carolina Educational Policy Center University of South Carolina, College of Education Wardlaw Suite 010, Columbia, SC 29208 P: 803-777-8244 F: 803-777-0220 E: dmonrad@gwm.sc.edu (Diane Monrad) Office of Program Evaluation University of South Carolina, College of Education Wardlaw Suite 023, Columbia, SC 29208 P: 803-777-3402 F: 803-777-8838 E: tsdicken@gwm.sc.edu (Tammiee Dickenson)