Presented by: Tamara M. Walser , Ph.D., Lead Evaluator &

1. Hill Reading Achievement Program Student Achievement Results for Brunswick County Schools, 2009-2011 Presented by: Tamara M. Walser, Ph.D., Lead Evaluator & Emily R. Grace, M.Ed., Project Coordinator The University of North Carolina Wilmington March 29, 2012

3. Research Questions and Measures

4. Students, Schools, and Teachers Students First Year 2009-2010 Sample (n = 325) Two Year 2009-2011 Longitudinal Sample (n = 164) Schools and Teachers 2009-2010: 10 Elementary, 4 Middle Schools 32 Teachers 2010-2011: 10 Elementary, 3 Middle, 3 High Schools 37 Teachers

5. Woodcock-Johnson III Tests of Achievement Results

6. Description of Subtests Used Letter-Word Identification: List of individual words for student to read aloud Reading Fluency: Short sentences for student to read quickly and identify as true or false Passage Comprehension: Short sentences containing a blank for student to read and supply appropriate word Word Attack: List of “nonsense” words for student to read aloud, measuring phonics skills

7. What are Age-Referenced Standard Scores? Age-referenced standard scores account for expected academic growth. A student who makes expected growth would have the same standard score at pretest and posttest. An increase in a standard score indicates greater than expected growth; a decrease indicates the opposite. The average standard score for the WJ-III is 100; the average range is 90-110.

8. Means, Standard Deviations (SD), and Change Scores for First Year Sample and Two Year Longitudinal Sample Age-Referenced Standard Scores on Woodcock Johnson Tests of Achievement III Reading Subtests an size for First Year Sample is 324 for Reading Fluency and Word Attack subtests *p <.05. **p < .01

9. All Students in Two Year Longitudinal Sample (n = 164)

10. Elementary Students Fall 2009 & Spring 2010 (n = 257) Spring 2011 (n = 113)

11. Middle School Students Fall 2009 & Spring 2010 (n = 68) Spring 2011 (n = 51)

12. White Students Fall 2009 & Spring 2010 (n = 170) Spring 2011 (n = 90)

13. Black Students Fall 2009 & Spring 2010 (n = 99) Spring 2011 (n = 42)

14. Hispanic Students Fall 2009 & Spring 2010 (n = 32) Spring 2011 (n = 14)

15. Students Identified to Receive Exceptional Children’s ServicesFall 2009 & Spring 2010 (n = 109) Spring 2011 (n = 68)

16. Students Not Identified to Receive Exceptional Children’s Services Fall 2009 & Spring 2010 (n = 216) Spring 2011 (n = 96)

17. North Carolina End-Of-Grade Reading Comprehension TestResults

18. NC End-Of-GradeReading Achievement Levels for Students in First Year Sample2009 (n = 244) 2010 (n = 302)

19. NC End-Of-Grade Reading Achievement Levels for Students in Two Year Longitudinal Sample2009 (n = 117) 2010 (n = 158), 2011 (n = 154)

20. NC End-Of-Grade Reading Achievement Levels for White Students in Two Year Longitudinal Sample (n = 58)

21. NC End-Of-Grade Reading Achievement Levels for Black Students in Two Year Longitudinal Sample (n = 32)

22. NC End-Of-Grade Reading Achievement Levels for Students Identified to Receive EC Services in Two Year Longitudinal Sample (n = 44)

23. NC End-Of-Grade Reading Achievement Levels for Students Not Identified to Receive EC Services in Two Year Longitudinal Sample (n = 69)

24. Conclusion Annual and longitudinal results provide support for the effectiveness of HillRAP in improving student academic achievement in reading. Overall, students demonstrated greater than expected growth on reading subtests of the Woodcock-Johnson III Tests of Achievement; NC End-of-Grade Reading Comprehension Test results showed a similar pattern of improved student reading achievement for HillRAP Students.

25. Related Presentations Walser, T. M., Parker, M. A., Grace, E. R., Bell, C. A., Dowcett, E. T., & Tart, M. E. (2012, April). The Hill Reading Achievement Program model replication: Results and implications. To be presented at the American Educational Research Association annual conference, Vancouver, BC. Walser, T. M., Parker, M. A., Grace, E. R., Dowcett, E. T., & Hodges, D. M. (2012, April). The value of replication: Learning about an educational model and evaluation practice. To be presented at the American Educational Research Association annual conference, Vancouver, BC. Walser, T. M., & Parker, M. A. (2012, April). Hill Reading Achievement Program evaluation project: Results and reflections. To be presented at the Watson School of Education Scholarship Brown Bag Series, Wilmington, NC. Walser, T. M., Grace, E. R., Bell, C. A., Parker, M. A., & Tart, M. E. (2012, February). Hill Reading Achievement Program Studies: What We Learned about Impacts on Student Achievement, Program Implementation, and Conducting Research in Schools. Presented at the North Carolina Association for Research in Education annual conference, Winston-Salem, NC.

26. Related Presentations Parker, M. A., Walser, T. M., Tart, M. E., & Bell, C. A. (2012, February). Methodological questions about aggregating results from Hill Reading Achievement studies across North Carolina. Presented at the Eastern Educational Research Association annual conference, Hilton Head Island, SC. Walser, T. M., & Parker, M. A. (2011, November). How to train your dragon: One story of using a quasi-experimental design element in a school-based evaluation study.  Paper presented at the American Evaluation Association annual conference, Anaheim, CA. Walser, T. M., Parker, M. A., Grace, E. R., & Hodges, D. M. (2011, November). The role of replication in evaluating complex systems in education. Paper presented at the American Evaluation Association annual conference, Anaheim, CA. Walser, T. M., Hodges, D. M., Wetherill, K., Grace, E. R. (2010, October). The role of program evaluation in replicating an educational model. Presented at the Consortium for Research on Educational Accountability and Teacher Evaluation annual conference, Williamsburg, VA. Walser, T. M. (2009, November). Context and quasi-experiments in schools: The case for cohort comparison groups. Paper presented at the American Evaluation Association annual conference, Orlando, FL.

27. Thank You!