Download
1 / 25
250 likes | 425 Views
External Evaluation of Reading First 2004-05 Summary of Findings. College of Education, UGA Donna Alvermann Michelle Commeyras Dorothy Harnish Steve Cramer. Purpose of RF External Evaluation.
E N D
External Evaluation of Reading First 2004-05Summary of Findings College of Education, UGA Donna Alvermann Michelle Commeyras Dorothy Harnish Steve Cramer
Purpose of RF External Evaluation Collect, analyze, and report data to answer the following questions about RF implementation and impact in Georgia: • Is the Reading First program being implemented by schools as intended in the Georgia Reading First plan? How does the level of implementation of Reading First relate to the results being achieved in Reading First schools? Is the level of Reading First implementation positively correlated with higher reading achievement? Are Reading First teachers more knowledgeable of scientifically based reading research after the three years of professional learning experiences? • What progress is being made by Reading First schools in improving student reading achievement? Where progress is not apparent, what are the reasons for this? What interventions are required? • What is the impact of Reading First on student achievement in reading as measured by standardized test scores? Is reading achievement in Reading First schools higher than in non-Reading First schools?
Overview of Evaluation Process IMPLEMENTATION OF READING FIRST • Observations of classroom instruction in RF schools by teams of UGA observers (Fall and Spring) • Monthly online surveys and end-of-year interviews with Literacy Coaches • Surveys of RF Teachers, RF School Administrators, Parents of RF students, Literacy Coaches, and Regional RF Consultants (Spring 2005) • RF Teacher Knowledge Survey: pre-assessment (Summer 2004)
Overview of Evaluation Process PROGRESS AND IMPACT OF READING FIRST • Student progress in reading from beginning to end of school year, based on DIBELS test scores in grades K-3 and on PPVT in Kindergarten • Student reading achievement gains from 2004 to 2005, based on ITBS reading tests in grades 1-3 (grade level and cohort analyses) • Confirmatory evidence on student achievement gains from 2004 to 2005, based on CRCT reading tests in grades 1-3 • Comparison of RF and non-RF comparison schools on ITBS and CRCT reading test gains
Analysis of DIBELS Testing Data Comparison of RF students’ DIBELS reading scores at beginning, mid-year, and end of school year to identify progress in reading: • Has the percent of students meeting the benchmark goals for each DIBELS measure improved from the beginning to the end of the school year? • How do differences in improvement within each year vary for each grade level and for each reading measure? • Which schools are making the greatest and least progress, based on DIBELS scores?
90 76.3 80 70 62.1 60 53.3 beg 50 Percent mid 40 33.3 end 28.3 30 22.1 18.4 20 4.6 10 1.6 0 low risk some risk high risk Grade 1 Phoneme Segmentation Fluency
Analysis of PPVT Testing Data Comparison of RF kindergarten student PPVT scores at beginning and end of school year to identify gains in oral vocabulary • What progress did kindergarten students make as measured by the PPVT? • Which schools made the greatest and least progress? • Data submitted by RF schools to UGA
PPVT Test Results Gain in NCE mean score = 4.39
Analysis of ITBS Testing Data Grade level analysis of ITBS results from Spring 2004 testing to Spring 2005 testing • Has the percent of students reading at/above grade level improved in RF schools compared to the previous year for each grade level? • Is there an improvement in ITBS mean NCE scores for students in RF schools compared to the same grade level in the previous year?
Analysis of ITBS Testing Data Cohort (same students) analysis of ITBS results from Spring 2004 testing to Spring 2005 testing • Has the percent of students reading at/above grade level improved in RF schools compared to the performance of these same students in the previous year? • Is there an improvement in ITBS mean NCE scores for students in RF schools compared to the same students in the previous year?
Analysis of ITBS Testing Data Comparison of RF students with those in non-RF schools • Is student achievement on third grade ITBS reading tests different in schools using RF and a sample of schools not using RF? • Schools administering ITBS spring 2004 and spring 2005 in third grade, matched by race/ethnic, LEP, and economic disadvantage
Summary of ITBS Findings On the ITBS, across all analyses: • First grade showed gains in all but one area at the 25th percentile; only spelling did not show improvement; overall gains were smaller compared to other grade levels • Second grade showed greatest gains overall compared to other two grade levels, with gains in all but one area; only vocabulary did not show improvement; cohort results were less positive than grade level analysis, with few gains for the second grade cohort • Third grade showed consistent gains in Spelling and Language Total; greatest losses were in Vocabulary, Listening, and Reading Total
Analysis of CRCT Testing Data Comparison of RF and non-RF schools on CRCT reading tests as confirmatory evidence of RF impact • Is student achievement in reading as measured by CRCT different in schools using RF and those not using RF? • What percentage of students showed improvement within each group from spring 2004 to spring 2005? • How does the percent of students meeting or exceeding state standards in CRCT reading improve for cohorts of RF students each year?
