Project MORE 2001-2008 Independent Evaluation

1. Project MORE 2001-2008 Independent Evaluation Completed by The Center for Evaluation Services Bowling Green State University Updated 11/12

2. Project MORE Evaluation Results 2001-2008 • Independent Evaluation conducted from 2001-to 2008 by Center for Evaluation Services at Bowling Green State University • Consistent Causal Comparative Results at the Statistically Significant Level (.05) over six years. • Evaluation Results to be published in Education and Training in Developmental Disabilities (December, 2007) to qualify as Scientifically-Based Research.

3. First, CES looked at Project MORE reading gains with BGSU Informal Reading Assessment and the STAR Reading Assessment • Results: Students with specific learning disabilities, cognitive disabilities, emotional disturbances, and Title 1 students had statistically significant gains, and in many cases there were month-for-month reading gains. • Project MORE students with disabilities and students receiving Title 1 services were compared to similar students based on the State of Ohio’s Similar District Software. Both the Project MORE students and the control students were receiving either Title 1 or special education prescribed services. However, Project MORE students received HOSTS as or in addition to their prescribed Title 1 or special education services. Results are presented below. Project MORE Results (2001-2002)

4. First, CES looked at Project MORE reading gains with BGSU Informal Reading Assessment, the STAR Reading Assessment, and the Woodcock-Johnson Reading Achievement Assessment • Results: Students with specific learning disabilities, cognitive disabilities, emotional disturbances, and Title 1 students had statistically significant gains, and in many cases there were month-for-month reading gains on the STAR. • Project MORE students with disabilities and students receiving Title 1 services were compared to similar students based on the State of Ohio’s Similar District Software. Both the Project MORE students and the control students were receiving either Title 1 or special education prescribed services. However, Project MORE students received HOSTS as or in addition to their prescribed Title 1 or special education services. The Woodcock-Johnson Reading Achievement battery was piloted with 2 groups for the 2002-2003 school year. Results are presented below. Project MORE Results (2002-2003)

5. First, CES looked at Project MORE reading gains with BGSU Informal Reading Assessment and the Woodcock-Johnson Reading Achievement Assessment • Results: Students with specific learning disabilities, cognitive disabilities, and Title 1 students had statistically significant gains, and in many cases there were month-for-month reading gains. • Project MORE students with disabilities and students receiving Title 1 services were compared to similar students based on the State of Ohio’s Similar District Software. Both the Project MORE students and the control students were receiving either Title 1 or special education prescribed services. However, Project MORE students received HOSTS as or in addition to their prescribed Title 1 or special education services. Results are presented below. Project MORE Results (2003-2004)

6. Examined two Project MORE mentoring programs: HOSTS (established) v. Reading-tutors (pilot) v. control students • Students in HOSTS and Reading-tutors had similar reading gains although as expected the established program had a few more significant gains • Results: Students with specific learning disabilities, cognitive disabilities, and Title 1 students had statistically significant gains, and in many cases there were month-for-month reading gains. • Project MORE students with disabilities and students receiving Title 1 services were compared to similar students based on the State of Ohio’s Similar District Software. Both the Project MORE students and the control students were receiving either Title 1 or special education prescribed services. However, Project MORE students received HOSTS or Readng-tutors as or in addition to their prescribed Title 1 or special education services. Results are presented below. Project MORE Results (2004-2005)

7. Examined two Project MORE mentoring programs: HOSTS v. Reading-tutors v. control students in only 3 groups of students 2nd and 3rd grade students with specific learning disabilities and 3rd grade students receiving Title 1 services. • Students in HOSTS and Reading-tutors had similar reading gains although Reading-tutors program had a few more significant gains • Results: Students with specific learning disabilities and Title 1 students had statistically significant gains, and in many cases there were month-for-month reading gains. • Project MORE students with disabilities and students receiving Title 1 services were compared to similar students based on the State of Ohio’s Similar District Software. Both the Project MORE students and the control students were receiving either Title 1 or special education prescribed services. However, Project MORE students received HOSTS or Reading-tutors as or in addition to their prescribed Title 1 or special education services. Results are presented below. Project MORE Results (2005-2006)

8. Examined two Project MORE mentoring programs: Reading-tutors v. control students in only 3 groups of students 2nd, 3rd, and 4th grade students with specific learning disabilities. • Results: Students with specific learning disabilities had statistically significant gains, and in many cases there were month-for-month reading gains. • Project MORE students with disabilities were compared to similar students based on the State of Ohio’s Similar District Software. Both the Project MORE students and the control students were receiving special education prescribed services. However, Project MORE students received Reading-tutors as or in addition to their prescribed special education services. Results are presented below. Project MORE Results (2006-2007)

9. Examined two Project MORE mentoring programs: Reading-tutors v. control students in only 2 groups of students 3rd and 4th grade students with specific learning disabilities. • Results: Students with specific learning disabilities had statistically significant gains, and in many cases there were month-for-month reading gains. • Project MORE students with disabilities were compared to similar students based on the State of Ohio’s Similar District Software. Both the Project MORE students and the control students were receiving special education prescribed services. However, Project MORE students received Reading-tutors as or in addition to their prescribed special education services. Results are presented below. Project MORE Results (2007-2008)

10. Project MORE Results (2007-2008) Regression Discontinuity:In 2007-2008 CES utilized a Regression Discontinuity Design to evaluate the effects of Project MORE on students’ reading fluency gains. The Oral Reading Fluency section of DIBELS was given to all 2nd, 3rd, and 4th graders at grade level for the 11 Project MORE schools (not all schools used Project MORE at all three grade levels). The Fall Benchmark was the pretest, and the Spring Benchmark was the posttest. Each grade level had a different cut-off score to determine whether students were in the MORE group (intervention) or the control group. However, the cut-off score differed depending on school. For the overall analysis, we chose the highest Project MORE words-per-minute score at Fall Benchmark as the cut-off score. The control group students were all students at Project MORE schools who had higher words-per-minute scores on the Fall Benchmark, but did not receive the Project MORE intervention. Thus, the Project MORE group (intervention) were students with disabilities or at-risk for reading failure. The control group were general education students. The intervention group (Project MORE students), which has the lower reading pretest scores, must have significantly higher reading gains (in this case, words-per-minute) than their fellow students with higher pretest scores (control group). It is very difficult for an intervention to produce this type of result at a statistically significant level. RESULTS:The results indicated that the DIBELS posttest scores increased at a statistically significantly higher rate for Project MORE students (with disabilities and at-risk) than for their classmates (without disabilities) in 2nd, 3rd, and 4th grades during the 2007-2008 school year.

11. 48 GROUP 200 CONTORL INTRVNT 150 f r o d t 100 s o p 50 0 0 20 40 60 80 100 120 140 160 predorf 2nd Grade Regression Discontinuity Results and Summary 2nd grade results:Fall-to-spring reading gains on DIBELS Oral Reading Fluency for Project MORE students (with disabilities and at-risk) were compared to their classmates (without disabilities) to measure reading growth rate due to the Project MORE intervention at 10 schools. The results indicated that the DIBELS posttest scores increased at a statistically significantly higher rate for Project MORE students (with disabilities and at-risk) than for their classmates without disabilities (T = 7.55 p < .00).

12. 3rd Grade Regression Discontinuity Results and Summary 3rd grade results:Fall-to-spring reading gains on DIBELS Oral Reading Fluency for Project MORE students (with disabilities and at-risk) were compared to their classmates (without disabilities) to measure reading growth rate due to the Project MORE intervention at 9 schools. The results indicated that the DIBELS posttest scores increased at a statistically significantly higher rate for Project MORE students (with disabilities and at-risk) than for their classmates without disabilities (T = 4.20 p < .00).

13. 4th Grade Regression Discontinuity Results and Summary 4th grade results:Fall-to-spring reading gains on DIBELS Oral Reading Fluency for Project MORE students (with disabilities and at-risk) were compared to their classmates (without disabilities) to measure reading growth rate due to the Project MORE intervention at 6 schools. The results indicated that the DIBELS posttest scores increased at a statistically significantly higher rate for Project MORE students (with disabilities and at-risk) than for their classmates without disabilities (T = 2.25 p < .03).

14. Over the 7 year period, the evaluation has established Project MORE students have significant reading gains that many times are month-for-month. Overall, Project MORE students outperformed the comparison students in the numerous comparisons on both informal and standardized assessments. Over the course of seven years, there were no analyses in which the control group outperformed Project MORE students at a statistically significant level. Piloting an Implementation Integrity Checklist for Reading-tutors program in was completed in the Spring of 2008-2009. Utilizing a Regression Discontinuity Design, the results indicated that the DIBELS posttest scores increased at a statistically significantly higher rate for Project MORE students (with disabilities and at-risk) than for their classmates (without disabilities) in 2nd, 3rd, and 4th grades. These results will be submitted for publication in the Fall of 2009. Project MORE Evaluation was published in Education and Training in Developmental Disabilities (December, 2007) Overall Project MORE Results and New Directions