Dollars for Tutoring: The Cost of SES and District Tutoring Program Effectiveness

1. Dollars for Tutoring: The Cost of SES and District Tutoring Program Effectiveness Tracy Alberry, M.S. CSU San Bernardino Deborah Stine, Ph.D., Michael Verdi, Ph.D., Doris Wilson, Ed.D. CSU San Bernardino

2. Background • According to NCLB, low-income students who attend schools that have been identified as “in need of improvement” for three consecutive years are eligible to receive free math and reading tutoring services known as Supplemental Educational Services or SES. • School districts offer afterschool tutoring programs to students scoring below proficient on the California CSTs. • Districts in Program Improvement cannot be SES providers.

3. Supplemental Educational Services (SES) • Since the passage of the No Child Left Behind (NCLB) Act of 2001, schools in Year 2 of Program Improvement must offer supplemental educational services to low-income students. Year 2 Program Improvement schools are schools that receive Title 1 funds and have not made Adequate Yearly Progress (AYP) for 3 consecutive years. • Supplemental educational services (SES) are tutoring services offered to low income students by outside tutoring companies or agencies.

4. Questions: Is tutoring effective in improving academic achievement? How can effectiveness be measured? In this study, Student academic achievement, as measured by the California Standardized Tests, of students attending a district tutoring program will be compared to students attending Supplemental Educational Services (SES) tutoring, mandated by No Child Left Behind (NCLB) and to students who receive no academic after school tutoring.

5. Purpose of the Study • The purpose of this study is to determine what are the most effective tutoring programs in increasing student achievement as measured by the California Achievement Tests.

6. Target Population • Administrators and lawmakers who work with the No Child Left Behind Act of 2001. • Researchers and Providers of Supplemental Educational Services.

7. The Study In this study, specifically the expected growth on the CST will be compared to the actual growth of students attending a district tutoring program, students attending Supplemental Educational Services (SES) tutoring, mandated by No Child Left Behind (NCLB) and to students who receive no academic after school tutoring. School and grade level growth will be examined to identify any trends or irregularities. The statistical method to be used is still in development.

8. Supplemental Educational Serives • According to Burch (2007) only Minneapolis and Chicago school districts have done studies attempting to assess SES’s impact on learning outcomes “Research to date offers only limited understanding of what kinds of assessment might be useful in determining the costs and benefits of various SES models” (p.15).

9. Quantitative analysis will be completed using archived data from a large urban school district and the SPSS program.

10. Analysis • Quantitative 2 x 2 MANCOVA • Two IVs (SES/District Tutoring) with 2 levels- No SES tutoring yes tutoring/No District tutoring yes District Tutoring.

11. Student Groups Examined from schools that are title I or Program Improvement: • Students’ CST scores will be examined in Reading Language arts and Mathematics: 1)Student performance with district tutoring 2) Student performance without any tutoring 3) Student performance with SES tutoring 4) Student performance with SES and district tutoring (If any)

12. Variables • Independent: Supplemental Services (categorical), District Tutoring, • Dependent: The Math scores and reading scores on standardized tests (continuous) *Student may receive tutoring in reading or math or both

13. Covariates • # hours • Free or reduced lunch or paid lunch • Social Economic Status • Gender • Ethnicities • Grade • School • Subject tutored in • Provider of SES

14. Goals • Identify if there are academic gains in math or English after students receive district tutoring or SES tutoring. • To determine if students who receive tutoring score significantly higher on CSTs than like students who do not receive tutoring. • To determine if District provided tutoring is more effective than SES tutoring. • To determine which SES companies are the most effective.

15. Methods Collect archived data to include: A. Student Demographics • Tutoring program hours • Tutoring programs attended SES/District • Ethnicity • School/Grade • CST scores in English and Math • Social Economic Status • Gender • Teachers • SES Companies assigned to Students

16. Proposed Outcomes, Hypothesis: • Students receiving District tutoring will increase their CST scores more than students who do not receive tutoring. • Students receiving District Tutoring will increase their CST scores as much if not more than students attending SES Tutoring. • Students receiving SES tutoring will increase their CST scores more than students who receive no tutoring.

17. Hopeful Changes and Implications • The data from this study could be used to change educational policy relating to NCLB. If District provider could be as effective as SES providers, Districts in Program Improvement should be able to be SES providers themselves. • Potential to change NCLB policies • Possibility of increasing tutoring use • Improve quality of tutoring programs by identifying companies who need to improve based on lack of score increase.

18. Problems faced: The debate continues in the research on whether or not SES services are effective. Findings vary by researcher with effectiveness ranging by provider. The problem researchers face is that there is a lack of comparable data to use to measure effectiveness. Each provider uses their own pre and post test. It is argued that criterion reference standardized tests scores may not be accurate as they do not vertically compare from one grade to the next. With normalized scores though they may be the best that we have to measure effectiveness.

19. Silver Bullet Solution • After-school programs are presented as the latest “silver bullet solution” to social and educational challenges, but successes will be questionable if positive results can not be found and measured (Miller, 2001). • If we really want after school programs to become at least a part of the solution, we need to provide them with the financing staff and knowledge base to become highly effective program (Miller, 2001). • If a program is to be a solution to increasing student achievement, the program should be measurable.

20. The Research • Bloom (1984) found that 80% of students who did poorly in conventional education profited from tutoring. The 1:1 feedback and correction contributed to success. • Cohen , Kulick & Kulick (1982) in their meta analysis of peer tutoring found that students who were peer tutored outperformed those not tutored on two levels: subject matter and positive attitudes toward school work (Rabow, P. xxiv, 1999).

21. Cohen 2003 • Cohen (2003) also stated that supplemental providers needed experienced tutors and must have demonstrated success with students. Furthermore, supplemental services should have provided research-based instruction and proven strategies to help students learn. Also, each student’s program must be designed to meet his or her specific needs. Because students were diverse in their methods of learning, their programs must be customized while meeting state standards. • The No Child Left Behind Act defines supplemental educational services as tutoring and research based academic enrichment programs that supplement but do not replace-instruction provided during the school day. (Cohen 2003) and “…must be designed to increase students‘ academic achievement and help them meet state standards.” p.36

22. Miners 2007 • Miners (2007) in “SES Effectiveness Is a Matter of Debate” reviewed two recent reports on SES with differing opinions using data from different sources. One reported SES is giving gains in student achievement while the other reported evidence is inconclusive due to states’ lack of monitoring capabilities. It shows how data from different sources can be used to see two completely different outcomes. The article highlights states abilities to monitor SES programs. The article failed to go in depth regarding the data used in the studies.

23. Burch (2007) • There seems to be no consistency between states on implementing and monitoring SES programs. The policy brief by Patricia Burch (2007) called “Supplemental Services Education Services under NCLB”, examined several studies and national information to make recommendations for SES, which is as of 2007, is entering its sixth year of implementation. • Highlighted are the facts that states and districts often face limitations such as lack of funding, in implementing and monitoring the effectiveness and policies of SES programs. Patricia Burch (2007) recommended the NCLB law be redesigned.

24. Final Recommendations & Questions • A uniform procedure such as a uniform pre and post test for tutoring providers needs to be implemented that would allow for student achievement to measured in a meaningful way. • If achievement can not be measured, this may not be the best use of time and Federal funds. • A closer look at the curriculum used and what constitutes effectiveness is needed. For instance would learning study skills be more effective in increasing achievement than a workbook in math facts? • I would recommend a mixed methods study that incorporates observations, interviews and surveys from parents and student.