1 / 25

Team Read Improving Literacy in the Seattle School District

Team Read Improving Literacy in the Seattle School District. PA 590C – Program Evaluation Fall 2007 Anne Debuyserie.

truman
Download Presentation

Team Read Improving Literacy in the Seattle School District

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Team Read Improving Literacy in the Seattle School District PA 590C – Program Evaluation Fall 2007 Anne Debuyserie

  2. “...a Seattle School district tutoring program dedicated to increasing the reading skills of elementary students through year-long coaching by high school students ... The program's goals for participating high-school tutors are to develop work experience, a sense of responsibility and accomplishment, and to learn the rewards of community service.” (Team Read Mission Statement, 1998)

  3. What is Team Read? • Private-public partnership to improve literacy in Seattle Public Schools • Cross-age tutoring program using high school student as tutors • Started in March 1998 with a three-phase implementation • June 1999: 10 schools, 335 students, 300 coaches • June 2000: addition of 7 schools

  4. Team Read- Org Chart

  5. Team Read Stakeholders • Craig McCaw (and wife Susan): Businessman • John Stanford: Seattle Schools Superintendent • Joan Dore: Seattle Schools Reading Specialist • Tricia McKay: Team Read Program Mgr • Team Read Advising Board: Seattle School District Staff, Alliance for Education pgm representatives, Mc Caws, community representatives. • Additional investors joined by year 2.

  6. Who is Team Read? • Coaches • Strict selection guidelines • High expectations from Team Read leaders • Compensation • Readers • School selection comes first • Student selection based on reading level • Eligibility based on need • First-come, first-serve basis • Low socio-economic background

  7. Who is Team Read? • Site Coordinators • Part-time, teachers or school staff • Meet informally with Tricia McKay for feedback • Volunteers • Needed due to amount of work • Partnership with VISTA/Americorps and U. of Washington Pipeline • Assist site coordinators

  8. Team Read Session • Two 1 hr session per week during school year • Standardized session format and structure • 3.00 PM: coaches arrive • 3.15 PM: snack with reader • 3.30 pm: reading session begins • 4.15 PM: reading session ends/ coaches fill out paperwork • 4.30 PM: reading coaches leave • Opportunity for site coordinators to be creative

  9. Team Read Evaluation Results • 1st year evaluation: not as positive as expected • 2nd year: promising but Team Read’s impact not as promising as hoped • Tricia McKay disappointed by results and wonders if findings provide her with the information needed for improvements • Questions about the accuracy of results.

  10. Team Read Evaluations • Evaluation conducted by Margo Jones, statistician by trade. • 17 schools in the program by June 2000 but only 10 used in the evaluation. • 3 main objectives to show Team Read’s Impact: • Goal#1: Do the reading skills of the student readers improve significantly during their participation in the Team Read program? • Goal#2: How does the program affect the reading coaches? • Goal#3 What is working well, and what can be improved?

  11. Evaluation Methods Goal #1 • Goal #1: Reading skills. Two-fold approach

  12. Apples and oranges

  13. Evaluation Methods Goal #1, Cont’d • 2nd method: determine proportion of Team Read participants who moved from below grade level test score to at-or above level compared to all students from entire district • Note: one of the selection criteria to be in Team Read is to be within bottom 25% of district reading test score

  14. Evaluation Methods Goal #1 • Both methods for goal 1 are flawed. • Note from Jones in report confirming problems with different tests. • “The question then becomes ‘Is it reasonable to assume that the pre- and post-tests measure the same skills?’ The evaluator adopted a correlation criterion for answering this question. If the correlation between pre- and post-tests was near or above .8 (see footnote 5) (as was the case for the 2nd grade, where pre- and post-tests were essentially the same), the pre- to post-test change score was interpreted as a gain score. If the correlation coefficient was much less than .8, no analysis was performed.”

  15. Evaluation Methods, Goals #1- Assessment • Visible problems with tests: • Pre and post-tests do not measure the same skills except in 2nd grade • All Team Read participants are within the bottom quartile of the reading test scores and can’t be compared with students district-wide. • Analysis could not be carried out for some grades (3rd) due to incompatibility of reading tests

  16. Evaluation Method, Goal #2 • Method: questionnaire given to coaches during their last week of coaching. • Goal: to assess the impact of Team Read on coaches. • Identify 3 main areas of job satisfaction: • How positive is their experience with TR? • Extent to which the coaches feel that TR helps students readers • How supportive is TR’s site coordinators and staff?

  17. Evaluation Method, Goal #2 - Assessment • Jones results showed that coaches were satisfied with their job and felt a sense of pride and accomplishment. • Test performed by Jones was successful in meeting its goals, unlike in Goal #1. • Some questions provided answers as to what to improve in the program. • Test is useful to the program manager

  18. Evaluation Method, Goal #3 • Method: interviews with major stakeholders, one site visit, review of last 5 years’ research literature on cross-age tutoring and last 3 years research on best practices. • Goal: assess the overall progress of the program.

  19. After the Evaluation • Margo Jones makes her recommendations based on her findings to improve the efficacy of Team Read. • Tricia McKay reads the evaluation but fail to see what the results are telling her about the program

  20. Evaluation Methods - Summary

  21. Evaluation Problems • Academic v. Practitioner approach. • Result presentation. • Research design flaws. • Not all the data was analyzed • Comparison with dissimilar items • Statistical analysis stretched • Evaluation only covers 3 goals. • Lack of communication between the evaluator and the program manager.

  22. What Could Have Been Done Better?

  23. Impact: 2005-2006 Reading Results From Team Read’s 2006 Annual Report. http://www.teamread.com/downloads/team_read_106903_report_proof.pdf

  24. Team Read Update - 2006 • Program Evaluation Results for 2005-2006 School Year    • 70% of 2nd graders and 52% of 3rd graders were reading at/above or approaching grade level • 54% of 2nd graders and 35% of 3rd graders gained greater than 1.5 grade levels in reading • 98% of the parents of 2nd & 3rd graders and 92% of their teachers reported increased reading skills as a result of participation in Team Read • 76% of 2nd & 3rd graders said that reading was more fun since joining Team Read

More Related