1 / 48

Washington State LASER Highlights From Recent Evaluation Studies

Washington State LASER Highlights From Recent Evaluation Studies. Dave Weaver RMC Research Corporation 111 SW Columbia, Suite 1200 Portland, Oregon 97201. Agenda For This Segment. Impact on Student Achievement Studies Student-Level analysis of 2006 Science WASL results

saeran
Download Presentation

Washington State LASER Highlights From Recent Evaluation Studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Washington State LASERHighlights From Recent Evaluation Studies Dave WeaverRMC Research Corporation111 SW Columbia, Suite 1200Portland, Oregon 97201 RMC Research Corp.

  2. Agenda For This Segment • Impact on Student Achievement Studies • Student-Level analysis of 2006 Science WASL results • School-Level analysis of 2007 Science WASL results • West Valley School District • Implementation Study • Study of Highly Successful Schools • Evaluation activities for 2007-2008 RMC Research Corp.

  3. Student-Level Analysis of 2006 Science WASL Results Spring & Summer 2007 RMC Research Corp.

  4. Student-Level Analysis of Grade 5 WASL Data for 2006 • Factors considered • Student demographics (i.e., socioeconomics, race/ethnicity) • Grade 5 science WASL results for 2006 • Grade 4 math and Reading WASL results for 2005 • The number of LASER PD hours the student’s Grade 5 teacher received each year and prior to the administration of the 2006 WASL • The number of LASER PD hours the student’s Grade 4 teacher received each year and prior to the administration of the 2005 WASL. RMC Research Corp.

  5. 2-Years of Teacher Influence 2006 Student-Level Science Data for Grade 5 PD Hours of Student’s Grade 5 Teacher in 2006 Linked By Teacher Linked By Student 2005 Student-Level Math Data for Grade 4 PD Hours of Student’s Grade 4 Teacher in 2005 Linked By Teacher RMC Research Corp.

  6. Grade 5 Regression Analysis Results Dependent Variable: 2006 Grade 5 Science Scale Score RMC Research Corp.

  7. Factors Influencing 2006 Grade 5 Science WASL Results 32,228 Students 860 Schools 1325 Classes R2 = .645 RMC Research Corp.

  8. Grade 8 Regression Analysis Results Dependent Variable: 2006 Grade 8 Science Scale Score RMC Research Corp.

  9. Factors Influencing 2006 Grade 8 Science WASL Results 7,432 Students R2 = .634 RMC Research Corp.

  10. Finding 1 • After controlling for demographics and entry skill levels of students, the results reveal that the current year of LASER professional development is a significant positive predictor of scores on the science WASL results. • If all things remain equal, 25 hours of professional development translates into a 1-point gain in the science scale score. RMC Research Corp.

  11. Finding 2 • The professional development provided during the year that the teacher had the student in class had the greatest impact on student performance. • The accumulated effect of previous years of professional development diminished predictive value. RMC Research Corp.

  12. Finding 3 • The current year of professional development of the 4th and 5th grade teachers is a significant positive predictor of student subscale scores for: • Inquiry in Science (.039) • Open-ended responses (.029) • Applications of Science (.020) RMC Research Corp.

  13. Finding 4 • Factors beyond the control of LASER (entry skills, demographics, poverty) account for 90% of knowable variance. • Changes in any of these other factors can easily overshadow the positive effect of LASER professional development. RMC Research Corp.

  14. School-Level Analysis of 2007 Science WASL Results Fall 2007 RMC Research Corp.

  15. What We Did • Data Sources • School Level WASL Data from 2007 • LASER Professional Development Database • School Demographic Data for 2006-07 • Regression Analysis • Independent Variable: • Total PD hours of core teachers in a school for 2 years prior to 2007 WASL per FTE • Dependent Variable: • Percent of students in the school that met the Grade 5 science standard on the 2007 WASL • Control Variables: • Percent of students who qualify for free or reduced price lunch • Percent of students in the school that met the Grade 4 math standard on the 2006 WASL • Students per classroom teacher RMC Research Corp.

  16. Grade 5 Regression Analysis Results R2 = .617 RMC Research Corp.

  17. Findings • The number of hours of LASER professional development per FTE over a 2 year period prior to the WASL is a significant predictor of student achievement on the Grade 5 science WASL above and beyond what can be explained by the number of students who qualify for free or reduced price lunch, class size, or student skills. • This finding was not evident at Grade 8. RMC Research Corp.

  18. 2-Year PD Hours Per FTE vs. Student Achievement RMC Research Corp.

  19. Quartile Grouping of Schools RMC Research Corp.

  20. Total School 2-Year PD Hours Per FTE Quartile Ranking vs. Percent of Students Who Met The Grade 5 Science WASL in 2007 Note: Means adjusted for Percent of students on free or reduced price lunch, class size, and previous year mathematics achievement. RMC Research Corp.

  21. Distribution of Study Schools By LASER Alliance RMC Research Corp.

  22. Analysis of West Valley School District Data Fall 2005 RMC Research Corp.

  23. West Valley School District study • Peter Finch, Assistant Superintendent • Pilot test involving phased implementation of inquiry-based instructional materials • Noticed an important relationship between the number of modules students experienced and Grade 5 science WASL scores. RMC Research Corp.

  24. Science WASL – WVSD Grade 5 % students meeting standardStudents receiving LASER Science instruction State Average RMC Research Corp.

  25. Regression Analysis of 2004 Grade 5 Science WASL Results RMC Research Corp.

  26. Regression Analysis of 2005 Grade 5 Science WASL Results RMC Research Corp.

  27. Conclusion The number of science modules that students experience has a significant positive impact on their science WASL achievement above and beyond what can be explained by demographic factors.(FRL, Special Education, & %White) RMC Research Corp.

  28. Study of Highly Successful Schools Winter & Spring 2007 RMC Research Corp.

  29. Purpose of Study • To identify schools that have demonstrated a consistent increases in the percentage of students who met the science standard on the Washington State Assessment of Student Learning (WASL). • Interview the staff of those schools to determine to what they attribute their success. RMC Research Corp.

  30. Criteria for Selecting of Schools • Focus on Grade 5 • Very few schools could demonstrate consistent improvement at Grade 8 and 10. • At least 10 Grade 5 students complete the science WASL in 2004, 2005, and 2006. • An increase of at least 6% or more students meeting the standard each year. • An increase of at least 17.5% or more students meet the science standard across the 2 time intervals. • Started with at least 20% of the students meeting the science standard in 2004. RMC Research Corp.

  31. Description of Selected Schools • 40 schools selected for interview • 18 (45%) were in districts served by a LASER Alliance. • 31 (78%) were in districts that participated in at least 1 LASER Strategic Planning Institute • 10 (25%) are in districts that participate in one of the state MSP projects • 4 (10%) are participating in the NCOSP • 1 school did not participate in the interviews RMC Research Corp.

  32. Interviews • During December 2006 and January 2007. • Conducted By: • RMC Research staff • Graduate students from the science education department of Portland State University • Using structure interview protocol • Developed with input from LASER project leadership • Training provided • With person most knowledgeable about success • Initial contact with Principal and sometimes other school staff RMC Research Corp.

  33. Two Tiered Approach • Open ended initial question • Summarized increases in WASL scores. • What is your school doing to improve science teaching and learning? To what do you attribute this success? • Followed by questions about the contribution of specific aspects such as • Professional development • Instructional materials • Assessments • LASER services RMC Research Corp.

  34. Finding 1– Instructional Materials • All schools used inquiry-based instructional materials – Mostly FOSS • 49% (19) of the schools indicated that recent changes to the science instructional materials attributed to their success • Some customized or locally developed kits • Heavily supplemented • Non-LASER Alliance schools use a wider variety of supplementary materials and approaches. RMC Research Corp.

  35. Finding 2 – WASL Preparation • Initially, 54% (21) of the schools indicated that they engage students in some form of preparation for the WASL. • Upon questioning, 95% (37) indicated that they engage students in some form of preparation for the WASL. RMC Research Corp.

  36. Finding 2 – WASL Preparation • Most use WASL release items • Prepare students before WASL administration • Develop WASL-like practice assessment during the year • Explicit emphasis on • WASL “format” • WASL vocabulary RMC Research Corp.

  37. Finding 3 – Professional Development • 82% (32) of the schools indicated that professional development contributed to improving student WASL scores • PD in Non-LASER Alliance schools dominated by kit and WASL training • PD in LASER Alliance schools is more varies • Journaling/Notebooks • Leadership • Science Content • Inquiry • Coaching, Mentoring, & Consultant • EALRs and GLEs • More Organized School-Based PD RMC Research Corp.

  38. Finding 4 – Science Advocate • 77% (30) of schools interviewed identified 1 or 2 people who were advocates for science education. • Passionate about science – Made things happen • Roles included: • Science specialist (16) • Teacher leaders (9) • Strong teacher or team (6) • Community member (3) • Administrator (2) RMC Research Corp.

  39. Finding 5 – EALRs & GLE • 51% (20) of the schools attributed the success of their students to increased emphasis on and alignment of curriculum to the EALR’s and GLEs. • 11 are Non-LASER Alliance schools • 9 are LASER Alliance schools RMC Research Corp.

  40. Finding 6 – Instructional Time • 38% (15) of the schools attributed their success at least partially to increased science instructional time • 8 are Non-LASER Alliance schools • 7 are LASER Alliance schools RMC Research Corp.

  41. Recommendations From Schools • Interviewees were asked: • What 2 recommendations would you give to the staff of another school working to improve science teaching and learning that would improve student performance on the science WASL? • Responses varied considerably RMC Research Corp.

  42. Top 6 Recommendations • Use released items & PCAs to develop skills needed to be successful on the WASL (18). • Provide adequate professional development (13) on materials and WASL. • Increase science instruction time (10). • Engage students in hands-on, inquiry-based science activities (9). • Provide teacher collaboration time (7). • Align with EALRs and GLEs (6). RMC Research Corp.

  43. Conclusions • LASER actively promotes most of the key findings from this study • Inquiry-based instructional materials • Preparing students for the science WASL • Professional development • Science advocacy (leadership) • Alignment of curriculum to EALR’s & GLEs • Increased science instructional time RMC Research Corp.

  44. Evaluation Plans for 2007-08 RMC Research Corp.

  45. Sentinel Site Visits • Identify of approximately 30 schools in 2007–08 and 30 in 2008–09 • Schools With Significant LASER Participation • Two Groups of 15 schools each • Demonstrated significant positive gains • Demonstrated little, no, or negative gains • Each site visit: 1 ½ to 2 days RMC Research Corp.

  46. Site Visit Data Collection • Interviews with principals and/or science teacher leader • Interviews with teachers • Observations of at least 3 lessons where inquiry–based instructional materials are used • Based on STAR Protocol • Rubrics for rating level of implementation of inquiry-based science • Analyze results for significant relationships between level of implementation to student achievement RMC Research Corp.

  47. Assessment of Student Content Knowledge • Obtain permission to use Bellevue School District assessments for Grades 6 & 7. • Identify teacher participants • 25 first time teachers & 25 veteran teachers • Data collection for each student • Pre and post student scores • Student demographics • Special program participation • SPED, FRL, ELL, Migrant, etc. • Gift certificate compensation to teachers RMC Research Corp.

  48. Questions ! Dave Weaver RMC Research Corporation111 SW Columbia, Suite 1200Portland, Oregon 97201 (800) 788-1887 dweaver@rmccorp.com RMC Research Corp.

More Related