1 / 67

New Small Schools Evaluation Board Presentation

New Small Schools Evaluation Board Presentation. September 26, 2007. 2007-2008 School Portfolio Management Timeline. - 1 -. Small Schools Evaluation Agenda. PHASE 1 OF 2. I. Small Schools Movement in Oakland Historical Context Incubation Process II. External Evaluation

sara-monroe
Download Presentation

New Small Schools Evaluation Board Presentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Small Schools EvaluationBoard Presentation September 26, 2007

  2. 2007-2008 School Portfolio Management Timeline - 1 -

  3. Small Schools Evaluation Agenda PHASE 1 OF 2 • I. Small Schools Movement in Oakland • Historical Context • Incubation Process II. External Evaluation • Methodology • Results III. New Small Schools Investment IV. Reflections and Lessons Learned A New Small School is a school that is created around a common vision; has an explicit theory of action around closing the achievement gap; and develops a clear plan for a culture of high, non-negotiable expectations for student achievement. Small size is a facilitating condition that enables personalized learning structures for students; personalized involvement and leadership opportunities for families; and personalized learning structures for staff. - 2 -

  4. Part 1: Small Schools Overview HISTORICAL CONTEXT • Oakland’s Small Schools Movement was community-based and led by parents organized primarily by OCO. • Parents wanted an alternative to some of the most overcrowded and lowest performing public schools in California. • Parents and teachers were inspired by visits to successful small schools in Chicago & New York. • OUSD School Board unanimously passed the New Small Autonomous Schools (NSAS) policy in May 2000, and 9 schools were opened as a result. • In 2003 OUSD moved to full implementation and has opened 48 new small schools to date. • New small school creation is one of six foundational strategies for Expect Success. Graphic: Courtesy of Oakland Community Organizations (OCO) - 3 -

  5. Part 1: Small Schools Overview 48 New Small Schools in Oakland: 2000 - 2007 Fall 2004 Fall 2003 Fall 2000 Fall 2005 Fall 2006 Fall 2001 Fall 2007 Fall 2002 Coliseum College Prep West Oakland MS Bridges Academy BEST HS Sankofa Academy KIPP Greenleaf Elementary MetWest CBITS Explore ACORN Woodland *School of Social Justice & Community Development Reach Academy New Highland Roots Int’l Learning Without Limits Futures Elementary RISE Community School EnCompass Academy Leadership Prep Think College Now *Kizmet Academy Alliance Academy ASCEND Elmhurst Community Prep Global Family School East Oakland School of the Arts *Community United Elementary *East Oakland Community High School Manzanita SEED Media College Prep Urban Promise International Community School Esperanza Elementary YES EXCEL HS United for Success Academy Mandela East Oakland PRIDE Peralta Creek MS Melrose Leadership Academy College Prep & Architecture *Closed *Opening 2007, Incubating 2007-08 *New York model, no OUSD incubation Life Academy Robeson School of Visual & Performing Arts Manzanita Community School *International HS Fred T. Korematsu Discovery Academy PLACE @ Prescott Graphic: Courtesy of OUSD New School Development Group - 4 -

  6. Part 1: Small Schools Overview Incubation Process: 2004-2007 New School Incubator Curriculum – 1st Semester Primary focus = to help design teams build background knowledge, conduct research and plan toward developing a thoughtful School Design Proposal that has a very explicit vision and a clearly articulated theory of action that addresses the needs of and ensures equitable outcomes for all children. Graphic: Courtesy of OUSD New School Development Group - 5 -

  7. Community Analysis Theory of Action Vision Part 1: Small Schools Overview OUSD INCUBATION PROCESS: 2004-2007 New School Incubator Curriculum – 2nd Semester May - Jun Mar - Apr Jan - Feb Instructional Program Instructional Program Graphic: Courtesy of OUSD New School Development Group SCHOOL DESIGN PROPOSAL = THE FOUNDATION - 6 -

  8. Part 2: External Evaluation of Small Schools RESEARCH QUESTIONS OUSD contracted with Strategic Measurement and Evaluation, Inc. to answer the following questions: - 7 -

  9. Part 2: External Evaluation of Small Schools Methodology

  10. Part 2: External Evaluation of Small Schools QUESTION #1: What is the demographic and academic profile of students attending New Small Schools? RELEVANCE: To identify the intended population at the New Small Schools METHODOLOGY: Analysis of the OUSD student data files focusing on gender, ethnicity, free/reduced lunch status, English language development status, disability status, and previous CST scores

  11. Part 2: External Evaluation of Small Schools SMALL SCHOOLS DEMOGRAPHICS in 2006-07 • Compared to the District overall, New Small Schools currently serve: • a greater percentage of Latino students (53% @ New Small Schools vs. 34% @ district) • a greater percentage of English Learner students (38% vs. 29%) • a greater percentage of students receiving free/reduced lunch than the district as a whole (78% vs. 69%). - 10 -

  12. Part 2: External Evaluation of Small Schools SMALL SCHOOLS STUDENT ACADEMIC PERFORMANCE • Compared to the District overall performance in 2005-06, New Small Schools serve: • a lower percentage of students designated as “Proficient” or “Advanced” based on state test scores (e.g. CST), and a higher percentage of students designated as “Below Basic” or “Far Below Basic”. *CHASEE results are based on 2005-06 10th grade performance. - 11 -

  13. Part 2: External Evaluation of Small Schools SMALL SCHOOLS ACADEMIC PERFORMANCE For the 2006-2007 school year, the New Small Schools had fewer students who performed at “Proficient” or “Advanced” on the CST and CAHSEE exams. Compared to Comparison schools, the New Small Schools had a lower average number of days absent per student and lower suspension rate. The differences are not statistically significant. - 12 -

  14. Part 2: External Evaluation of Small Schools STAKEHOLDER SURVEY RESPONSES FINDINGS: The New Small Schools had significantly higher Use Your Voice ratings from students, parents, and teachers than did the comparisons schools. The New Schools compared favorably to the District ratings. N=11,929 N=10,735 N=6,796 N=1,256 - 13 -

  15. Part 2: External Evaluation of Small Schools STAKEHOLDER SURVEY RESPONSES FINDINGS: The New Small Schools had significantly higher Use Your Voice ratings from students and teachers than did the comparisons schools in the categories of Clean Learning Environment, Safety, Caring and Support, and College Readiness. - 14 -

  16. Part 2: External Evaluation of Small Schools • QUESTION #2: • Is there evidence that the New Small Schools are accelerating student achievement? • RELEVANCE: On average, students who enter the Small Schools during 2006-07 generally demonstrate academic performance that are below the District average. How can we show that small schools truly accelerate student achievement in a way that factors in student initial academic achievement? • METHODOLOGY: • To show students performance relative to their starting point, a statistical model was created to link student demographic characteristics and previous year’s CST test performance to current year CST scores. • Use the model to calculate projected CST ELA and math scores for each student in each school year and grade level. • Compare average projected CST scores of students within each school to the average CST scores actually obtained by the same students. • If the average CST score obtained by a group of students is significantly higher than projected, there is evidence that the school may have accelerated student achievement. - 15 -

  17. Part 2: External Evaluation of Small Schools • SMALL SCHOOLS STUDENT PERFORMANCE • Student groups at the New Small Schools demonstrated accelerated CST ELA achievement 22% of the time and underperformed only 7% of the time. • Student groups at the Comparison Schools demonstrated accelerated CST ELA achievement only 16% of the time and significantly underperformed projected performance 25% of the time. A similar pattern was obtained on the CST math tests. • Both groups of schools tended to perform as projected on the CAHSEE. - 16 -

  18. Part 2: External Evaluation of Small Schools QUESTIONS #3: How do the outcomes achieved by the New Small Schools compare to the outcomes produced by the schools they replaced? RELEVANCE: By comparing outcomes for similar groups of students before and after they enrolled in the New Small Schools it is possible to track changes that can be attributed to the New Small Schools. METHODOLOGY: Sufficient data on attendance (days absent) and CST/CAHSEE test scores (percent of students proficient or higher) existed to make comparisons based on these variables. The family tree charts were used to identify the relevant transitions from a traditional school to New Small School(s). - 17 -

  19. Part 2: External Evaluation of Small Schools SMALL SCHOOLS PERFORMANCE FINDINGS: The percent of students labeled proficient or higher on the CST was compared as students moved from traditional schools to the New Small Schools. The chart below provides CST ELA data related to four transitions. Over time, these transitions generally show an increase in the percent of students performing Proficient or higher on both ELA and Math CST when compared to the larger school that they came from. *The “n” is based on the number of students with valid CST scores. - 18 -

  20. Part 2: External Evaluation of Small Schools COMPARISON SCHOOL PERFORMANCE FINDINGS:The chart below provides the percent of students labeled Proficient or higher on the CST ELA at the 10 comparison schools. The pattern of performance change over the course of four years between the Comparison schools varied by as much as 23 percentage points in the percent of students performing Proficient or higher. The range was between +10% points to -13% points. Overall, the 10 Comparison schools made a 3.1 point gain in the percent of Proficient and Advanced students in ELA over the course of 4 years. - 19 -

  21. Part 2: External Evaluation of Small Schools Graduation Rates: New Small Schools vs. Comparison Schools The percent of 2005-06 graduates (based on NCES calculations) in small high schools are higher than Comparison Schools, district, county, and state. The percent of small high school graduates who meet A-G requirements show more variability than Comparison Schools. - 20 -

  22. Part 2: External Evaluation of Small Schools New Small Schools vs. Comparison* Schools Academic Performance Index (API) Academic Performance Index (API) *Comparison School = School that students would have attended if the new school had not opened. - 21 -

  23. Part 2: External Evaluation of Small Schools Model Implementation & School Functioning QUESTION #4: Have the New Small Schools been successful at implementing the district-approved design models? Do the New Small Schools function effectively as organizations? RELEVANCE: To understand the degree to which the district-approved design model has been implemented at each site. Evidence of effective school functioning provides additional context from which to judge the success of design model implementation. METHODOLOGY: Content analysis of each original design proposal identified specific organizational and instructional elements associated with each New Small School. A customized survey was designed for teachers to assess the degree to which each element was present at a particular school. A second survey was developed to assess the degree to which the New Small Schools exhibited characteristics associated with effective organizations (e.g., clear communication, strong leadership, shared purpose, etc.).

  24. Part 2: External Evaluation of Small Schools School Functioning Model Implementation: Teachers at the New Small Schoolstended to agree that the schools attempted to implement the organizational and instructional elements outlined in the approved design model. Their average model implementation rating was 3.85 on a scale of 1 (no implementation) to 5 (full implementation). Model Implementation n=322 School Functioning:Teachers at New Small Schools tended to agree that their schools demonstrated characteristics associated with effective organizational functioning. Their average school functioning rating was 3.71, significantly higher than the functioning rating obtained from teachers at a sample of comparison schools (3.71 vs. 3.44). n=322 n=125

  25. Part 2: External Evaluation of Small Schools School Performance Over Time QUESTION #5: How do the number of years a New Small School has been open relate to measures of model implementation and effective school functioning? RELEVANCE: The time needed to demonstrate sufficient implementation and functioning informs strategies for incubation and district support. METHODOLOGY: The model implementation ratings and effective school functioning ratings were plotted by the number of years a New Small School had been open.

  26. Part 2: External Evaluation of Small Schools Model Implementation & School Functioning FINDINGS: The average model implementation ratings were not significantly different based on the number of years a New Small School had been open. The 12 New Small Schools opened in 2006-2007 compared favorably to older New Small Schools. Model Implementation School Functioning The average school functioning ratings were not significantly different based on the number of years a New Small School had been open. There was significant variation at the individual school level. The district may benefit from reviewing organizational and personnel factors associated with higher functioning schools.

  27. Part 3: External Evaluator Summary of Key Findings Most of the 39 new small schools open in 2006-07 appear headed in a positive direction: - 26 -

  28. One-Time Investment in New Small Schools Part 4: Small Schools Investment OUSD’s one-time financial investment in the creation of new small schools has relied on Expect Success funding. From the 2005-06 to the 2007-08 school year OUSD has invested $5.64M or, on average $201K per new small school. These dollars were invested in a number of activities including: • Incubator services, • Support for principals in training, • School improvement coaching services, and • Subsidies for the opening and closing of schools. * - Assumes that 28 new small schools have been opened between September 2005 and September 2007. - 27 -

  29. One-Time Investment in New Small Schools Part 4: Small Schools Investment In addition to the financial investment made by OUSD, there are also other intangible costs to new small school development. These costs primarily have an impact on the capacity of district staff to serve schools. These include: • New principal recruitment, • New teacher recruitment, • High impact on Service Org to support new staff (e.g. FS), • High impact on Administration, and • Facilities. - 28 -

  30. Part 4: Spending Per Student, Small vs. Other Schools Overall spending per student is higher among 1st year, new small schools due to categorical funding allocations that are driven by federal and state formulas based on the type of students in the school. OUSD Small School Cutoff Elementary – 360 students Middle – 300 students High – 400 students NOTE: Existing Small Schools + 2nd Year and Above New Small Schools are identified as those schools that fell below the small school cut-off as defined by OUSD. 1st Year, New Small Schools are those schools that are opening in the 2007-08 school year.

  31. Part 4: Subsidizing 1st Year and Existing Small Schools OUSD provided additional subsidies starting in 2006-07 through the Targeted Instructional Improvement Grant (TIIG). There are two types of subsidies that new, small schools received: (1) For being in their 1st or 2nd year of operation and (2) For being classified as a small school. Footnote: New, small schools and existing small schools received subsidies because their total enrollment fell below the small school cutoff established by OUSD.

  32. Part 4: Comparison of allocation methods The proposed “gap student” allocation method reverses which schools get the most subsidy Total subsidy allocation at a school Binary allocation method Illustrative “Gap student” allocation method 0 200 360 cutoff 100 # of students enrolled

  33. Part 5: District Reflections • New Small Schools bring stakeholders together in new ways to create the conditions for student success, which is a major but not singular step in raising student achievement. • There are a number of new small schools that have leveraged these conditions to accelerate student achievement. • Having established these preconditions in almost all the new small schools, it will now be critical to press forward with academic rigor and quality of instruction while holding school accountable for performance with adequate levels of support. • OUSD has shifted our small schools strategy over time, based on lessons learned, to focus on providing more structure and support for the incubation process and ongoing monitoring and intervention.   We believe that this approach will improve the performance of all of our small schools. • In order to continue to improve the effectiveness of new small schools we need to do a more in-depth analysis about how years of implementation, type of incubation, district support, and differences at elementary, middle and high school impact student achievement results. - 32 -

  34. Phase II Evaluation Focus Areas Measures of Success • Analysis of performance at elementary, middle and high school level in alignment with District academic goals especially college readiness and post-secondary experiences/ choices of students who attend new small schools. • Analysis of student growth for students who participate in new small school for 2 or more years, and movement between performance bands on CST, especially out of far below and below basic. • Analysis of teacher and leader recruitment, quality, and retention as both a measure of success and as a variable that impacts student achievement. • Further analysis of how and to what degree new small schools have increased family and community engagement and ownership at a school level. • Further analysis of how new small schools have addressed family and community concerns around equity issues, especially related to the overcrowding and school quality.

  35. Phase II Evaluation Focus Areas (continued) School Quality and Functioning • Analysis of how decision making to incubate, incubation approach, level of community engagement, options/ choice, age of school, and specific program design elements impact measures of success. • Increase depth of analysis of school fidelity to implementation models through on-site observations and evaluation using school quality rubrics. Financial • Analysis of the resources required for full initial implementation and on-going sustainability of new school models. • Study and illustrate the intended and unintended intersections and impacts of new small schools with Results Based Budgeting and service economy/ central office services. • Analysis of the investment per graduate of new small schools including the impact of increased graduates in our community.

  36. New Small Schools EvaluationBoard Presentation September 26, 2007 - 35 -

  37. Small Schools EvaluationAppendix - 36 -

  38. Performance Before and After Small Schools CONCLUSIONS: Before Small Schools Initiative and Expect Success This map illustrates OUSD schools in 1999 and their API performance range. Notice the number of red schools (n=42). OUSD Schools: 1999 API Performance Academic Performance Index (API) <500 500-599 600-699 700-799 >800 - 37 - - 37 -

  39. Performance Before and After Small Schools CONCLUSIONS: After Small Schools initiative and Expect Success This map illustrates OUSD schools in 2006 and their API performance range. Notice the number of red schools (n=3). Eight new Small Schools were opened in 2005-06. OUSD Schools: 2006 API Performance Academic Performance Index (API) <500 500-599 600-699 700-799 >800 *First-year schools in 2006-07 - 38 - - 38 -

  40. Performance Before and After Small Schools CONCLUSIONS: After Small Schools initiative and Expect Success This map illustrates OUSD schools in 2007 and their API performance range. Notice the number of red schools (n=5). Twelve new Small Schools were opened in 2006-07. OUSD Schools: 2007 API Performance Academic Performance Index (API) <500 500-599 600-699 700-799 >800 *First-year schools in 2006-07

  41. Performance Before and After Small Schools Academic Gains: New Small Schools vs. Comparison* Schools *Comparison School = School that students would have attended if the new school had not opened. - 40 -

  42. Academic Gains: New Small Schools vs. Comparison* Schools - 41 -

  43. Academic Gains: New Small Schools vs. Comparison* Schools *Comparison School = School that students would have attended if the new school had not opened. - 42 -

  44. Academic Gains: New Small Schools vs. Comparison* Schools *Comparison School = School that students would have attended if the new school had not opened. - 43 -

  45. Academic Gains: New Small Schools vs. Comparison* Schools *Comparison School = School that students would have attended if the new school had not opened. - 44 -

  46. Academic Gains: New Small Schools vs. Comparison* Schools *Comparison School = School that students would have attended if the new school had not opened. - 45 -

  47. - 46 -

  48. - 47 -

  49. - 48 -

  50. - 49 -

More Related