210 likes | 437 Views
EVALUATION & RESEARCH IN WAKE COUNTY (NC) PUBLIC SCHOOLS. Presented by David Holdzkom, Assistant Superintendent, Evaulation & Research Wake County Public Schools. CONTEXT.
E N D
EVALUATION & RESEARCH IN WAKE COUNTY (NC) PUBLIC SCHOOLS Presented by David Holdzkom, Assistant Superintendent, Evaulation & Research Wake County Public Schools
CONTEXT • WCPSS serves about 134,000 students, making it the 19th largest district in the U.S. and the largest district in North Carolina • The district has 93 elementary schools, 28 middle schools, 22 high schools and 4 special schools. • About 30% of students qualify for FR; 14% are SWD; and 7.4% are LEP.
CONTEXT (cont’d) • Wake County includes 12 incorporated towns plus suburbs and rural communities. • Since 1990, the county population has increased by 92%. • Much of this in-migration has been from areas of the country that have different traditions of school district organization. • Major employers in the county include IBM, GlaxoSmithKline, SAS, and Cisco • 3 research universities within 25 miles
ORGANIZATION OF EVAL&RES DEPARTMENT • Admin: Assistant Superintendent and 3.5 support staff; • Testing: Sr. Director and 2 professionals plus 4 support staff; • School Accountability: Sr. Director and 5 professionals plus 2 support staff; • Program Accountability: Sr. Director and 4 professionals plus 3 support staff; • Grants Admin: Sr. Director plus 2 support staff.
STUDENT INFORMATION RESOURCES • NC WISE: A comprehensive data base designed by state to provide information (demographic, achievement data) for every student. • Student Information Files: Maintained by WCPSS on a mainframe to provide supplementary data for every student. • Testing files: Designed by the state to provide both achievement and growth data for all students in Grades 3-8 plus selected high school course takers. • Local test files: Designed by WCPSS adds achievement data for students in K-2 as well as some additional information for students in Grades 3-5.
THE EFFECTIVENESS INDEX • Designed in WCPSS E&R, this analysis tool results from regression analyses for individual students that control for FR, concentration of FR in school, special education status, special education service method, etc. • Provided for each student, with reports aggregated at the class/course and school levels.
KINDS OF RESEARCH • Policy Evaluation: Healthy Schools, Magnet Schools, Student Assignment • Program Evaluation: Implementation of PBS, AVID, Standards-based grading, IRIS, etc. • School Status Evaluation: Accountability measures of state and NCLB. • Knowledge Creation: Effectiveness Studies; IRIS
Program Evaluation Standards • Utility Standards: Study serves the info needs of the audience. • Feasibility Standards: Study is realistic, diplomatic, and economical. • Propriety Standards: Study is legal and ethical. • Accuracy Standards: Study reveals and conveys technically adequate info.
POLICY EVALUATION • Healthy Schools: Values for 100 variables (both input and outcome) are calculated annually. This information is used by the Leadership Team to ensure that schools remain “healthy”. • Magnet Schools: Measure the degree to which Board goals for these schools are met. • Student Assignment: How well do different schools serve sub-groups of students?
Magnet Program Review (2007) • Review examined degree that board’s goals for magnets were realized: 1. Utilization of schools; 2. Use of choice to promote diverse student populations, reduce concentration of poverty; 3. Expand educational opportunity; 4. Promote program innovations.
Magnet Review (cont’d) • Examined student assignment data and student demographic for indications of race, FRL-status, assignment to schools. • Compared all schools with respect to concentration of FRL • Conducted focus group interview/survey of principals to identify innovative programs. • Examined differences of achievement between magnet/base populations, compared to district.
PROGRAM EVALUATION • Schools seek to implement nationally-developed programs (PBS, AVID, Reading Recovery) to meet specific needs. Evaluation focuses on fidelity of implementation as well as change in student outcomes. • Locally developed programs (Standards-based grading, PES, etc.) try to focus more on logic modeling and outcomes.
Evaluation of SES • A Title I school in year 2 of school improvement was required to offer Supplemental Education Services. • 252 students were provided tutoring service at a cost of $303,000. • Interviews were conducted with teachers and parents; student achievement data were examined.
SCHOOL STATUS EVALUATION • Accountability measures of state and NCLB. • Calculation of absolute measures of achievement (i.e., percentage of students at/above grade level) and calculation of growth. • Calculation of student residuals/effectiveness index.
Knowledge Creation • Effectiveness Studies: Using Effectiveness index to identify higher/lower performing teachers of 5 high school subjects, studied classroom practice; this method was adapted for elementary & middle schools • IRIS: An experimental program designed to provide reading intervention early. • Professional Learning Communities: Surveys intended to measure the quality/degree of implementation of PLCs in all schools.
Project IRIS • As part of an OCR consent decree, WCPSS agreed to analyze process of referrals to special education services and to design data-driven interventions to improve reading instruction. • Intensive Reading Intervention Study (IRIS) was developed to respond • The logic: too many poor readers are referred to student support teams and ultimately to special education. If we can improve reading early, referrals will decrease.
Project IRIS (cont’d) • IRIS involved random assignment of 169 students to Control or Treatment groups. • Treatment lasted for 45 minutes 4 times a week for ten weeks. • Students were tested on a variety of measures before treatment, after treatment and end of year. Controls had “mirror” assessments.
Project IRIS (cont’d) • At the end of the study, treatment group had more positive outcomes, but these did not reach statistical significance. • Three years later, students from both groups were identified and achievement data analyzed. Again, no significant differences were found.
RANDOMIZED TRIALS • Only 1 study (IRIS) has used randomization. • Teachers, principals, parents object to randomization: “students aren’t guinea pigs”. • How does intervention support/detract from the core mission of the school? • Currently pursuing a randomized trial project with Duke University related to PLCs.
FUTURE DIRECTIONS/ISSUES • Designing training in data use for principals & teachers and experimenting with delivery methods/formats • Increased emphasis on logic models and fidelity of implementation • Assisting schools to design accountability schemes that include but are not limited to test scores
FUTURE DIRECTIONS (CONT’D) • Enhancing teachers’ ability to develop formative assessment plans/items • Implementing the evaluation recommendations of the Curriculum Management Audit conducted in 2007