1 / 14

Symposium: Assessment, Accountability, Instruction, and Learning in Urban Districts

Symposium: Assessment, Accountability, Instruction, and Learning in Urban Districts Research funded by the Joyce Foundation and Helen Bader Foundation Center for Systemic Reform in the Milwaukee Public Schools (SSR-MPS) Wisconsin Center for Education Research (WCER)

wyatt
Download Presentation

Symposium: Assessment, Accountability, Instruction, and Learning in Urban Districts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Symposium: Assessment, Accountability, Instruction, and Learning in Urban Districts • Research funded by the Joyce Foundation and Helen Bader Foundation • Center for Systemic Reform in the Milwaukee Public Schools (SSR-MPS) • Wisconsin Center for Education Research (WCER) • University of Wisconsin-Madison • 3 papers: • - William Clune, et al. Milwaukee Middle School Proficiencies • - Robert Meyer Value-added & School Performance • - Norman Webb Assessment Literacy • All focus on Milwaukee district • 3 Commentators: • - Warren Chapman Joyce Foundation • - Deborah Lindsey Milwaukee Public Schools • - Andrew Porter WCER • Papers on website: http://www.wcer.wisc.edu/mps

  2. The Milwaukee Middle School Proficiencies: Systemic school reform through high stakes assessments and a network of schools • William H. Clune, with Sarah Mason, Cecilia Pohs, • Chris Thiel, and Paula A. White • Paper prepared for the annual meeting of the AERA, • New Orleans, April 2, 2002 • Paper on website: http://www.wcer.wisc.edu/mps

  3. Overview of Points • What are the Proficiencies? • Research Methods • Impact on instruction and achievement • Centralized/ decentralized policy & implementation • Value of standardized and performance assessments • Importance of evaluation

  4. The Milwaukee Middle School Proficiencies: • What are they? • Promotion requirements from 8th grade, year • 1999-2000 through 2001-02 • 4 Areas: Communications, Mathematics, Science, • Research Paper • Multiple assessments, multiple opportunities to pass, • grades 6-8 • Purpose: assure readiness for high school (not just • pass or fail)

  5. Assessment Types • Traditional standardized tests (e.g., State of Wisconsin) • On-demand district performance assessments • Performance assessments embedded in instruction • (tasks, scoring rubrics) • Most weight put on embedded assessments • Can pass w/o "proficient" on State test • Alternative completion project (all 4 proficiencies) • (Adopt-a-City)

  6. Research Methods • 17 interviews in Spring 2000 • 11 school sites (range of reputed success) • 9 interviews with learning coordinators, 3 principals • 5 with respondents from school network ("Middle School Collaborative") • Taped, transcribed, coded with NUD*IST 4 • Names & schools not disclosed in paper

  7. Findings • Strong impact on instruction, learning, school • organization • Unclear impact on student achievement, weak • positive evidence • Centralized/ decentralized implementation/ • policy formation

  8. Finding 1: Strong Impact on Instruction, Learning, School Organization • Instruction •  Provided a focus for teaching (9 respondents) • Students re-do work w teachers (4) • Increased hands-on work (4) • Aligned with curriculum (9) • Learning •  Students: • - take greater responsibility (6) • - improved in writing (5) • - improved in reading (3) • - improved in math/ science (6) • - improved on standardized tests (2) • School organization •  Schools did major re-organization (4) • Special proficiency classes helpful (4) • Summer school programs helpful (3)

  9. Finding 2: Unclear Impact on Student Achievement, weak positive evidence • Intended to improve achievement (proficiency beyond assessments) • Performance assessments not statistically reliable • Annual 8th grade testing poor method of evaluation • Wisconsin 8th grade scores rose for two years then fell • Technical problems: change of test date, test forms • District study: increase in high school grades and 9th grade promotion • Also more transitional ("8-T") students in grades 8-9 • Much better: value-added from annual standardized tests (Meyer paper)

  10. Finding 3: Centralized/ decentralized implementation/ policy formation • In general • Unfolding requirements and implementation (incremental "roll out") • Much formative activity by District staff, Middle School Collaborative, • Learning Coordinators, Lead Principals • The Middle School Collaborative (network of Middle School Principals) • Had independent funding (Danforth) • Summer retreat for guiding vision ("all children can succeed") • Prevented repeal of Proficiencies • Successfully advocated alternative completion mechanism (Adopt-a-City) • Advocated fewer midstream policy changes

  11. Discussion/ Significance of Findings • Policy strength from the top and bottom • Tradeoff of measurement reliability and • instructional validity

  12. Discussion point 1: Policy strength from the • top and bottom • Good fit with Porter et al (1988) framework (authority, power, consistency, • specificity) • Importance of infrastructure in systemic reform (e.g., Clune, 2001) • At the top • - Authority (School Board, broad support for performance assessments) • - Power (promotion for students, high resources) • - Specificity (clear expectations for students) • - Consistency (coherent design across subjects, grades) • From the bottom • - Authority (Principals in Collaborative) • - Power (huge voluntary resources for implementation) • - Specificity (many details worked out between and w/i schools) • - Consistency (Vision and details managed decentrally)

  13. Discussion point 2: Tradeoff Of Measurement Reliability And Instructional Validity • Traditional standardized tests: high reliability, low • instructional validity • District performance assessments: low reliability, high • instructional validity • Standardized performance assessments rejected as too expensive • Two good options for district: • - Combination of both, but move to annual standardized testing for evaluation • - Revisit standardized performance assessments

  14. Conclusion: Importance of Evaluation • Important for both successful and unsuccessful • programs • To discontinue, continue, refine • Proficiencies used enormous resources (mostly labor) • Intensive resources = political vulnerability = need for • good evaluation

More Related