Learning from Student Assessment Results: Lessons for New York State

1. Learning from Student Assessment Results: Lessons for New York State Richard J. Murnane and Nancy S. Sharkey Harvard University Graduate School of Education March 2004

2. Standards-Based Reform • Content and performance standards • Assessments • Instructional materials and professional development • Incentives for educators and students

3. New York State • Has strong standards and assessments • Pressure to meet standards: • High standards to earn a high school diploma • No Child Left Behind

4. Lessons for New York State The State should: • Provide student assessment results rapidly • Develop a formative assessment for districts • Continue on plan to develop a statewide student identification system • Compare NYC high stakes test scores to NYS and NYC NAEP scores • Examine effectiveness of BOCES

5. Uses of Student Assessment Results • Instrumental: To make programmatic decisions • Conceptual: To enrich dialog • Symbolic or Political: To support a predetermined stance or decision

6. Necessary Conditions for Effective Data Use • Assessments aligned to standards and curriculum • Technology needs to be user friendly • Teachers need to know how to do analyze data and translate findings into instruction • Teachers need time to do this work • School culture must support this work

7. New York City’s Challenge • More than 1 million students • 139,000 receive special education services • 126,000 are English Language Learners • 147 languages are spoken in the schools • In 2002-2003, 59% of elementary and middle school students did not meet state standards in reading; 58% did not meet standards in math

8. New York City’s Response • Assessment system • High stakes citywide exams in grades 3, 5, 6, 7 • Statewide high stakes tests in grades 4 and 8 • Citywide formative assessments 3 times/year • Monthly formative assessments in Region 2 • Handling data • The Grow Network • Princeton Review

9. The Grow Network • Summative assessment data reporting • Paper reports of how students performed by standard • Computer display mirrors paper report, with links to content and pedagogical support • Links for parents and students

10. Strengths and Limitations • Strengths • Reports are easy to read • Enable teachers to differentiate instruction • Curriculum neutral approach; geared to standards • Limitations • Timing • Limited questions and broad standards make standards-based analysis hard to interpret • Curriculum neutral approach

11. Princeton Review • Formative assessments three times/year • Test developed, scored, and analyzed • Tests online or on paper • Reports online only • Online content and pedagogical links • Links for parents and students

12. Strengths and Limitations • Strengths • Reports of performance against standards and by individual test item • Helps educators see where students are throughout the year • Aligned with state standards • Limitations • Assessments not linked to school curriculum • Technological hurdles • Hard to identify who is using reports (one teacher may print out the report for many other teachers)

13. The Princeton Review in Region 2 • Monthly formative assessments in math in grades 2 through 10 • Assessments on paper • Math consultants provide in-person professional development, based on needs identified on assessments

14. Common Concerns • Do scores on the assessments accurately reflect mastery of state learning standards? • Are teachers teaching to the test or teaching to standards? • Are formative assessments being used as low-stakes tests, high-stakes tests, or not at all?

15. Lessons for New York State The State should: • Provide student assessment results rapidly • Develop a formative assessment for districts • Continue on plan to develop a statewide student identification system • Compare NYC high stakes test scores to NYS and NYC NAEP scores • Examine effectiveness of BOCES