Assessing Student Growth to Foster School Excellence

1. Assessing Student Growth to Foster School Excellence Coalition for Psychology in Schools and Education Education Leadership Conference American Psychological Association September 18, 2006

2. Overview • Goal: To understand current uses of assessment in NCLB and to clarify consensus on how APA might influence NCLB reauthorization. • Presentations • Status, improvement, and growth models of school accountability (Jeff Braden) • Educational evaluation of English Learners: Issues and answers for measuring progress fairly (Sam Ortiz) • What is school excellence? Is it the same for everyone? (Gary Stoner) • Discussion: How should we work to maintain/change NCLB? (Steve Rollin)

3. Status, Improvement, and Growth Models of School Accountability Jeffery P. Braden, PhD Committee on Psychological Tests & Assessments North Carolina State University

4. Overview • Identify adequate yearly progress (AYP) definitions under No Child Left Behind • AYP models • Status • Improvement (aka “Safe Harbor”) • Growth • Promises and Problems with Growth Models

5. No Child Left Behind:Adequate Yearly Progress • AYP currently defined by three types of targets • Participation in annual state tests • Performance on state tests (% proficient) • One non-test indicator (attendance, graduation rate) • NCLB mandates testing for… • Reading & Mathematics (and in 2007, Science) • Grades 3-8, once in HS (but different for Science) • Specific populations w/in schools

6. AYP Targets Defined by… • Test outcomes = 4 dimensions • Participation (> 95%) & Performance (% proficient) • For two subject areas (Reading & Math) • 2 subject areas X 2 types of goals = 4 dimensions • Student groups = 9 • All students • Ethnicity (American Indian/Alaska Native, Asian/PI, black, Hispanic, white) • Economically disadvantaged • Limited English Proficient • Students with Disabilities

7. AYP Targets For A School • Pool data from all eligible grades • Determine if minimum sample size is reached for the 9 target groups • If yes, target applies • If no, target does not apply and data “roll up” to district, state • Compare test data to participation & performance goals for all eligible groups • Failure to miss any target for any group = failure to meet AYP for school

8. Reading Composite American Indian Asian Black White Hispanic Students with Disabilities Low Income LEP Math Composite American Indian Asian Black White Hispanic Students with Disabilities Low Income LEP AYP is determined by making it over all 18 hurdles (9 hurdles for reading and 9 for math) by disaggregation of data.

9. AYP = Status Model Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

10. AYP Safe Harbor: Improvement Model Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

12. Comparison of Models • Promise of status model • Easy to understand • Same goals for all (equity) • Leads to 100% proficiency by 2014 • Problems of status model • Fails to accommodate differences in student populations between schools • Fails to identify/reward progress toward proficiency • Increasingly difficult targets yield high rates of school failure

13. Growth Model Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

14. Comparing Models:A Hypothetical Example • North Carolina AYP Performance Targets • 76.7% annual target for Reading • 95% participation goals • Failing AYP • Not meeting AYP in three consecutive years • No meaningful improvement; fails to meet safe harbor provisions • Yet strong growth for students entering in 3rd grade over three consecutive years

15. M = 251 State Proficiency Level for 5th Grade = 247 M = 216

16. Additional Issues for Growth Models • Value added • Implies causality by attempting to factor out other influences (e.g., demographics, typical school gains) • What constitutes appropriate growth? • Progress toward proficiency standard • Improvement generally • Can growth be measured longitudinally? • Qualitative vs. quantitative change across academic domains, scaling issues

17. Using Growth Models for AYP • Promises • Accommodates differences between schools in student populations • Could reward progress • Potential linkage to progress monitoring • Problems • Capacity to measure & predict change over time • Mobility • Equity • Practicality

18. Educational evaluation of English Learners: Issues and answers for measuring progress fairly (Sam Ortiz) • What is school excellence? Is it the same for everyone? (Gary Stoner)

19. Discussion

20. Proposed Changes to NCLB • Drop 100% proficiency goal by 2014 • Replace with “goals that are ambitious, scientifically achievable, and increase regularly…” • Drop safe harbor (improvement) provisions • Replace with “growth concurrent with achieving proficiency on State annual assessments or on scientifically based measures for student progress monitoring (as defined by the National Center on Student Progress Monitoring)”

21. Proposed Changes to NCLB • Require states to align proficiency definitions to NAEP (prevents low/varying standards) • Increase test inclusion requirement for students in US schools from three to six years. • Alternate assessments • Change “significant [cognitive] disability” • Increase cap counted toward proficiency from 1% to 2% • Modified achievement standards cap from 2% to 3%