230 likes | 332 Views
Children Left Behind in AYP and Non-AYP Schools: Using Student Progress and the Distribution of Student Gains to Validate AYP. Kilchan Choi Michael Seltzer Joan Herman Kyo Yamashiro.
E N D
Children Left Behind in AYP and Non-AYP Schools: Using Student Progress and the Distribution of Student Gains to Validate AYP Kilchan Choi Michael Seltzer Joan Herman Kyo Yamashiro UCLA Graduate School of Education & Information StudiesNational Center for Research on Evaluation,Standards, and Student Testing (CRESST)
Research Questions • Are there schools that meet AYP yet still have children who are not making substantial progress? i.e., leaving some children behind? • Are there schools that do not meet AYP yet still enable students to make substantialprogress? • Do AYP schools achieve a more equitable distribution of student growth? Are students at all ability levels making progress in AYP schools? • Are there non-AYP schools that are reducing the achievement gap?
Sample • Large, Urban District in WA • 2,524 students • 2 time-point ITBS reading scores (Grade 3 in 2001 & Grade 5 in 2003) • Standard Errors of Measurement (SE) on ITBS reading scores (Bryk, et.al., 1998) • 72 schools • Average # students/school: 35 • Average % qualifying for FRPL: 36.4% • Average % Minority (African American, Native American, or Latino): 68.6%
AYP vs. Non-AYP schools In WA • School AYP decision made based on 4th grade performance on WA Assessment of Student Learning (WASL) • 51 schools made AYP; 21 did not make AYP in baseline year (2002), according to WA State Dept of Ed • Our study re-evaluates AYP and non-AYP schools with a new value-added model (an advanced hierarchical Modeling technique)
A New Methodology for School Effect / Accountability: Latent Variable Regression in Hierarchical Model • Additional Questions and Interest using LVR-HM • Move beyond school mean growth rates and examine hidden/underlying process • How equitably is student achievement distributed? (The distribution of student growth: Children Left Behind or No Child Left Behind) • Why is it that student achievement is distributed in a more equitable fashion in some schools than in other schools?
Distribution of Student Growth(Relationship between initial status and rate of change)
Why a New Value-Added Model (LVR-HM)? • Gains or Growth might be highly dependent upon a status at certain point of time (i.e., initial status) • Initial status can be a strong and important factor to “valued-added gain or growth” • New value-added gain or growth: • Adjusting student intake characteristics PLUS student initial difference • Adjusting school intake characteristics, policies and practice PLUS school initial difference • Thus, providing value-added gain or growth PLUS revealing the distribution of student achievement
Latent Variable RegressionHierarchical Model (LVR-HM) • Level 1: Time series within student Yti = 0i + 1iTimeti + titi ~ N (0, 1) • Estimating initial status and gain for each student i with standard errors • Level 2: Student level 0i = 00 + r0ir0i ~ N (0, 00) 1i = 10 + b(0i - 00) + r1ir1i ~ N (0, 11) Cov(r0i , r1i ) = 0 • Gain for student i is modeled as function of his or her initial status
Different Levels of Initial Status • Many ways to define performance subgroups based on initial status • Examined gains for 3 performance subgroups within each school • Defined by initial status • Hi Performers: 15 pts above the school mean initial status • Mean: School mean initial status • Low Performers: 15 pts below the school mean initial status
Estimating Expected Gains for Different Levels of Initial Status • We estimate expected (predicted) gain for each of the performance subgroups using LVR-HM • Model-based estimation, not separate group analysis • Point estimate of gain & its 95% confidence interval (statistical inferences) • Possible to estimate expected gains after controlling for factors that lie beyond school’s control (e.g., student SES, school compositional factors)
Expected mean gain in ITBS reading scores for AYP schools • Only 12 of 52 AYP schools have 95% interval above the district avg. • 1 AYP school’s 95% interval includes 0
Expected mean gain in ITBS reading scores for non-AYP schools • 2 Non-AYP schools have 95% interval above district avg.
Expected gain for low-performing students (AYP schools) • 7 AYP schools’ 95% interval 30 • 3 AYP schools’ 95% interval includes 0 (low performers make no gains)
Expected gain for low-performing students (non-AYP schools) • 5 Non-AYP schools have gains for low performers >20
Expected gain for high-performing students (AYP schools) • 9 AYP schools’ 95% interval 30 • 3 AYP schools’ 95% interval < 10 (high performers make little or no gains)
Expected gain for high-performing students (non-AYP schools) • 5 Non-AYP schools’ 95% interval 30 • 3 Non-AYP schools’ 95% interval < 10 (high performers make little or no gains)
Distribution of Gains Within A School • Type I: Substantial gain across all performance subgroups (e.g., no child left behind – ex: AYP school #8, non-AYP school #26) • Type II: No adequate gain for high performers; substantial gain for low performers (ex: AYP schools #19, non-AYP school #27) • Type III: No adequate gain for low performers; substantial gain for high performers (ex: AYP schools, non-AYP school #6 )
Different Growth By Performance Subgroups & Demographic Subgroups
Conclusions • Analyses using our alternative approach: • More informative picture of growth using individual, longitudinal student gains • More complete picture of how student growth is distributed within a school • Stimulate discussion among teachers and administrator to identify students in need earlier (Seltzer, Choi & Thum, 2003) • Encourage educators to think about achievement levels rather than (or in addition to) current subgroup categories - may be more productive and actionable