1 / 17

Measurement of Academic Growth of Individual Students Toward Variable and Meaningful Academic Standards

Measurement of Academic Growth of Individual Students Toward Variable and Meaningful Academic Standards. S. Paul Wright, William L. Sanders, June C. Rivers November 2005. Measurement of Academic Growth of Individual Students Toward Variable and Meaningful Academic Standards. 1. Introduction

jerome
Download Presentation

Measurement of Academic Growth of Individual Students Toward Variable and Meaningful Academic Standards

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measurement of Academic Growth of Individual Students Toward Variable and Meaningful Academic Standards S. Paul Wright, William L. Sanders, June C. Rivers November 2005 (C) SAS Institute Inc. 2005. All Rights Reserved.

  2. Measurement of Academic Growth of Individual Students Toward Variable and Meaningful Academic Standards 1. Introduction 2. Some Longitudinal (Growth) Models 2.1   Nested model 2.2   Cross-classified model 3. EVAAS Projection Methodology 3.1   The methodology 3.2   A connection to “growth models” 3.3   Simulations 3.4   EVAAS Projection Advantages 4. Using Projections to Enhance NCLB and Other Important Educational Objectives 5. Discussion/Conclusions (C) SAS Institute Inc. 2005. All Rights Reserved.

  3. Nested Model j = school i = student within a school, t = time Xij = student-level covariate Wj = school-level covariate Level-1: Ytij = π0ij + π1ij·t + εtij. Level-2: π0ij = β00j + β01j Xij + r0ij, π1ij = β10j + β11j Xij + r1ij. Level-3: β00j = γ000 + γ001 Wj + u00j, β01j = γ010 + γ011 Wj + u01j, β10j = γ100 + γ101 Wj + u10j, β11j = γ110 + γ111 Wj + u11j. (C) SAS Institute Inc. 2005. All Rights Reserved.

  4. A Simplified Combined Nested Model: Ytij = (γ000 + γ001 Wj + γ010 Xij + u00j + r0ij) + (γ100 + γ101 Wj + γ110 Xij + u10j + r1ij)·t + εtij. (C) SAS Institute Inc. 2005. All Rights Reserved.

  5. Simple Cross-Classified Model(no covariates) Level-1: Ytij = π0ij + π1ij·t + εtij. Level-2: π0ij = θ0 + r0i + u0j, π1ij = θ1 + r1i. Combined: Ytij = (θ0 + r0i + u0j) + (θ1 + r1i)·t + εtij. (C) SAS Institute Inc. 2005. All Rights Reserved.

  6. Cumulative Cross-Classified Model(with covariates) Level-1: Ytij = π0ij + π1ij·t + εtij. Level-2: π0ij = θ0 + β0 Xi + r0i + ∑j ∑h≤t Dhij (γ0 Wj + u0j), π1ij = θ1 + r1i. Combined: Ytij = [θ0 + β0 Xi + r0i + ∑j ∑h≤t Dhij (γ0 Wj + u0j)] + [θ1 + r1i]·t + εtij. Dhij=1 if student “i” was in school “j” at time “h”, Dhij=0 otherwise. (C) SAS Institute Inc. 2005. All Rights Reserved.

  7. EVAAS Projection Methodology Projected_Scorei = MY + b1(X1i − M1) + b2(X2i − M2) + ... MY, M1, etc. are estimated mean scores for the response variable (Y) and for the predictor variables (Xs). “Non-Standard” Features (vs. “regression”) • Not every student has the same set of predictors • Means represent “average schooling experience” • Hierarchical data structure (C) SAS Institute Inc. 2005. All Rights Reserved.

  8. EVAAS Projection Methodology Projected_Scorei = MY + b1(X1i − M1) + b2(X2i − M2) + ... Regression slopes: b = CXX−1 CXY CXX = Cov(X), pooled-within-schools CXY = Cov(X, Y), pooled-within-schools (C) SAS Institute Inc. 2005. All Rights Reserved.

  9. A Connection to “Growth Models” A simplified nested linear growth model: Level-1: Yti = π0i + π1i·t + εti. Level-2: π0i = β00 + r0i, π1i = β10 + r1i. Combined: Yti = (β00 + β10·t) + (r0i + r1i·t + εti) = μt + δti. Ci = var(δi) = Zi T ZiT + Iσ2 where σ2 = var(εti), assumed same for all “t” and “i”; T = var({r0i, r1i}), assumed same for all “i”; Zi has: a column of “1”s (intercept), a column of “t”s (slope). (C) SAS Institute Inc. 2005. All Rights Reserved.

  10. A Connection to “Growth Models” Linear Growth versus EVAAS: Linear GrowthEVAAS Data: Vertically linked Unrestricted Means: Linear Unrestricted Covariances: Structured Unstructured (C) SAS Institute Inc. 2005. All Rights Reserved.

  11. Simulations Data generation: Model:  Yti = (β00 + r0i) + (β10 + r1i)·t + εti  =  μt + δti, i = 1, …, 2500; t = 0, 1, 2, 3; β00 = 400; β10 = 100; μt = {400, 500, 600, 700}; σ2 = var(εti) = 52 = 25; τ00 = var(r0i) = 152 = 225; τ11 = var(r1i) = 52 = 25; τ01 = cov(r0i, r1i) = 0. (C) SAS Institute Inc. 2005. All Rights Reserved.

  12. Simulations Parameter estimation and projections: EVAAS: Two samples 1. For parameter estimation 2. For projections to t = 3 Linear Growth Model: Second sample only Parameter estimation using t = 0, 1, 2 Projections to t = 3 (C) SAS Institute Inc. 2005. All Rights Reserved.

  13. Simulations Results: Mean prediction error (bias): MPE = Σi [projected(Y3i) − Y3i] / 2500. Mean squared prediction error: MSPE = Σi [projected(Y3i) − Y3i]2 / 2500. (C) SAS Institute Inc. 2005. All Rights Reserved.

  14. Simulation Results Original data: μt = {400, 500, 600, 700}. • EVAAS: MPE = −0.11; MSPE = 65.1. • Growth: MPE = −0.19; MSPE = 65.1. Variation #1: μt = {400, 500, 600, 700}, τ11 = 0. • EVAAS: MPE = +0.14; MSPE = 32.0. • Growth: MPE = −0.19; MSPE = 32.0. Variation #2: μt = {400, 505, 605, 700}, τ11 = 0. • EVAAS: MPE = +0.14; MSPE = 32.0. • Growth: MPE = +8.15; MSPE = 98.3. Variation #3: μt = {400, 510, 610, 700}, τ11 = 0. • EVAAS: MPE = +0.14; MSPE = 32.0. • Growth: MPE = +16.48; MSPE = 303.5. (C) SAS Institute Inc. 2005. All Rights Reserved.

  15. EVAAS Projection Advantages • Test scores need not be vertically linked • No assumption about shape of growth curve • Missing values are easily handled • Massive data sets are readily accommodated (C) SAS Institute Inc. 2005. All Rights Reserved.

  16. EVAAS Projection Applications • NCLB Safe Harbor • Individual student counseling (C) SAS Institute Inc. 2005. All Rights Reserved.

  17. Issues Growth/projection versus Value-added models Modeling problems • Fractured student records • Varying test scales • Changing testing regimes (C) SAS Institute Inc. 2005. All Rights Reserved.

More Related