1 / 33

Applying SGP to the STAR Assessments

Applying SGP to the STAR Assessments. Daniel Bolt Dept of Educational Psychology University of Wisconsin, Madison. Some Unique Features of STAR Assessments. Same CAT assessment is administered within and across years (possible to generate look-up tables for the calculation of SGP)

gillian-henry
Download Presentation

Applying SGP to the STAR Assessments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applying SGP to the STAR Assessments Daniel Bolt Dept of Educational Psychology University of Wisconsin, Madison

  2. Some Unique Features of STAR Assessments • Same CAT assessment is administered within and across years (possible to generate look-up tables for the calculation of SGP) • Multiple STAR administrations to a student are possible throughout the year, and can occur at different times and/or frequencies for different students

  3. Why SGP may be Useful with STAR Assessments • Different STAR scores may be associated with different amounts of measurement error (e.g., extremely low or high STAR scores are sometimes of questionable validity) • There often exists more/less variability in growth observed across students at different initial STAR score levels

  4. Some Practical Issues Related to Administration of STAR Assessments • How frequently and at what intervals should STAR be administered to get reliable estimates of end-of-year scores? Does the answer depend on the initial (fall) score of a student? • Is there practical value in the use of SGP for answering this question (in contrast to alternative approaches, such as ordinary least squares---OLS--- regression methods)?

  5. Goals of the Present Study • Examine SGP as a methodology for quantifying growth and for studying the precision of end-of-year growth predictions using STAR Assessments • Compare SGP against competing methodologies (OLS regression) in terms of their reported precision of end-of-year predictions

  6. Sample Sizes (Students with Fall, Winter & Spring Assessments)

  7. Evaluating the Precision ofSpring Score Predictions How well do winter assessments improve our predictions of end-of-year outcomes? Are the winter assessments more/less useful depending on the fall score obtained by the student? In answering these questions, we find it useful to examine changes in the confidence intervals for spring scores defined by the SGP percentile cuts

  8. Example of SGP Percentile Cuts, One Covariate

  9. Example of SGP Percentile Cuts, Two Covariates

  10. Comparison of SGP & OLS Intervals, Math Grade 1 According to SGP %ile cuts According to OLS Standard Error of Prediction

  11. STAR Math, Grades 1-6 SGP OLS

  12. STAR Math, Grades 7-12 SGP OLS

  13. STAR Reading, Grades 1-6 OLS SGP

  14. STAR Reading, Grades 7-12 OLS SGP

  15. STAR Early Literacy, Grades K-4 OLS SGP

  16. Adding Winter Scores as Covariates with SGP • Using 80% interval width curves as a baseline, we can further examine how much the intervals are reduced when adding a winter assessment • The decline in the 80% interval can be used as an indicator of the added precision provided by the winter assessment

  17. STAR Math Example

  18. STAR Reading Example

  19. STAR Early Literacy Example

  20. Some Examples of STAR Score Patterns

  21. STAR Math Example SGP OLS

  22. STAR Math, Grade 1-6 SGP OLS

  23. STAR Math, Grades 7-12 OLS SGP

  24. STAR Reading Example SGP OLS

  25. STAR Reading, Grades 1-6 OLS SGP

  26. STAR Reading, Grades 7-12 OLS SGP

  27. STAR Early Literacy Example SGP OLS

  28. STAR Early Literacy, Grades K-4 OLS SGP

  29. Evaluating Predicted Spring Scores in Terms of State Proficiency Thresholds • The accuracy of SGP and OLS predictions can also be compared against the thresholds associated with state-specific proficiency categories • By assuming normally distributed residuals (with constant variance) for OLS, SGP and OLS can each be used to define a probability that the spring score will exceed a predefined threshold

  30. Example of SGP Percentile Cuts against State-Defined Proficiency Category Proficiency Threshold

  31. Comparing SGP and OLS on Accuracy of Proficiency Predictions • aEstimated STAR cutscores for the Tennessee Comprehensive Assessment Program (TCAP) • bEfron’s Pseudo R2

  32. Extending SGP to Accommodate Multiple Intermediate Assessments How can additional intermediate assessments be used in SGP to further improve predicted spring scores? Challenge: Handling varying-time point assessment schedules One possible solution: Linear interpolation to fixed node locations

  33. Conclusions and Future Directions • Our SGP analyses suggest substantial variability in the precision of spring score predictions for STAR Math, Reading and Early Literacy depending on fall scores • There is clear value in incorporating winter assessments into SGP---the largest value occurs for students with extreme fall scores in STAR Math, intermediate fall scores in STAR Reading • More experimentation needed to determine how best to make use of multiple intermediate assessments within SGP

More Related