1 / 69

Review and Validation of ISAT Performance Levels for 2006 and Beyond

Review and Validation of ISAT Performance Levels for 2006 and Beyond. MetriTech, Inc. Champaign, IL. Objectives for State Testing Review Committee. Review cut score validation process Review modifications to grade 8 mathematics cut scores Review grade 5 mathematics cut score proposal

ely
Download Presentation

Review and Validation of ISAT Performance Levels for 2006 and Beyond

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Review and Validation of ISAT Performance Levels for 2006 and Beyond MetriTech, Inc. Champaign, IL

  2. Objectives for State Testing Review Committee Review cut score validation process Review modifications to grade 8 mathematics cut scores Review grade 5 mathematics cut score proposal Review new ISAT reporting scale proposal

  3. Purpose of the Validation Panel Review cutoff values in relationship to Performance definitions Assessment frameworks Representative item difficulties District/school performance profiles Judge the reasonableness of the proposed cutoffs

  4. Process Review development of ISAT performance levels Review equating process to SAT-10 vertical scale Consider proposed cutoffs in relationship to frameworks, definitions, data Evaluate the reasonableness of the outcomes

  5. Constraints NCLB requirements for continuity in school/district accountability Grades 3, 5, 8 frame possibilities for grades 4, 6, 7

  6. Outcome

  7. ISAT/SAT-10 Bridge Study

  8. Bridge Study Purposes SAT-10 will be the basis for the enhanced ISAT vertical scales Vertical scale will be the basis for establishing intermediate grade performance levels (reading/math grades 4, 6, 7)

  9. Description of the Equating Sample • 9793 SAT-10 Records • 9 districts • 47 schools • grades 3, 4, 5, 7, 8

  10. ISAT/SAT-10 Data Record Match

  11. Descriptive Statistics for Matched Samples: Reading Tests

  12. Descriptive Statistics for Matched Samples: Mathematics Tests

  13. Descriptive Statistics for Matched Samples: Science Tests

  14. Statistical Comparison of Matched Sample SAT-10 Means With SAT-10 Spring Norms

  15. Statistical Comparison of Matched Sample ISAT Means With ISAT 2005 Test Population

  16. Correlation Between Corresponding ISAT/SAT-10 Ability Estimates

  17. Equating Methods • Rasch (outliers included) • Rasch (outliers excluded) • Equipercentile • Equipercentile with linear smoothing

  18. Equating Outcomes • Rasch/Equipercentile results similar with respect to cut points • Rasch: 635.6 • Equipercentile: 635.3 • Outlier impact • Tended to eliminate higher ISAT/lower SAT-10 scores • Inconsistent with past practice

  19. Constants For Transforming ISAT Scales to the SAT-10 Vertical Scales

  20. Table for Transforming ISAT Scores to SAT-10 Vertical Scales

  21. Transformation of the Performance Level Cut Scores

  22. ISAT Performance Category Score Ranges on the Current Scales

  23. ISAT Cut Scores Expressed on the SAT-10 Vertical Scales (Lower Bounds)

  24. Reading Cut Scores Expressed on the SAT-10 Vertical Scale

  25. Science Cut Scores Expressed on the SAT-10 Vertical Scale

  26. Mathematics Cut Scores Expressed on the SAT-10 Vertical Scale

  27. Adjustment to the Grade 8 Mathematics Cut Scores

  28. Reasons for Reexamining the Cut Scores • Large Discrepancies Across Grades (Grade 3: 79%; Grade 5: 73%; Grade 8: 54%) • Multiple Panels Used in the Original Derivation (162, 149) • Discrepancies in National Percentile Ranks

  29. National Percentile Ranks Corresponding to Each ISAT Cut Score

  30. Estimating Performance Level Cut Scores for Intermediate Grades

  31. Reading Cut Scores With Interpolations for All Grades

  32. Mathematics Cut Scores With Interpolations for All Grades

  33. The Evaluation Process Review performance definitions and assessment frameworks Review difficulty-ordered item booklets Review district/school performance profiles Make initial judgments Large group discussion Final judgments

  34. Difficulty-Ordered Item Booklets Represent a range of probable performances by students at different levels Primarily represent the Below Standards/ Meets Standards performance range

  35. Difficulty-Ordered Reading Booklets

  36. Difficulty-Ordered Mathematics Booklets

  37. District 1 Performance Profile

  38. District 1 Performance Profile

  39. District 1 Performance Profile

  40. District 1 Performance Profile

  41. District 2 Performance Profile

  42. District 2 Performance Profile

  43. District 2 Performance Profile

  44. District 2 Performance Profile

  45. School 1 Performance Profile

  46. School 1 Performance Profile

  47. School 1 Performance Profile

  48. School 1 Performance Profile

  49. Evaluation Worksheet

  50. Evaluation Outcomes

More Related