What Does “Proficiency” Mean for Students with Cognitive Disabilities

1. What Does “Proficiency” Mean for Students with Cognitive Disabilities Dr. Ron Cammaert Riverside Publishing Company Council of Chief State School Officers

2. Introduction • Goal - recommend performance thresholds • Variation on a holistic method - consider whole of individual’s work • Loosely based on Jaeger and Mills’ method (Cizek, 2001) Council of Chief State School Officers

3. Introduction • Classify student work into one of four categories • Category decided from a minimum subject rating and a minimum rating on generalization skills Council of Chief State School Officers

4. Description of Scores • Part I Content Area • Part II Generalized Skill • Total score = progress in content area (Part I score) + mod + set + soc+ self Council of Chief State School Officers

5. Level Total Score Part I Score Required 4 16 to 20 3 or 4 3 12 to 15 2 or 3 or 4 2 8 to 11 1 or 2 or 3 or 4 1 5 to 7 1 or 2 or 3 Decision Rule for Determining Level of Performance on WAAS portfolio *

6. Score  Level Con M/A S/C S R S D 16 4 3 4 4 3 2 16 3 2 4 4 3 3 16 2 1 4 4 4 3 Relationship Among Total Score, Level and Score Patterns

7. Initial Procedure • A sample of portfolios from 2001 and 2002 with adequate evidence selected • 14 portfolios selected - 32 different scoring patterns Council of Chief State School Officers

8. Step 1 - Training • Information about standard setting and how results will be used Council of Chief State School Officers

9. Step 2 - Review of the Assessment Material • Participation criteria for WAAS • Content validity evidence for the assessment and the scoring processes • Exemplars of student work for each score point • The Portfolio Scoring Summary • EALR Extensions Council of Chief State School Officers

10. Step 3 - Understanding the Definitions of the Standards • Brainstorm - students and student work that typifies each category • Academic Achievement Standard Descriptions Council of Chief State School Officers

11. Step 4 - Round 1 • Using the table of frequencies of scoring patterns and the descriptors for each standard panelists decided on the category (“meets” or “not meets” proficiency) for each score pattern Council of Chief State School Officers

12. Step 5 - Round 2 • Feedback on the panel cut scores • Given the set of 14 scored portfolios • Working individually reviewed each portfolio • Decisions for each portfolio and reset, if needed, the minimum score for each level • In groups discuss ratings • Individually completed the rating form for Round 2 Council of Chief State School Officers

13. Step 6 - Review of Impact Data • Statewide frequencies for each score pattern from 2002 portfolio scores • Panelists discussed the impact of standards • Gave final rating • The final cut scores the mean of the minimum total scores for each category Council of Chief State School Officers

14. Step 7 - Evaluation of the Standard-Setting Process • Questionnaire Council of Chief State School Officers

15. Documentation • Minutes from National TAC • Script developed • Observer during session – follow script • Technical Report Council of Chief State School Officers