1 / 16

What (the Heck) Does “Proficient” Mean? Standard Setting on the MCAS Alternate Assessment

What (the Heck) Does “Proficient” Mean? Standard Setting on the MCAS Alternate Assessment. Daniel J. Wiener Massachusetts Dept. of Education. CCSSO - San Antonio, TX June 2003. Standard Setting: Important Decisions. Naming and defining “Performance Levels” Which raw scores count for PL?

cdarlene
Download Presentation

What (the Heck) Does “Proficient” Mean? Standard Setting on the MCAS Alternate Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What (the Heck) Does “Proficient” Mean? Standard Setting on the MCAS Alternate Assessment Daniel J. Wiener Massachusetts Dept. of Education CCSSO - San Antonio, TX June 2003

  2. Standard Setting: Important Decisions • Naming and defining “Performance Levels” • Which raw scores count for PL? • How will raw scores be combined to get a PL? • What combination of scores yields which PL?

  3. MCAS-Alt Performance Levels:Describe how well student has learned the General Curriculum? • Awareness: Very little, if at all • Emerging: Just getting the hang of it • Progressing : Moving right along on challenging standards, but below grade level • Needs Improvement: Doing okay at grade level • Proficient: Doing well at grade level • Advanced: Excelling at grade level

  4. Which Scores Count Toward the Performance Level? Based on stakeholders’ advice, “Parallel the General Assessment” Therefore, Rubric scores in: • Complexity, Accuracy, and Independence are combined to determine an overall Performance Level; • Not Self-Evaluation, Generalized Performance

  5. Two Possible Ways to Combine Rubric Scores EITHER: 1) Calculate a total numerical score, based on individual rubric scores; then, set cut score for each Performance Level Example: Complexity X (Accuracy + Independence) = Total score

  6. Two Possible Ways to Combine Rubric Scores (continued...) OR: 2) Describe the characteristics of student performance for all score combinations (no total numerical score) Example: What does a score of LC=3, DSC=4, Ind=3 indicate about a student’s performance?

  7. Student Performance: A Balancing Act Complexity Accuracy Independence LOW HIGH

  8. Performance Level: Awareness Complexity Accuracy Independence LOW HIGH

  9. Performance Level: Awareness Complexity Accuracy Independence LOW HIGH

  10. Performance Level: Emerging Complexity Accuracy Independence LOW HIGH

  11. Performance Level: Emerging Complexity Accuracy Independence LOW HIGH

  12. Performance Level: Progressing Complexity Accuracy Independence LOW HIGH

  13. Analytical Rubric: Determine Performance Level from Sub-scores If Level of Complexity = 3 (‘entry points’), and Demonstration of Skills & Concepts (Accuracy) = Independence =

  14. Calculating an Overall Performance Level • Average the Performance Level subscores in each content area StudentStrandPerformance Level #1#2#3 A Aw (1) - Aw (1) - Em (2) = Awareness (ave: 1.33) B Aw (1) - Em (2) - Em (2) = Emerging (ave: 1.67) CEm (2) - Pg (3) - NI (4) = Progressing (ave: 3.0)

  15. So, What Is Proficient, Anyway? When the student: • Addresses challenging, achievable outcomes, based on academic standards • Demonstrates knowledge and skills with >50% accuracy and independence

  16. Changes Since 2001 • Fine-tune the Analytical Rubric • We know more than we did in 2001 • Report “Incomplete” portfolios separately • Previously reported as Awareness (lowest score)

More Related