1 / 23

Overview of Standard Setting

Overview of Standard Setting. Leslie Wilson Assistant State Superintendent Accountability and Assessment August 26, 2008. Purpose of Presentation. To present the standard setting process To provide background information to the Board in preparation for setting standards for

ojewett
Download Presentation

Overview of Standard Setting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of Standard Setting Leslie Wilson Assistant State Superintendent Accountability and Assessment August 26, 2008

  2. Purpose of Presentation • To present the standard setting process • To provide background information to the Board in preparation for setting standards for • Modified High School Assessments • Alternative Maryland School Assessments

  3. Setting Performance Standards Content Standards Voluntary State Curriculum Cut Scores that match students to their appropriate Performance Categories The Tests Expertise: Student Knowledge/ Subject Matter Setting Performance Standards

  4. The Threshold Student • A borderline or minimally qualified student • In terms of performance • The least qualified of the Proficient or Advanced students

  5. Mod-HSA Test Performance Standards threshold students Basic Proficient Advanced Recommended Standards

  6. State Board of Education Set Standards Standard Setting Work Flow Content Groups Psychometric Council Review & Articulation Committee State Superintendent

  7. Step 1: Content Groups Participants all have appropriate content and grade-level expertise • Include representatives from local Central Offices as well as school level staff • Participate in separate content and grade level discussions

  8. Responsibilities of Content Group • Review Assessment • Discuss what barely proficient and advanced students know and can do • Propose cut scores and discuss • Make recommendations to Review and Articulation group

  9. Content Group Process: Round 1 • Break into small groups to encourage independent decision-making • Read, discuss ordered-item booklets • Individuals vote (privately) for a cut score • Small group discussion of votes

  10. The Process: Round 2 • Individuals vote (privately) for a cut score • Small group discussion of votes • Examine, discuss impact data

  11. The Process: Round 3 • Small groups converge • Individuals vote (privately) for a cut score • Discussion • Individuals vote (privately) for a cut score • The median cut score is the committee’s recommendation.

  12. A 15 14 13 12 11 10 9 P 8 7 6 5 4 3 2 1 Ordered Item Booklet Bookmark Standard-Setting Procedure Test items students must know and be able to do to be considered Advanced Test items students must know and be able to do to be considered Proficient

  13. Standard-Setting Process Individual Analysis Content every student should know Items most students get correct Items fewer students get correct

  14. Standard-Setting Process Round 1 Median Review test; vote Discuss data and distribution Round 2 Median Vote View impact data and discuss

  15. Standard-Setting Process Round 3 Median Small groups join; vote Median cut score is committee’s recommendation

  16. Step 2: Psychometric Council • Nationally-recognized experts • Trained in measurement and psychometrics • Independent • Familiar with Maryland tests • Similar to Technical Advisory Panels in other states

  17. Psychometric Council Responsibilities • Reviews work of all groups • Ensures quality controls were followed • Ensures standard-setting process technically sound • Does not make judgments on whether standard is “good” or “bad” • Forwards comments and recommendations to Review & Articulation Committee

  18. Step 3: Review and Articulation • Superintendents • Assistant Superintendents • Local Accountability Coordinators • Advocates for students with special needs (special education and ELL) • Parents • Local Boards of Education • Teachers Union representatives • Business Community members • MSDE

  19. Review & Articulation Committee Responsibilities include: • Reviewing work of all groups and Psychometric Council • Ensuring rigor is equivalent among grades • Reviewing articulation among grades • Forwarding recommendations to the State Superintendent

  20. Step 4: State Superintendent of Schools • Reviews work of all groups, Psychometric Council, and Review & Articulation Committee • Makes recommendation to the State Board of Education

  21. Step 5: State Board Approval • Reviews proposed standards • Approves standards for implementation

  22. Next Steps • Review and approve proposed standards to be used to determine proficient and advanced performance on the modified HSAs. The scores will be used to determine AYP for Maryland high schools next year. • Review and approve proposed standards for the Alt-MSA. Science will be reported this fall for the first time.

  23. Overview of Standard Setting Questions and Discussion

More Related