1 / 26

Maryland School Assessments (MSA) and High School Assessments (HSA) Standard-Setting Process

Explore the detailed process of setting standards for Maryland School Assessments and High School Assessments, including the review by the Psychometric Council and the State Superintendent. The standard-setting workflow involved multiple rounds of decision-making with various stakeholders to establish cut scores for proficiency levels.

sfairbanks
Download Presentation

Maryland School Assessments (MSA) and High School Assessments (HSA) Standard-Setting Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Standard Setting:MSA(Maryland School Assessments)IMAP(Independence Mastery Assessment Program)HSA(High School Assessments)Graduation Rate May 20, 2003

  2. Standard-Setting Work Flow 13 groups Psychometric Council Week of July 14 Review & Articulation Committee State Superintendent July 22-23 Set MSA Standards State Board of Education August 26-27 Set HSA Standards

  3. 13 Standard-Setting Groups • MSA Reading (4 groups) Grades 3,5,8,10 • MSA Math (4 groups) Grades 3,5,8 mathGeometry (formerly known as a High School Assessment) • High School Assessments (4 groups) English 1, algebra/data analysis, government, biology • IMAP (1 group)

  4. Standard-Setting Overview • Week of July 14, 2003 • Participants all had content, grade-level expertise. • Included teachers, principals, LEA staff, stakeholder groups, policymakers • Superintendents, State Board members, media, parents observed

  5. Standard-Setting Process • For each test, 20-25 participants • Several decision-making rounds

  6. Round 1 • Break into small groups to encourage independent decision-making. • Take the test. • Read, discuss ordered-item booklets. • Get familiar with the scale. • Individuals vote (privately) for a cut score. • Small group discussion of votes

  7. Round 2 • Individuals vote (privately) for a cut score. • Small group discussion of votes • Examine, discuss impact data.

  8. Round 3 • Small groups converge • Individuals vote (privately) for a cut score. • Discussion • Individuals vote (privately) for a cut score. • The median cut score is the committee’s recommendation.

  9. The Bookmark Standard-Setting Procedure A 15 14 13 Test items students must know and be able to do to be considered “Advanced” 12 11 10 9 P 8 7 6 5 4 3 2 1 Ordered Item Booklet Test items students must know and be able to do to be considered “Proficient”

  10. Standard-Setting Process Individual Analysis Content every student should know Items most students get correct Items fewer students get correct

  11. Standard-Setting Process Round 1 Median Review test; Vote. Discuss data and distribution. Round 2 Median Vote View impact data; Discuss.

  12. Standard-Setting Process Round 3 Median Small groups join; Vote. Median cut score is committee’s recommendation.

  13. Psychometric Council • Reviews work of all groups • Ensures quality controls were followed • Ensures standard-setting process technically sound • Does not make judgments on whether standard is “good” or “bad” • Forwards comments, recommendations to Review & Articulation committee

  14. Review & Articulation Committee • Reviews work of all groups, Psychometric Council • Ensures rigor equivalent among grades and subjects • Reviews articulation among grades and subjects • Forwards recommendations to the State Superintendent

  15. State Superintendent of Schools • Reviews work of all groups, Psychometric Council, Review & Articulation Committee • Makes recommendation to the State Board of Education

  16. State Board of Education Decision Timeline May 2003 Approved MSDE’s standard- setting approach July 2003 Set MSA standards August 2003 Set HSA standards September 2003 Consider tying HSA performance to diploma

  17. State Board of Education In July, the Board set MSA and IMAP standards for 2014, as required by No Child Left Behind. • MSA 3,5,8,10 reading 3,5,8 math geometry • IMAP 3,5,8, & high-school reading and math

  18. MSA: Percent of Maryland students meeting 2014 target in 2003 assessments

  19. IMAP: Percent of Maryland students meeting 2014 target in 2003 assessments

  20. Graduation Rate Target for 2014: 90%

  21. Next Steps for Maryland School Assessments… • Aug. 1- double check the data for accuracy at school and system level • Mid Aug.- determine status of 114 Title I Low Performing Schools (locals notify parents of options) • Aug. 22- MSA, IMAP, attendance, graduation rate data on line at www.mdreportcard.org • Late Sept.-Parent reports on MSA • Oct.- Parent reports on schools, systems, state • Late Oct.- Parent reports on 2003 HSA

  22. State Board of Education In August, the Board will set set High School Assessment performance standards. • English 1, algebra/data analysis, biology, government • Setting standards for 2008, 2011, 2014 graduating classes

  23. How will HSA standards be phased in? • Performance Level I 2004 entering ninth-graders, 2008 graduating class • Performance Level II 2007 entering ninth-graders, 2011 graduating class • Performance Level III 2010 entering ninth-graders, 2014 graduating class

  24. How will HSA standards be phased in? • Set performance standards I and III. • Calculate standard II. • Review performance standards, content alignment every five years.

  25. Standard by Entering and Graduating Classes

  26. More Information? • MSDE Hot Line • 1-888-246-0016 • MSDE Information Website • marylandpublicschools.org • MSDE School Improvement Website • mdk12.org • Md. School Report Card Website • mdreportcard.org

More Related