1 / 20

Establishing A Passing Standard

Establishing A Passing Standard. Paul D. Naylor, Ph.D. Psychometric Consultant. Standard Setting. The process used to arrive at a passing score Lowest score permitting entry to the field Recommended Standard. Validity is About Interpretation. Evidence to support the interpretation

linnied
Download Presentation

Establishing A Passing Standard

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Establishing A Passing Standard Paul D. Naylor, Ph.D. Psychometric Consultant

  2. Standard Setting • The process used to arrive at a passing score • Lowest score permitting entry to the field • Recommended Standard Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  3. Validity is About Interpretation • Evidence to support the interpretation • What standard setting system was used? • What was the rationale for selecting it? • What was the qualification of those who participated? Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  4. Standards • Mandated • Norm-referenced • Criterion-referenced Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  5. Mandated Standards • Often used in licensing • Difficult to defend • Not related to minimum qualification Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  6. Norm-referenced Standards • Popular in schools • Limits entry • Inconsistent results Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  7. Criterion-referenced Standards • Wide acceptqnce in professional testing • Determines minimum qualification • Not test population dependant • Exam or item centered Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  8. Procedures • Angoff (modified) • Nedelsky • Ebell • Others Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  9. Minimally Competent Performance • Minimum acceptable performance • Minimal qualification • Borderline • It’s all relative Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  10. Angoff Method • Judges • Selection • Training • Probabilities • Would vs Should • Rater Agreement • Tabulation Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  11. 1. How many goals are on a basketball court? A. 1 B. 2 C. 3 D. 4 Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  12. 2. How many halves are in a college basketball game? A. 1 B. 2 C. 3 D. 4 Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  13. 3. How many quarters are in a high school basketball game? A. 1 B. 2 C. 3 D. 4 Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  14. 4. How many points does a 3- point field goal account for in a basketball game? A. 1 B. 2 C. 3 D. 4 Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  15. 5. How many teams are in the NCAA Men’s Basketball National Championship Tournament? A. 48 B. 64 C. 65 D. 32 Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  16. Application • Adjustment • Angoff Values • Alternative Forms of Exam • Passing Score Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  17. Validity Evidence • How was the passing score set? • Who set the passing score? Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  18. Minimum Requirements • Description of method • Results of method used • List of participants and qualifications • Demographics • Credentials • Affiliations Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  19. Additional Evidence • Data from method used • Description of training given to participants • Definition of minimum qualification Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

  20. References • Livingston, S.A. & Zieky, M.J. (1982). Passing Scores. ETS. • Cizek, C.J. Standard setting guidelines. Educational Measurement: Issues and Practices, 15 (1), 13-21, 12. • CLEAR Exam Review (Winter 2001, Summer 2001, and others) Presented at CLEAR's 2004 Annual ConferencePresented at the 2004 CLEAR Annual Conference September 30 – October 2 Kansas City, Missouri

More Related