1 / 21

MTLE Passing Score-Setting

MTLE Passing Score-Setting. Board of Teaching October 15, 2010. Passing Score-Setting Agenda. Regulatory overview Review of test development activities Standards & Rules input Score-setting discussion and action Panel-based passing score recommendations

faith-rasmussen
Download Presentation

MTLE Passing Score-Setting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MTLE Passing Score-Setting Board of Teaching October 15, 2010

  2. Passing Score-Setting Agenda • Regulatory overview • Review of test development activities • Standards & Rules input • Score-setting discussion and action • Panel-based passing score recommendations • Additional considerations: all fields • Field-specific considerations

  3. Minnesota Law • 122A.09, Subdivision 4(b) and (e) – requirements for basic skills testing in reading, writing, mathematics; requirement for pedagogy and content knowledge tests; requirement for reading standards • 122A.18, Subdivision 2(b), (c), (d) – requirement for basic skills tests; requirement for remediation; requirement for legislative reports; requirements /conditions for issuing one-year licenses to candidates who have not passed the basic skills tests; requirement for making the basic skills a requirement to earn a first continuing license • Note: The legislative history suggests that this does not mean that all teachers must take the basic skills, but rather that people renewing a license under Subdivision 2(c) would not be able to earn a continuing license until they had passed the basic skills. (Session Law 1994: Chapter 647, Article 6, Section 14) • 122A.23, Subdivision 2 (b), (d), (e) – requires out-of-state applicants to pass Minnesota’s licensure examinations

  4. Minnesota Law • 122A.18, Subdivision 2a.Reading strategies. (a) All colleges and universities approved by the Board of Teaching to prepare persons for classroom teacher licensure must include in their teacher preparation programs research-based best practices in reading, consistent with section 122A.06, subdivision 4, that enable the licensure candidate to know how to teach reading in the candidate's content areas. These colleges and universities also must prepare candidates for initial licenses to teach prekindergarten or elementary students for the assessment of reading instruction portion of the examination of licensure-specific teaching skills under section 122A.09, subdivision 4, paragraph (e).

  5. Minnesota Law • 122A.09, Subdivision 4(e) The board must adopt rules requiring candidates for initial licenses to successfully complete an examination of general pedagogical knowledge and examinations of licensure-specific teaching skills. The rules shall be effective by September 1, 2001. The rules under this paragraph also must require candidates for initial licenses to teach prekindergarten or elementary students to successfully complete, as part of the examination of licensure-specific teaching skills, test items assessing the candidates' knowledge, skill, and ability in comprehensive, scientifically based reading instruction under section 122A.06, subdivision 4, and their knowledge and understanding of the foundations of reading development, the development of reading comprehension, and reading assessment and instruction, and their ability to integrate that knowledge and understanding. • NOTE: The amendment to subdivision 4, by Laws 2009, chapter 96, article 2, section 17, is effective May 17, 2009, and applies to teacher candidates beginning February 1, 2012. Laws 2009, chapter 96, article 2, section 17, the effective date.

  6. Minnesota Rule 8710.0500: Examinations for Teacher Licenses • Subpart 1A – requirement for basic skills testing in reading, writing, mathematics • Subpart 1B – requirement for pedagogy and content knowledge tests; requires tests to be adopted by the BOT; requires content test for additional fields of licensure, but new fields do not require another pedagogy test • Subpart 2 – requirements for selection and adoption of a testing system • Subpart 3(1) – requires BOT to set passing scores; requires candidates to achieve passing scores • Subpart 4 – requires passing scores to be published in the State Register and BOT to notify MN colleges and universities of passing scores

  7. Minnesota Rule 8710.0500: Examinations for Teacher Licenses • Subpart 5 – allows MN approved programs to attest to the passing status of applicants • Translation: The folks in Educator Licensing do not receive score reports to verify passing the tests for MN completers; the recommendation from an approved program includes this as a requirement. • Subpart 6 – provides guidance on test administration, scoring, and reporting • Subpart 7 – allows candidates to be charged a fee for testing

  8. Minnesota Rule 8710.0500: Examinations for Teacher Licenses • Subpart 8 – requires candidates to attempt the basic skills tests prior to beginning their teacher preparation program • Subpart 10 – allows for retakes of tests • Subpart 11 – requires out-of-state applicants to meet MN’s testing requirements • Subpart 12 – requires BOT to periodically review the testing system and to publish changes in the State Register

  9. MTLE Design: RIGOR • History: Social Studies broad-field licensure test • MTLE sub-test model inherently increases the rigor and accountability for more narrow bands of content within a licensure field • For example: • Elementary content knowledge • 5-12 Mathematics content knowledge

  10. Test Development Activities • Summer 2009: Program planning • Fall 2009 • Advisory Committees established • Bias Review Committee (BRC) – 35 participants • 37 Content Advisory Committees (CAC) - 300 participants • Targeted a 70/30 split: licensed teachers/higher education faculty members • Test frameworks developed • Reading stakeholder meeting • Content Advisory Committee preliminary online framework review • Bias Review Conference for framework review

  11. Test Development Activities • Fall-Winter 2009-2010: Item banks developed • February 2010: Test Materials Review Conference • Bias Review Committee (4 days) • 24 participants • Item review • 9,200+ items reviewed!! • Content Advisory Committees (8 days) • Framework review AND item review • 260 participants; 22 institutions represented • Spring 2010: Content Validation Surveys

  12. Test Development Activities • April 2010 • Content Validation online surveys • 200 randomly selected teachers per licensure field • 100 randomly selected faculty members per licensure field • Survey analysis • Field Testing • 22 institutions participating • Statistical analysis of results to finalize item banks • Out of 5,550 items field tested, 22 items were deleted

  13. June Passing Score Activities • Composition of panels • Content-specific • Targeted 70/30 split of teachers and faculty members • Average of 10 members per panel • Many, but not all, had been part of earlier test development work • Types of panels • Multiple choice item banks • Marker response selection for constructed response items

  14. June Passing Score-Setting Activities • Training provided to panel members • Recognition of performance: “would” vs. “should” • Reasonable expectation to begin practice • Reasonable expectation to maintain public trust • 2 rounds of passing score judgments: “Imagine a hypothetical group of individuals who are just at the level of knowledge and skills required to perform the job of an educator receiving an initial license in Minnesota. What percent of this group would answer the item correctly?”

  15. June Passing Score-Setting Activities • The result: • An item-by-item review by Minnesota experts • Typically, panelist item judgments fell within a range of 3-5 rating scale points (on a scale of 1-10) • A median of all ratings computed to provide a measure of expected performance of acceptably qualified for each item • A formula-based computation applied to the item ratings to generate a recommended score

  16. BOT Passing Score-Setting • Information available by subtest * • # of scorable items • Standard Error of Measurement • Panel-based passing score recommendation • Score points at -2 SEM, -1 SEM, +1 SEM, +2 SEM • Relationship between recommended raw score performance and # of scorable items * Remember: The subtest model inherently increases rigor and accountability for content.

  17. BOT Passing Score-Setting • Additional information / considerations • Impact data where n>10 in the September administration • Test objectives aligned to BOT standards • New test structure, format for candidates • Expanded study guides available for K-6, Early Childhood, Basic Skills; limited preparation materials in other fields • Structure of multiple subtests increases content accountability

  18. BOT Passing Score-Setting • Field-specific considerations • Basic Skills • New reading standards • Elementary and Early Childhood • 19 content areas • Revised standards • Reading Teacher • Library Media • Computers, Keyboarding, and Related Technology Applications • Vocal and Instrumental Music test • World Language and Culture tests

  19. MTLE Test Fields • Basic Skills • Reading Subtest • Writing Subtest • Mathematics Subtest • Pedagogy Tests: • Early Childhood (Birth to Grade 3) • Elementary (Grades K-6) • Secondary (Grades 5-12)

  20. MTLE Test Fields Content Area Tests: Agricultural Education (Grades 5-12) Business (Grades 5-12) Chemistry (Grades 9-12) Communication Arts/Literature (Grades 5-12) NEW Computer, Keyboarding, and Related Technology Endorsement (Grades K-12) Early Childhood Education (Birth to Grade 3) Earth and Space Science (Grades 9-12) Elementary Education (Grades K-6) English as a Second Language (Grades K-12) Dance (Grades K-12) Family and Consumer Sciences (Grades 5-12) Health (Grades 5-12) Instrumental and Vocal Classroom Music (Grades K-12) Library Media Specialist (Grades K-12) Life Science (Grades 9-12)

  21. MTLE Test Fields Content Area Tests (continued): Mathematics (Grades 5-12) Middle Level Endorsements (Grades 5-8) Communication Arts/Literature Mathematics Science Social Studies Physical Education (Grades K-12) Physics (Grades 9-12) NEW PrePrimary Endorsement (Age 3 to Prekindergarten) Reading Teacher (Grades K-12) Social Studies (Grades 5-12) Special Education Core Skills (Birth to Age 21) Technology (Grades 5-12) Theatre (Grades K-12) Visual Arts (Grades K-12) World Language and Culture (Grades K-12): NEW Chinese (Mandarin), French, German, Spanish

More Related