Producer Licensing Public Hearing: Testimony

1. Producer Licensing Public Hearing: Testimony Christopher K. BeerSenior Content Developer August 17, 2010

2. Cut Scores • Purpose: to determine the point of “minimal competence” • This point is the pass/fail cut line • Determined by a committee of Subject Matter Experts (SMEs) selected and/or approved by the state’s insurance department • The committee conceptualizes what a minimally competent candidate “is” • They relate the knowledge/skills/abilities of this minimally competent candidate to the items

3. Cut Scores • Two primary methods to establish the cut score are used: • Angoff – used when statistics are not available • Bookmark – used when statistics are available • These methods are both an art and a science

4. Item and Test Development • SMEs review and write items at virtual and in-person test development meetings • SMEs are advised to write items on an 8th grade reading level (except for insurance terminology) • Items are edited and new items written and referenced directly to state law • Item performance statistics for each item are shared with the SMEs to assist them in their review • The meetings are always facilitated by a Pearson VUE content developer • This helps to ensure SMEs follow content development best practices

5. Item Bias and Difficulty • An analysis of item bias is performed in several states where required by law • Item and test difficulty are continually monitored by Pearson VUE’s Measurement team • Test forms that show disparate pass rates are flagged and “fixed”

6. Revalidation of Exam Content • General product knowledge items are revalidated every 2 to 4 years • State-specific items based on state laws and rules are revalidated every 1 to 2 years • How often the state-specific items are reviewed is specified by the state insurance department • Most states review their major lines items annually

7. Pre-licensing Education • Some states ask Pearson VUE to collect PE provider information from each candidate after they sit an exam via a survey • States decide whether to share pass rates by provider with the public

8. Standardization • The General Product Knowledge section: • Developed nationally via a job task analysis • SMEs licensed across all 51 jurisdictions participate • The same exam items administered across all client states • The right mix of general content applicable to all producer/agent candidates and state-specific knowledge important to each individual jurisdiction

9. Study Guides • Because it would be a conflict of interest, Pearson VUE does not develop study material for insurance licensing exams • A content outline is made public that serves as a “blue print” for content that may be tested on the exam • This is sent to publishers and providers several weeks before changes to the exams are made • It is posted to Pearson VUE’s website and inserted into candidate handbooks • The outline includes references to state laws and rules with which candidates are expected to be familiar