1 / 16

Item Analysis

Item Analysis. What makes a question good??? Answer options?. Multiple Choice Questions. Elements of a good multiple choice question????. Item Analysis. How can we determine if it is a good question????? Distractor power Item difficulty Item discrimination. Distractor power.

naif
Download Presentation

Item Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Item Analysis What makes a question good??? Answer options?

  2. Multiple Choice Questions Elements of a good multiple choice question????

  3. Item Analysis How can we determine if it is a good question????? • Distractor power • Item difficulty • Item discrimination

  4. Distractor power Strength of each distractor (should be equal) # answering incorrect # of distractors

  5. E.G. • There are 4 possible answers • Option a is correct • if 9 people choose option b (9/3 = 3) • and 12 people choose option c (12/3 = 4) • and 15 people choose option d (15/3 = 5) • then option d is the strongest distractor

  6. Item Difficulty = p(percent of people passing) p = number answering correctly total number answering • If 100 people take the test and 34 get question 1 correct, what is the difficulty of that question? p1 = 34/100 = .34 • For 65 correct p would be 65/100 = .65 SO… the higher the p, the easier the question (score must be between 0 and 1)

  7. Item Difficulty is... • A behavioral measure • Based on the current group • Allows internal comparison • Also allows external comparison

  8. In construct based tests • Item difficulty is similar to item endorsement • Number agreeing with an item

  9. Item endorsement also... • A behavioral measure • Based on the current group • Allows internal comparison • Also allows external comparison

  10. Item discrimination = d Does the question discriminate between those who do well overall and those who do poorly? (or on a psychological test, those scoring high or low on the construct)

  11. Distribution To determine level upper (U) = top 25 to 33 % of scores lower (L) = bottom 25 to 33 % of scores

  12. Determining item discrimination d = U-L n where d = item discrimination U = number in the upper group who get question correct L = number in the lower group who get question correct n = number in one group (U or L)

  13. Examples of d • With 33 in U and L • if all in U and all in L get it correct 33 - 33 / 33 = 0 • if all in U get it correct, and none in L 33 - 0 / 33 = 1 • if none in U get it correct, and all in L 0 - 33 / 33 = -1 • if 30 in U and 10 in L get it correct 30 - 10 / 33 = .61

  14. Value of d • Range for d = -1 to 1 • higher d indicates more discriminating (1 = all in upper group got it correct and none in lower group) • negative d indicates a really bad question (-1 indicates all in lower group got it correct and none in upper group)

More Related