Download
1 / 21
210 likes | 230 Views
Learn how to identify and improve test items through item analysis, diagnose student errors, and enhance instructional design. Understand item difficulty and discrimination to optimize test validity and reliability.
E N D
Objectives: • Introduction to Item Analysis • To begin to understand how to identify items that should be improved or eliminated.
Item Analysis • No item is perfect. • An item might be ambiguous, too simple, too difficult, or non-discriminating. • Non-discriminating means the item can not be used to measure individual differences on the trait that is measured by the test.
Achievement Tests • Item analysis can help diagnose student errors. • It can help improve the quality of tests. • It can lead to instructional improvements.
Achievement Tests • Item analysis can identify problems with the answer key on a teacher-made test, or problems with the machine scoring on a standardized test. • It can help isolate items where the students “guessed” a lot.
The Basic Indexes of Item Analysis • Difficulty – What percentage of respondents got the item “right” or indicated that they possess the trait being measured? • Discrimination – The extent to which the item differentiates between persons with high and low scores on the test.
The Basic Indexes of Item Analysis • Difficulty – Measured by a simple percentage. • Discrimination – Measured by the difference between high and low scoring groups on the proportion answering the “right” answer.
Discrimination • Items that are poor discriminators should be eliminated or modified. • Balance content validity with construct validity. • High discrimination tends to increase reliability.
Discrimination and Difficulty • Discrimination and difficulty are related. • With very difficult items it is harder to show high discrimination. • Balance purpose of assessment with the range of difficulty of items. • Generally .2 - .8 difficulty is desired.
Discrimination • For educational achievment tests, you want to look at the discrimination for the “distractors” or wrong options on a multiple choice item. • Ideally, you want them to be selected mostly by the low scoring respondents.
Discrimination • For Educational tests – • Form a 2 x 2 matrix that crosses “Right” vs. “Wrong” on the item by “High” vs. “Low” on the total score of the test. • High and Low can be determined by a median split, or by quartiles, taking the highest and lowest quartile.
Discrimination • For Psychological tests – • Form a 2 x 2 matrix that crosses “High” vs. “Low” on the item by “High” vs. “Low” on the total score of the test. • High and Low can be determined by a median split, or by quartiles, taking the highest and lowest quartile.
An Example from the PRI • “I am able to ask for emotional support.” • Part of the Social Resourcefulness Factor • Part of the Assistance in Relationships subscale.
