1 / 25

Authors ： Shu -Chen Cheng,

Estimation of Item Difficulty Index Based on Item Response Theory for Computerized Adaptive Testing. Authors ： Shu -Chen Cheng,. Guan-Yu Chen. Outline. 1. Introduction 2. Literature Reviews 3. Methods 4. Experiments and Results 5. Conclusions. 1. Introduction (1/2 ).

komala
Download Presentation

Authors ： Shu -Chen Cheng,

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Estimation of Item Difficulty Index Based on Item Response Theory for Computerized Adaptive Testing Authors：Shu-Chen Cheng, Guan-Yu Chen

  2. Outline 1.Introduction 2.Literature Reviews 3. Methods 4. Experiments and Results 5. Conclusions

  3. 1.Introduction(1/2) • Computerized Adaptive Testing • Item Response Theory • Advantage: Personalized test, Shorter test length. • Shortcoming: The number of pre-test samples. • IRT-1PL: 20 items, 200 testees(Wright & Stone, 1979) • IRT-2PL: 30 items, 500 testees(Hulin et al., 1982) • IRT-3PL: 60 items, 1000 testees(Hulin et al., 1982) ( There are 1,513 items in our item bank！)

  4. 1.Introduction(2/2) • Test System = Item Bank + Item Selection • Item Difficulty Index Answers Abnormal Rate • Dynamic Item Selection Strategy  Particle Swarm Optimization

  5. 2.Literature Reviews 2.1Computerized Adaptive Testing 2.2 Item Difficulty Index 2.3 Item Response Theory

  6. 2.1 Computerized Adaptive Testing(1/2) • To select the item that its difficulty is most consistent with testee’s ability. • To assess testee’s ability immediately. • The difficulty of next item is affected by previous answer.

  7. 2.1 Computerized Adaptive Testing(2/2) • To test for different abilities through dynamitic item selection strategy. • High ability testee  No too easy items. • Low ability testee  No too difficult items. • A personalized test.

  8. 2.2 Item Difficulty Index (1/2) • Method 1： P : Item difficulty. R : The number of correct answers. N : The number of total testees.

  9. 2.2 Item Difficulty Index (2/2) • Method 2： P : Item difficulty. PH : Correct rate of high score group. PL : Correct rate of low score group. (Generally take 25%, 27%, 33%, etc.)

  10. 2.3 Item Response Theory(1/2) • Item Response Theory (Lord, 1980) • To estimate testee’s ability, aptitude, or location of other continuous psychological interval by the information of their item responses. • Ability location  Item response (Psychometric theory) • In addition to the model of IRT, without any other information to describe the item responses.

  11. 2.3 Item Response Theory(2/2) • Three-Parameter Logistic Model(Birnbaum, 1968) Pi(θ) : Correct probability of item i for ability θ. ai: Discrimination parameter of item i. bi: Difficulty parameter of item i. ci: Guess parameter of item i.

  12. 3. Methods (1/4) • Answers • Testees’ ability>Item difficultyindex  Most testees are supposed to answer correctly. • Testees’ ability<Item difficultyindex  Most testees are supposed to answer wrong. • Testees’ ability=Item difficultyindex  The correct answer rate is 50%.

  13. 3. Methods (2/4) • Answers Abnormal • Violations of any one of these above 3 assumptions among answers are answers abnormal. • 1st group with wrong answers.(Testee’s ability >Item difficulty) • 2nd group with correct answers. (Testee’s ability <Item difficulty) • 3rd group, correct answer rate ≠ 0.5. (Testee’s ability =Item difficulty)

  14. 3. Methods (3/4) • Answers Abnormal Rate ：Answers abnormal rate of item i with difficulty j. • h ：1st group (Testee’s ability >Item difficulty). • l ：2nd group(Testee’s ability <Item difficulty). • e ：3rd group (Testee’s ability =Item difficulty). T：The number of correct answers. F：The number of wrong answers. N ：The number of total testees.

  15. 3. Methods (4/4) • Item Difficulty Difficulty j, let be the smallest. : Item difficulty index of item i. : Answers abnormal rate of item iwith difficulty j. 15

  16. 4. Experiments and Results 4.1 System Descriptions 4.2 Experiment Descriptions 4.3 Results and Discussions

  17. 4.1 System Descriptions (1/3) http://ilearning.csie.stust.edu.tw/EST/Dedault.aspx

  18. 4.1 System Descriptions (2/3)

  19. 4.1 System Descriptions (3/3) PSO Dynamic Item Selection Strategy • Item Difficulty • Knowledge Weights • Item Exposure Rate

  20. 4.2 Experiment Descriptions • Method: Online test • Item Bank: • Items: 1,513 • Initial Difficulty: 0.5(9 levels, 0.1~0.9) • Participants: • Students: 51 • Initial Ability: 0.2(9 levels, 0.1~0.9) • Periods: 6weeks

  21. 4.3 Results and Discussions (1/3)

  22. 4.3 Results and Discussions (2/3)

  23. 4.3 Results and Discussions (3/3)

  24. 5. Conclusions • Each test item is treated as independent, and the item difficulty can be estimated individually. Therefore, the item bank can be expanded easily at any time. • The estimation based on the answers abnormal rate proposed in this study can estimate the item difficulty index quickly and reasonably without too many pre-test samples.

  25. The End ~ Thanks for your attention!

More Related