Simulation Study of Rasch Measurement Precision for Dichotomous Items

1. 29.09.2012 Rasch Measurement Precision 1 Simulation Study of Rasch Measurement Precision for Dichotomous Items Anatoli A. Maslak, Slavyansk-on-Kuban State Pedagogical Institute, Russia

2. 29.09.2012 Rasch Measurement Precision 2 Purpose of the Study Investigation of Rasch measurement precision depending on number of dichotomous items.  The relevance of this study is caused by the fact that the costs of measurement substantially depend on number of items. Therefore it is important to choose such number of items which provides the given precision of latent variable measurement.

3. 29.09.2012 Rasch Measurement Precision 3 Generation of Data of Simulation Experiment Data were generated in accordance with dichotomous Rasch model pij = e?i-?j / (1 + e?i-?j), where pij - probability of a correct answer of person i on item j, ?i -level of person ability of examinee i (in logits), ?j - difficulty of item j (in logits). Here is the level of person ability ?i is the latent variable to be measured.

4. 29.09.2012 Rasch Measurement Precision 4 Design of Simulation Experiment There were conducted ten simulation experiments. In all of them the range of person ability varies from -4.0 logits up to +4.0 logits, adjacent levels differ from each other in .5 logits. So 17 levels of person ability are used: -4.0; -3.5; -3.0; �; + 4.0. For convenience of statistical analysis of experiment results each level is used in triple, so there are 51 true values of person ability in simulation experiment. Difficulty of items also varied from -4.0 logits up to +4.0 logits. In the first experiment the number of items was equal 10, in the second experiment - 20, in the third experiment - 30, �, in the tenth experiment - 100. In each of experiments item difficulties were distributed uniformly from -4.0 logits up to +4.0 logits.

5. 29.09.2012 Rasch Measurement Precision 5 Data Generationof Simulation Experiment First of all, according to the Rasch model there is calculated pij - probability of a correct answer of i-th person to j-th item. Then based on probabilities pij results of testing are calculated xij = Int (pij + 1 - Rnd), where Int(Y) - the whole part of number Y, Rnd - a random number uniformly distributed in the interval (0; 1).

6. 29.09.2012 Rasch Measurement Precision 6

7. 29.09.2012 Rasch Measurement Precision 7 ABSOLUTE ERROR OF ESTIMATE OF THE LATENT VARIABLE

8. 29.09.2012 Rasch Measurement Precision 8 Absolute error of measurement (number of items n = 10)

9. 29.09.2012 Rasch Measurement Precision 9 Absolute error of measurement (number of items n = 30)

10. 29.09.2012 Rasch Measurement Precision 10 Absolute error of measurement (number of items n = 50)

11. 29.09.2012 Rasch Measurement Precision 11 Absolute error of measurement (number of items n = 100)

12. 29.09.2012 Rasch Measurement Precision 12 Conclusion from Figures Absolute error of measurement of person ability is approximately the same on the whole range of its variation. With increase in number of items the average absolute error decreases, so at 10 items the maximal value of an absolute error exceeds 2.5 logits, at 100 items the absolute error does not exceed .5 logits.

13. 29.09.2012 Rasch Measurement Precision 13 ANOVA of absolute error depending on person ability (factor A) and numbers of items (factor B)

14. 29.09.2012 Rasch Measurement Precision 14 Absolute error of measurement of person ability depending on number of items

15. 29.09.2012 Rasch Measurement Precision 15 Thank you!