Using Simulation to evaluate Rasch Models

Using Simulation to evaluate Rasch Models John Little CEM, Durham University www.cemcentre.org

Outline • Background • Approach • Uses and Examples • Conclusions

Background • Important questions without easy answers • Simulation is clean and easy • Real data is dirty and unknown • Understanding respondent approaches

Approach • Calculate probabilities • Simulate response as right or wrong • Create dataset • Analyse and compare

Adaptive testing Existing calibrated item bank Initial estimate of ability Suitable question asked Ability estimate updated Stopping criteria checked Ability reported

Adaptive testing Progression of Estimates

Adaptive testing Item bank restrictions

Adaptive testing Initial ability estimates

Adaptive testing Stopping criteria

Adaptive testing Initial item effects

Adaptive testing Fit statistics

Further questions • Effect of adding/removing items • The effect of biased or badly calibrated items • Comparison of real & simulated data • Effect of lengthening a test • Different initiation procedures • Effect of guessing • Effect of specific pupil behaviours

Conclusions • Simple • Powerful • Insightful

Using Simulation to evaluate Rasch Models

Using Simulation to evaluate Rasch Models

Presentation Transcript

Using Rubrics to Evaluate Assessments

Hydrologic Simulation Models

Using Multiple Measure to Evaluate Principals

Item Analysis Using The Rasch Model

Using Chesapeake Bay Models To Evaluate Dissolved Oxygen Sampling Strategies

Using scales to evaluate learning targets

Using global climate models to evaluate environmental problems and potential solutions

Causal Rasch Models

Using Life Cycle Models to Prioritize and Evaluate Actions

Best practices to evaluate land change models

Using FLUXNET data to evaluate land surface models

Using simulation to evaluate the comparability of different biological indicators

From Conceptual Models to Simulation Models

The Secrets to HCM Consistency Using Simulation Models

Using Run Charts to Evaluate Progress

Latent Change in Discrete Data: Rasch Models

Using Children to Evaluate Software

Using Logic Models to Plan, Monitor, and Evaluate Programs

Using FLUXNET data to evaluate land surface models

Building Simulation Models using ProModel

The Secrets to HCM Consistency Using Simulation Models

Simulation Models