1 / 21

An Investigation of The Response Time for Maths Items in A Computer Adaptive Test

C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK. An Investigation of The Response Time for Maths Items in A Computer Adaptive Test . Chris Wheadon and Qingping He ( CEM Centre, Durham University, UK ). C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK. Background.

prentice
Download Presentation

An Investigation of The Response Time for Maths Items in A Computer Adaptive Test

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK An Investigation of The Response Time for Maths Items in A Computer Adaptive Test Chris Wheadon and Qingping He (CEM Centre, Durham University, UK)

  2. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Background • The CEM Centre’s Baseline Assessments on Primary and Secondary School Students • The CEM Baseline Tests for Secondary Schools (for students aged 11-18: three separate projects) • Paper based, Efficiency, Cost • Use of CTT and IRT in Computerised Testing and to Interpret Test Data • To Develop A Computerised Test to Replace the Existing Paper-based Baseline Tests for the Three Secondary Projects

  3. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK CABT-The CEM Computer Adaptive Baseline Test • CABT comprises a calibrated item bank and a question display system • The adaptive test contains an adaptive maths test and an adaptive English vocabulary test • Testing is delivered through the Web or from the school’s local area network

  4. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Establishing A Calibrated Item Bank • Testing of items by school students from different year groups through the administration of a series of tests containing common items • Calibration of items in each test using the Rasch model (items meeting the Rasch requirements are used) • Setting of the reference test and equating of different tests using the Rasch model and common items • Further new item calibration through the embedding of new items in the adaptive test

  5. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Developing The Question Display System - Conducting Adaptive Testing • Realisation of adaptivity is achieved through the implementation in the question display system of the Rasch model for ability estimation (MLE method is used) and question selection • Rules for stopping test: minimum number of question; maximum number of questions; and convergence value • Variable starting difficulty for the first question for different year groups.

  6. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Preliminary Results from the Adaptive Maths Test • There are over 500 items in the maths item bank • Effort has been made to make the items as curriculum-free as possible • Items are content-independent to each other • Item types include MCQ, short free text entry questions and interactive questions • Items cover a wide range of difficulties in order for the three secondary projects (for three different year groups) to use the same adaptive test as their baseline test

  7. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Student: The Distribution of Ability and the Distribution of Item Response Time

  8. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Student: The Distribution of Ability and the Distribution of Item Response Time

  9. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK All Items: The Effect of Item Difficulty on Response Time

  10. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK A Selection of Items: The Effect of Age of Test Takers on Item Response Time

  11. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK A Selection of Items: The Relationship between Response Time for Correct Answers and Response Time for Incorrect Answers

  12. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Item: The Effect of Ability and Age on Response Time (Difficulty: -0.6 logits) – Easy Item

  13. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK

  14. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Ability Bands within a Year Group for the Specific Item (Year 7)

  15. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Item: The Effect of Ability and Age on Response Time (Difficulty: 1.1 logits) – Medium-difficulty Item

  16. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK

  17. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Ability Bands within a Year Group for the Specific Item (Year 10)

  18. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Individual Item: The Effect of Ability and Age on Response Time (Difficulty: 2.9 logits) – Difficult Item

  19. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK

  20. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Ability Bands within a Year Group for the Specific Item (Year 12)

  21. C. Wheadon & Q. He, CEM CENTRE, DURHAM UNIVERSITY, UK Conclusions and Further Work • Response time for all the maths items in the test generally increases with item difficulty but shows great variability • Item difficulty levels and the age and ability of test takers have significant influence on item response time • The information obtained in this study can be used for constructing more efficient tests • Further work will involve investigating the relationship between CABT results and students’ subsequent academic performance

More Related