An Exploration of Answer-Changing on a Computer-based High-Stakes Achievement Test

1. An Exploration of Answer-Changing on a Computer-based High-Stakes Achievement Test Gail Tiemann Neal Kingston

2. Purpose • Gather baselineanswer-changing patterns on a computer-based achievement test • Add to the knowledge base in this area

3. Methods • Third grade summative test in mathematics, one state testing program • One, 2012 form • 70 items, no time-limit • Students can review and change items as desired

4. Methods • Assessment System Logs • Linear path through the test • Time spent viewing an item • Number of reviews • Number and pattern of changes • Unique benefit of CBT

5. Sample and Test Characteristics

6. Test Score Distribution

7. Answer-Changing Items = 70 Students = 5328

8. Wrong-to-Right & Right-to-Wrong • Final change in pattern • WR • RWR • RWWWR

9. Wrong-to-Right by Items Changed Items = 70 Students = 5328

10. Wrong-to-Right by Items Changed Items = 70 Students = 5328

11. Wrong-to-Right by Items Changed Items = 70 Students = 5328

12. Wrong-to-Right vs. Right-to-Wrong

13. Wrong-to-Right vs. Right-to-Wrong

14. Answer-Changing Patterns

15. Answer-Changing Patterns

16. Scatterplots

17. Scatterplots

18. Elapsed Time Wrong-to-Right < 60 seconds

19. Wrong-to-Right < 10 seconds

20. Flagging Rule Application Mean Wrong-to-Right Count (Target Form Only) + 8 Standard Deviations

21. Discussion • Limitations • One form, one subject, one grade • Nested data • Distributions are not normal

22. Discussion • Answer patterns and elapsed time add to answer-changing interpretations

23. Future • Further exploration of wrong-to-right changes by elapsed screen time and position of change • Other sources of variance • Item-factors (difficulty, position) • Student-factors (demographics) • Nesting (buildings, districts, classrooms)

24. Thank you