1 / 20

Enhancing State Assessments Research on the Presentation Effect of Passage Based Computer Tests

Enhancing State Assessments Research on the Presentation Effect of Passage Based Computer Tests Michael Russell Technology and Assessment Study Collaborative Boston College Reading Comprehension/Text Presentation Study Research Questions

Gabriel
Download Presentation

Enhancing State Assessments Research on the Presentation Effect of Passage Based Computer Tests

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Enhancing State Assessments Research on the Presentation Effect of Passage Based Computer Tests Michael RussellTechnology and Assessment Study CollaborativeBoston College

  2. Reading Comprehension/Text Presentation Study • Research Questions • When 4th grade reading comprehension assessments are transitioned to computers, what is the impact on students’ test scores? • Are there differences in test scores when reading passages are presented on the computer using scrolling text versus whole page text? • How is student performance (modal effect) related to their computer literacy/use or fluidity?

  3. Text Presentation Study - Participating Schools and Design • 219 Vermont public school students were assessed • Participating Students • 37 students from urban schools • 83 students from suburban schools • 99 students from rural schools • Students were randomly assigned to take the assessment on paper, on laptop using scrolling text, or on laptop using whole page text

  4. Demographics of Sample

  5. Text Presentation Instruments • Assessment • 4 reading passages (2 NAEP, 1 PIRLS, 1 NH) • 19 multiple choice items • Computer Fluidity Exercises • Keyboarding • Click on moving object • Drag and drop • Arrow keys • Technology Survey • Computer Literacy • Eleven released VA and NC technology multiple choice items • Demo

  6. Text Presentation Study Analyses • Group score averages show a six point difference across groups

  7. Comparison of Scores Among the three Presentation Modes

  8. Item Level differences

  9. Computer Fluidity

  10. Computer Fluidity

  11. Computer Literacy

  12. Computer Fluidity

  13. Home Use

  14. Computer Fluidity

  15. Gender

  16. IEP Status

  17. Race/Ethnicity Status

  18. Multilingual Status

  19. Open-ended Survey • 82% easier on computer • Twenty-six students indicated that they had a problem while taking the test on computer. • not understanding the directions (1), problems scrolling (2), having problems using the mouse (2), not understanding how to end the test (2), and having the computer freeze (2) • 16.8% reported using the highlighting feature • 17.4% said that they did mark items for review

  20. Reading Comprehension/Text Presentation Study Analyses • There were no significant differences in reading comprehension scores across testing modes (paper = 58.1%, scrolling = 52.2%, whole page =56.9%) • There were no statistical differences in reading comprehension scores based on computer fluidity and computer literacy, but a pattern in performance may indicate that students are disadvantaged by the scrolling text mode, particularly those lower computer skills. • The majority of students who took the reading test on a computer indicated that they would prefer to take the test on computer. • Providing highlighters and review markers is useful for some students.

More Related