1 / 7

2009

Automated Scoring Yes, It Is Ready for Prime Time Julie Quinn Coordinator Test Administration and Data Assessment and Accountability Utah State Office of Education julie.quinn@schools.utah.gov. 2009. Utah Legislature required an online writing assessment for grades 5 and 8 (53A-1-603)

gale
Download Presentation

2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automated ScoringYes, It Is Ready for Prime TimeJulie QuinnCoordinatorTest Administration and DataAssessment and AccountabilityUtah State Office of Educationjulie.quinn@schools.utah.gov

  2. 2009 • Utah Legislature required an online writing assessment for grades 5 and 8 (53A-1-603) • Contract awarded for formative and summative online writing assessment, exclusive AI scoring, grades 5 and 8 • Many LEAs provide formative online writing for other grade levels at their own cost

  3. In the Beginning Formative • Educator skepticism about AI scoring validity and reliability • Positive response to immediate scoring and increased opportunities for writing feedback • Vendor and teacher prompts

  4. In the Beginning Summative • Misunderstandings about AI engine training and scoring rules • Positive response to online administration and AI identification of alert papers • Contract for AI scoring only, vendor provided human read-behind

  5. AI vs. Human Scoring 2010 Grade 5 Agreement Rates N=98

  6. AI vs. Human Scoring 2010 Grade 8 Agreement Rates N=99

  7. Outcomes • Students are more engaged in writing with immediate feedback • The number of writing events with feedback has greatly increased • AI is consistent and accurate • Utah will continue to use AI exclusively for summative writing assessments

More Related