K-12 Student Assessment Scoring and Reporting

1. K-12 Student Assessment Scoring and Reporting Dr. Qian Liu Gabriel Colorado 2019-2020 Annual District Assessment and Accountability Coordinators Meeting

2. Overview – Scoring and Reporting • File Layouts • Reports • Score Inquiries • Individual Score Reports Shipments • PreID Upload and Caveon

3. What’s New – File Layouts • District Student Results (DSR) file was updated to include test history information. • Grades 3-10 ELA and Grades 3-8 Mathematics • In Spring 2019, 4 years of prior year data were reported. • The performance level for the current and previous tests are displayed on the Individual Score Report.

4. What’s New – File Layouts

5. What’s New – File Layouts • Exact match on student last name, first name, date of birth and FLEID are needed to populate the test history information. • Therefore, even if a student participated in prior years, if the student’s record could not be matched to previous results, the information will not be displayed in the student report section.

6. Reports • Districts will receive paper copies of Individual Score Reports for distribution to parents/students. All additional reports including school-, district-, and state-level reports, can be found in the secure FSA Reporting System/PearsonAccess Next.

7. Reporting Category Scores • Individual Score Reports and School Reports of Students contain reporting category scores. • Each reporting category represents groups of similar skills, or benchmarks, that are assessed within each grade and subject. Reporting category performance is conveyed by displaying the points earned and points possible for each category.

8. Reporting Category Scores • Points Earned and Points Possible are also referred to as raw scores. It is important to identify comparisons that yield valid interpretations of student performance. • The same points earned by two students may produce the different scale scores due to pattern scoring.

9. Reporting Category Scores • Our tests are scored using Item Response Theory (IRT) pattern scoring to determine student ability. Using this method, the pattern of answers provided by a student is analyzed in combination with the statistical qualities of test items (discrimination, guessing, and difficulty) to determine the scoring weights for each item and the likelihood of individual student scores. Therefore, students could have the same raw score but a different scale score based on the items they got correct or incorrect.

10. What’s New – Individual Score Reports • Beginning with the Spring 2019 administration, grades 3 and 4 ELA Individual Score Reports include the following enhancement – Reading Scholarship. • Students in grades 3 and 4 that score a level 1 or level 2 on the FSA ELA assessment may be eligible for a scholarship that is intended to prepare students for the next grade. The scholarship can be used for instructional material, tutoring, and summer or afterschool educational programs. If a student receives a level 1 or level 2, information about the scholarship will appear next to the performance level on the score report as seen on the next slide.

11. What’s new

12. What’s New: Student Report Shipments • King Solutions, the shipping vendor for palletized FSA Individual Score Reports, sends out shipment notifications with tracking information. • Starting in Fall 2019, it is expected that AIR will send out tracking information for non-palletized shipments. Districts can also call the AIR Help Desk (866-815-7246) for tracking information.

13. What’s New: Student Report Shipments • Pearson sends shipping notifications to districts. At this time, the notifications do not include tracking numbers. The shipment details are provided in PearsonAccess Next once the items have shipped. • We are currently working with Pearson to determine if it is an option to provide districts with tracking information for the ISR shipments.

14. Score Flag/Not Reported Code • 1 / 9 = Score Reported (initial vs. late reporting) • 0 = Not Tested (NT) • 2 = Did Not Meet Attemptedness • 3 = Do Not Score/Invalidated during testing • 4 = Insufficient Match to TIDE (FSA PBT Only)/ Last Name = NONAME and the FLEID is FL000000000000 (NGSSSPBT Only) • 5 = Below Grade Tester • 6 = Duplicate Record • 7 = FDOE Hold • 8 = Caveon Invalidation

15. Score Flag/Not Reported Code • NR (Not Reported) indicates that no data are reported for a particular reason that was listed on the previous slide. • The NR codes will appear in the scale score column of the PDF version of the School Report of Students for all subjects except for FSA ELA Grades 4-10/Retake.

16. Score Flag/Not Reported Code • The score flag information for ELA Reading and ELA Writing will be available in the District Student Results files as well as in the Excel version of the School Report of Students in the FSA Reporting System. • Two separate columns display the ELA Reading score flags and the ELA Writing score flags for grades 4–10/Retake so that schools and districts can better understand why the student’s score was not reported.

17. Score Inquiries • The Score Inquiry System is the main tool for districts to submit score-related questions after the regular results release. • UNDO DNS • Missing Score • School Move • FLEID Update • Others

18. Score Inquiries • Providing accurate information is the key to resolving the inquiry in a timely manner: • Student Information: Last Name, First Name, FLEID • Test Information: UIN, Security Barcode Number for PBT • Additional Comments

19. What’s New – Resolve Duplicate FLEIDs One FLEID was used by two students with same last name, first name, and date of birth in two different districts • District Student Results (DSR) file: two records were assigned score flag of 6 at the last district. • Discrepancy Resolution System (DRS): one record was shown in DRS for each district based on where the paper test document was returned from.

20. What’s New – Resolve Duplicate FLEIDs Resolution: • S&R team sent the student list to the Education Data Warehouse (EDW). • EDW searched the data warehouse to find out who is the owner of each FLEID and returned the results to the S&R team. • S&R team informed the districts who needs to apply for a new FLEID and who owns the FLEID. • The update will be made to wave 2/wave 3 late reporting so duplicate records can be resolved.

21. PreID Dates – Fall 2019

22. PreID Dates – Winter 2019

23. PreID Dates – Spring 2020 • No dates finalized yet, but we understand the need to prepare so we will let you know as soon as we can. • Spring 2019: • FSA in January • NGSSS in February • We will try and send layouts as soon as possible.

24. PreID Changes – Spring 2020 • ELA, Mathematics, and EOC will be one administration in TIDE and use a single PreID layout. • Retakes (ELA and Algebra 1) will use a separate layout and be on an earlier timeline. • .XLSX PreID uploads now accepted (as of Fall 2019) in addition to .TXT and .CSV • .CSV and .TXT uploads look at student data starting on Row 1. • .XLSX considers Row 1 to be a header – student data should start on Row 2.

25. Don’t Forget To Run Reports! • Frequency Distribution Report for FSA • Operational Reports for NGSSS. • Once your PreID uploads have processed and reports are available, make sure there aren’t any glaring mistakes. • Examples: Giving an elementary school all Grade 8 Science, or accidentally having all Algebra 1 EOC testers as regular print accommodations.

26. Private Schools • Private-to-Public Students • Districts must submit student information to FDOE’s Education Data Warehouse (EDW) to request an FLEID assignment for testing. • Private Schools • For private schools approved to test through the application process, FLEID assignments will be addressed through our office – if you have any questions please direct them to Gabriel Colorado or Catherine Altmaier.

27. Educational Data Warehouse (EDW) • Educational Data Warehouse (EDW) assigns unique FLEIDs for students. • The FSA and Pearson help desks are able help you add students if you know you have the right FLEID number but the student is rejected during uploads (name mismatches). • Your MIS department can contact the EDW to request new FLEIDs for any true conflicts (two students using the same number).

28. ShareFile – Caveon Appeals • Student invalidations are posted to ShareFile following each administration. • The Caveon Appeals Guide & Parent/Guardian Letter templates are available on ShareFile. • All Caveon appeals must be submitted via ShareFile by the designated deadline. • If you have appeals to submit after the deadline, please contact the Scoring and Reporting team (Gabriel or Jim).

29. ShareFile – Caveon Appeals • When posting appeals, FDOE should get a notification. • Emailing to confirm always helps! • You don’t NEED to appeal all students in a case. • If parents of multiple students in a case request for an appeal, please submit even if you know it will not result in a release. • However, for straightforward cases such as admission of guilt, we only need an appeal for the student whose score you’re requesting to release.

30. Contact Information K-12 Scoring and Reporting Team Qian Liu 850-245-0867 Qian.Liu@fldoe.org Gabriel Colorado 850-245-0779 Gabriel.Colorado@fldoe.org Jessica Colvin 850-245-5048 Jessica.Colvin@fldoe.org Shukun Fang 850-245-0909 Shukun.Fang@fldoe.org Seung-Yeon Ha 850-245-0783 Seung-Yeon.Ha@fldoe.org Patrick Herrington 850-245-0524 Patrick.Herrington@fldoe.org Esra Kocyigit 850-245-0769 Esra.Kocyigit@fldoe.org Yingru Liu 850-245-5158 Yingru.Liu@fldoe.org James Low 850-245-0085 James.Low@fldoe.org