1 / 42

Hawaii Statewide Assessment Program: Maintaining Test Security and Reporting Testing Incidents

Hawaii Statewide Assessment Program: Maintaining Test Security and Reporting Testing Incidents. Fall 2008 Hawaii Department of Education Systems Accountability Office (SAO).

yvonne
Download Presentation

Hawaii Statewide Assessment Program: Maintaining Test Security and Reporting Testing Incidents

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hawaii Statewide Assessment Program:Maintaining Test Security andReporting Testing Incidents Fall 2008 Hawaii Department of Education Systems Accountability Office (SAO)

  2. This presentation focuses on testing incidents and the procedures for reporting testing incidents and inquiries into those reports.

  3. Board of Education Policy No. 2520STATEWIDE ASSESSMENT PROGRAMPOLICY “Systematic school, curricular, and program improvement efforts directed toward student attainment of the Hawaii Content and Performance Standards (HCPS) requires the collection of relevant and accurate student assessment information.”

  4. Maintaining Integrity of theHawaii State Assessment (HSA) Criterion Referenced Test (CRT) and Norm Referenced Test (NRT) data Curricular, instructional, personnel, and financial decisions Improved student achievement Strict adherence to test administration and security procedures is required to ensure that the student assessment results are accurate.

  5. HSA Procedures All school personnel involved in the administration of the HSA are accountable for knowledge of and adherence to all test security procedures as specified for their roles.

  6. HSA guidelines and procedures can be found in the Test Administrator’s Handbook (TAH), the Test Coordinator’s Handbook (TCH) and the Student Participation and Accommodations Handbook (SPAH). Additional mandatory resources: • HSA District Meeting sessions or school level trainings • DVD: Hawaii Statewide Assessment Program Watching the DVD and attending the mandatory training are not substitutes for reading the HSA handbooks.

  7. Hawaii State Alternate Assessment (HSAA) &Hawaiian Aligned Portfolio Assessment (HAPA) HSAA procedures can be found in the: • HSAA Rater/Coordinator Manual • HSAA Test Coordinator’s Handbook HAPA procedures can be found in the: • School Administrator - Test Administrator - Test Coordinator Handbook

  8. Testing Incidents Any event or procedure that could potentially impact the integrity of the tests, the data, and the test results before, during, and after the test administration is a testing incident. All testing incidents must be reported immediately.

  9. Types of Testing Incidents • Testing irregularity • Standardized administration irregularity • Test security violation

  10. Testing Irregularity Is an incident that affects the student’s performance on the test, but may, or may not, compromise the test result. • Must be reported • May or may not trigger an inquiry • May require the administration of the replacement test form • May require invalidation of the student’s score

  11. Testing Irregularity Examples Include, but are not limited to: • Electrical outage during testing • Fire alarm during testing • Many students taken ill due to vog • Hurricane evacuation during testing • Medical emergencies involving key personnel • Defective materials • Student bubbled the wrong portion of the response booklet

  12. Standardized Administration Irregularity Is a change in the established test administration procedure and protocols that may, or may not, compromise the test result. • Must be reported • May or may not trigger an inquiry • May affect the student’s performance on the test • May result in the administration of the replacement test form • May require invalidation of the student’s score

  13. Standardized Administration Irregularity Examples Include, but are not limited to: • Disallowed classroom displays containing content information visible during test sessions (see handbooks for details) • Test related content instruction, review or practice tests are provided between testing sessions (if it implies prior knowledge of test items, this could be a test security violation) • Test administrator fails to read the directions or conduct the examples as directed • Providing inappropriate accommodations • Selecting specific students to “not test” • Restricting or extending time to complete the testing sessions

  14. Test Security Violation A test security violation is a serious incident that compromises test security, data security, or both, and the integrity of the student score results. • Must be reported • Will trigger an inquiry into the incident • May affect the student’s performance on the test • May result in the administration of the replacement test form • May require invalidation of the student’s score • Superintendent will be notified

  15. Test Security Violation Examples Include, but are not limited to: • Use of mobile phones or cameras during testing sessions to record test items • Test items or student responses are copied, discussed, reviewed or analyzed before, during or after the assessment is administered • Test administrator or proctor leaves the classroom unattended or with an unqualified adult during a session (see handbooks for qualifications) • Test booklets are assigned to test administrator or proctors for their reference and review • Test responses are altered • Prompting (verbal or non-verbal) or giving answers to students during testing to review specific items • Students sharing answers, creating cheat sheets or developing group answers

  16. Procedure, Sequence and Expectations forReporting a Testing Incident

  17. Who Reports? All Hawaii Department of Education employees are required to report testing incidents to the school Administrator. Test Coordinators and school Administrators must report any knowledge of testing incidents to: • Student Assessment Section (SAS) Administrator • Complex Area Superintendent

  18. Students, Parents, and Community Members Non-DOE personnel may report testing incidents to school personnel. School personnel receiving such information are responsible for reporting the information to the school Administrator.

  19. Test Coordinators and School Administrators If you have witnessed, been informed of, or suspect the possibility of a testing incident that could potentially impact the integrity of the tests, the data, and the test results, immediately contact: • Student Assessment Section (SAS) Administrator at 808-394-1300 In no answer, call… • System Planning and Improvement (SPIS) Administrator at 808-586-3285

  20. If No Answer or After Hours Send a LN email to Kent Hinton (cc: Christina Tydeman and Cara Tanimura) Provide: • Your name, • Your school, • Phone numbers with hours for contact, and • The nature of your call. You will be contacted by one of the individuals within your specified hours of contact. The SAO Director will inform the Superintendent of all reported testing incidents.

  21. Besides School Personnel Reporting to SAS… State level data processing can reveal potential testing incidents or school level data quality concerns that may trigger an inquiry.

  22. “Red Flag” Examples Include, but are not limited to: • Reviews of student demographic data • Subgroup identifiers (i.e., ethnicity, SPED, ELL, disadvantaged) • Grade level/birth date • Analyses of testing data • Student performance trends (e.g., school level, grade level, subgroups) • Irregular scoring patterns • Inappropriate accommodations provided • Out of grade level testing • Missing testing materials • Altered documents • Test scores for students who have been exempted from taking the test by parent letter

  23. Testing Irregularity Form All schools reporting any testing incidents must complete and submit a “Testing Irregularity Form” for each incident to the Student Assessment Section.

  24. Information Requested When Reporting to SAS • Role/position of individual reporting • Name is optional • School name • Date(s) of incident(s) • Description of incident(s) • Type (e.g., cheating, missing materials, copying items) • Scope (e.g., individual, class, grade level, school-wide) • By whom (e.g., student, school personnel)

  25. Step 1: Did a Testing Incident Occur? • What happened? • Who was involved? • How widespread? • Who initiated the incident?

  26. Step 1: Outcomes If no testing incident is found, no further action is needed. If a test/administration irregularity is found, SAS shall direct school personnel with appropriate actions. If a test security violation is suspected, the school shall be directed to take immediate action (e.g., stop all testing) and a inquiry team shall be sent to the school.

  27. Step 2: If an Inquiry is Necessary School Administrator and the respective Complex Area Superintendent or Charter School Executive Director will be notified. Upon notification, School Administrator must immediately: • Cease testing • Provide full cooperation and assistance to the inquiry team • Private room for interviews by the inquiry team • Access to all individuals involved in the incident • Copies of documents upon request of SAO

  28. Documents to Have Handy • FERPAforms – with parent/guardian signature for students involved in the incident • Test Administration Training Session Attendance Verification Form • Hawaii State Assessment Security and Administration Procedures Acknowledgment Form

  29. Step 2: Outcomes Inquiry team is established. • Systems Accountability Office (SAO) personnel • Dispatched to the school upon notification of alleged test security violation

  30. Step 3: The Site Visit Upon arrival at the school, the team will: • Meet with Test Coordinator/School Administrator • Summarize and clarify the testing incident • Share the scope of the inquiry • Clarify the roles of school personnel within the inquiry • Discuss possible outcomes • Review copies of all FERPA forms • Conduct interviews • Staff may have union representation

  31. Scope of the Inquiry • Who will be interviewed • How the findings will be determined • When and if testing will be resumed • Possible consequences regarding test security and the validity of test scores • When the report will be completed

  32. At the End of the Site Visit The inquiry team will brief the Administrator and Complex Area Superintendent or Charter Schools Executive Director on the findings.

  33. Components of the Inquiry Report • School context • Summary of the inquiry methodology • Descriptions of the incident, findings, and conclusions • Recommended actions and timeline • Test security and administration procedures • Validity of scores No recommendations for disciplinary actions shall be included.

  34. Suppressed Test Scores In the event the inquiry team determines a student’s scores to be invalid, the scores and the related test materials will be suppressed. • In other words, it will be as if the student(s) did not take the test session(s). Schools may not view any testing materials or scores that have been suppressed as a result of a testing incident inquiry.

  35. Effect of Suppressed Scores Scores from individual testing sessions may be deemed invalid resulting in partial scores in the subject area for the affected students. If all of a student’s scores for a subject area are deemed invalid, the student will have no score and will be considered a non-participant.

  36. Step 3: Outcomes SAO makes the final determinations of score validity. Inquiry report is presented by SAO Director: • Complex Area Superintendent or Charter Schools Executive Director • State Superintendent. Superintendent approves the final recommended actions for improving test security and administration procedures.

  37. Possible Determinations • No testing incident occurred • Test/administration irregularity occurred = valid scores • Did not compromise test security • Did not compromise test results (i.e., scores) • Test/administration irregularity occurred = invalid scores • Did not compromise test security • Compromised test results (i.e., scores) • Test security violation occurred = invalid scores • Compromised test security • Compromised test results (i.e., scores)

  38. At the Superintendent’s Discretion… The Superintendent may share the testing incident inquiry report with the Board of Education or the Charter Schools Review Panel. The school may be required to cover the costs of any replacement test forms used and other related costs (e.g., re-printing of score reports and testing materials, hiring raters).

  39. Step 4: Public Disclosure Approved recommendations are implemented. The school Administrator must notify the affected parents in writing of the test security violation and its impact on their children’s scores. The school, in consultation with the DOE’s Communications Office, must respond to Freedom of Information requests regarding the incident.

  40. Overall Outcomes Improving test security contributes to • Improved validity and accuracy of assessment scores. • Transparency of the test security inquiry process for schools. • Public accountability through full compliance with No Child Left Behind. • Increased credibility and overall confidence in the student performance results.

  41. For more information regarding Testing Incidents and Inquiries, please contact: Systems Accountability Office at 808-586-3283

More Related