Strategies for Removing Accessibility Barriers for Students with Disabilities

1. Strategies for Removing Accessibility Barriers for Students with Disabilities Arizona State Schools for the Deaf and the Blind Northwest Evaluation Association June 21, 2010

2. Overview - General Many factors have contributed to increased discussion around accessibility for children with special needs. • Emergence of new technologies • Advances in pedagogy • Better understanding of disabilities • Clarifications of ESEA guidelines • Strong advocacy groups • Recent legal decisions • Funding of targeted research

3. Overview - Our Project Students with disabilities were taking computer-based formative tests two times per year. Test results were be used to manage instruction Accessibility issues resulted in student test data that was less than useful in planning instruction Could item enhancements improve accessibility and yet maintain test validity?

4. Questions Would enhancing computer displayed test items make test results more useful for managing instruction for students with disabilities? Specifically, would providing audio translation of item content for blind students and video-based American Sign Language (ASL) translations for Deaf students lead to test results that would more accurately indicate what the students knew?

5. Partnership To pursue answers to these question a partnership was formed between:

6. Partnership To pursue answers to these question a partnership was formed between: The staff and students at Arizona Schools for the Deaf and the Blind

7. Partnership To pursue answers to these question a partnership was formed between: The staff and students at Arizona Schools for the Deaf and the Blind Researchers at Northwest Evaluation Association &

8. Background ASDB Administering MAP tests in math, reading, language, and science two times per year for several years One of the goals was to use student achievement data to plan instruction on an individual basis A research paper in 2008 based on efforts at ASDB outlined the benefits of computerized adaptive tests for specialized populations. And, it also discussed eight inaccessibility issues.

9. Accessibility Issues Kamei-Hannan, C. (2008). Examining the Accessibility of a Computerized Adapted Test Using Assistive Technology. Journal of Visual Impairment & Blindness, 102(5), 261-271.

10. Accessibility Issues (cont.) • Arizona State Schools for the Deaf and the Blind • Measure of Academic Progress (MAP) • Students who are blind or visually impaired • Braille • Refreshable Braille display • Screen-reading software • Large Print • Zoom-Text 7.0

11. Identified Challenges • Mathematics test was eliminated • Difficulty translating the Nemeth code on a Braille display • Accessibility to tactile graphics

12. Testing Accommodations In Arizona, according to the Testing Accommodations Guidelines for 2009-2010, a Standard Accommodation for students with IEPs is “Read aloud in English or Sign the writing prompt, mathematics test items⁺, or science test items⁺”. ⁺ A test item includes both the question and the corresponding answer choices. Any stimulus preceding the item may also be read aloud. Arizona Department of Education (2009, August). Testing accommodations: Guidelines for 2009-2010 [Electronic version]. Phoenix, AZ: Arizona Department of Education.

13. Testing Accommodations Why target Presentation Accommodations? After evaluating accommodations used by students who are visually impaired, 29.12% of accommodations used were directly related to administration and materials. Jackson, L.M. (2003). The effects of testing adaptations on students’ standardized test scores for students with visual impairments in Arizona (Doctoral dissertation, The University of Arizona, 2003) Dissertation Abstract International.

14. Testing Accommodations • Presentation Accommodations – Changes that do not effect the level of difficulty of the item, but ensure accessibility of the item to the students. • Meeting the accessibility needs of students with visual impairments who read Braille • Meeting the accessibility needs for Deaf students who’s primary language (L1) is American Sign Language • Thompson, S.J., Morse, A.B., Sharpe, M., & Hall, S. (2nd Ed.).(2005, August). Accommodations manual: How to select, administer, and evaluate use of accommodations for instruction and assessment of students with disabilities. [Electronic version] The CCSSO State Collaborative on Assessment and Student Standards Assessing Special Education Students.

15. Testing Accommodations Presentation Accommodations for students with visual impairments who read Braille *Thompson, S.J., Morse, A.B., Sharpe, M., & Hall, S. (2nd Ed.).(2005, August). Accommodations manual: How to select, administer, and evaluate use of accommodations for instruction and assessment of students with disabilities. [Electronic version] The CCSSO State Collaborative on Assessment and Student Standards Assessing Special Education Students.

16. Testing Accommodations Presentation Accommodations for Deaf students who’s primary language (L1) is American Sign Language *Thompson, S.J., Morse, A.B., Sharpe, M., & Hall, S. (2nd Ed.).(2005, August). Accommodations manual: How to select, administer, and evaluate use of accommodations for instruction and assessment of students with disabilities. [Electronic version] The CCSSO State Collaborative on Assessment and Student Standards Assessing Special Education Students.

17. Testing Accommodations Who could benefit from the accommodations developed for MAP testing? *Thompson, S.J., Morse, A.B., Sharpe, M., & Hall, S. (2nd Ed.).(2005, August). Accommodations manual: How to select, administer, and evaluate use of accommodations for instruction and assessment of students with disabilities. [Electronic version] The CCSSO State Collaborative on Assessment and Student Standards Assessing Special Education Students.

18. Background • NWEA staff visited the school and observed testing sessions. • We witnessed the limitations of the due to accessibility issues. • Decided some changes were needed, but which ones? • Conduct some prototype tests to initiate study and dialogue on possible strategies to make test items more accessible

19. Research Proposal • Qualitative research • To better define the players and the environment • Use expertise of practitioners

20. Research Proposal • Qualitative research • To better define the players and the environment • Use expertise of practitioners • Blind/Visually Impaired • Enhanced items with human-voice audio clips • Support from certified describer on describing math equations and graphics

21. Research Proposal • Qualitative research • To better define the players and the environment • Use expertise of practitioners • Blind/Visually Impaired • Enhanced items with human-voice audio clips • Support from certified describer on describing math equations and graphics • Deaf/Hard of Hearing • Enhanced items with signing [ASL] video clips • Support from professionals experienced in American Sign Language

22. Summary of Steps • Identified a set of calibrated math/science items that displayed graphics and/or complex math expressions, graphs and tables • Enhanced each item with ASL video clips or human-voice audio clips • Created a web-based delivery system and a database to store all the key strokes • Administered the test to students and staff • Conducted focus groups with students and staff • Performed data analysis of item responses

23. ASL Translation • NWEA provided test items • ASDB provided management, talent and facilities to create the video clips • NWEA processed the clips

24. Summary of Steps Several steps in creating the audio clips: • Selection of appropriate items • Certified Describer to modify the script • Record audio from the script • Process the audio files

25. System Flow Control starts here

26. System Flow Control starts here Then is transferred to the Flash Item

27. System Flow Control starts here Then is transferred to the Flash Item Communication between the two happens here

28. System Flow Control starts here Then is transferred to the Flash Item This loops for each item in the test, with The data captured Logged in the DB Communication between the two happens here

29. System Flow Control starts here Then is transferred to the Flash Item This loops for each item in the test, with The data captured Logged in the DB Communication between the two happens here The session ends here after the last item

30. Data Model

31. Data Captured

32. Demo

33. Feedback (Qualitative) Results from focus group with local school administrators, teachers, and students • Positive responses to test events • Rich feedback from students and school staff • Significant number of suggestions for improvement

34. Feedback (Qualitative) Sample suggestions for improvements given focus groups: • Video buttons larger and displaying “ASL” • Provide hot keys for navigating audio items • Faster speech rate for some audio items • Ability to change responses

35. Feedback (Qualitative) Helpful comments provided during focus group sessions: • Students liked feeling of independence • Student expressed a higher degree of self-confidence while taking the test • Be aware of various signing styles

36. Next Steps Continue working within our guidelines: • Implement the principles of universal design to our assessments • Gather more data from students with disabilities via accessible prototype assessments • Continue collaboration between researchers and practitioners • Build on the demonstrated feasibility of using technology to create item enhancements

37. Next Steps • Develop and administer a prototype fully adaptive test for deaf and blind students

38. Next Steps • Develop and administer a prototype fully adaptive test for deaf and blind students • Begin work on calibrating enhanced items

39. Next Questions • Can we use this strategy to improve assessments for: • Students with other disabilities? • English Language Learners?

40. Potential Goals forMath and Science Fall Semester 2010 – Field testing of 20 item tests Spring Semester 2011 – Item calibration Fall Semester 2011 – Additional field testing Future – Availability of test for NWEA partners We are currently looking for potential study participants

41. Questions?

42. Contact Information • Pat Almond, razberi@teleport.com, 503-632-3615 • Elizabeth Barker, elizabeth.barker@nwea.org, 971-222-1046 • Rocky Harris, rocky.harris@nwea.org, 971-222-1070 • Lisa Jackson, ljackson@asdb.state.az.us, 520-770-3283 • Lingling Ma, lingling.ma@nwea.org, 503-212-3326   Thanks!