Loring Brinckerhoff, Ph.D. Educational Testing Service Ruth Loew, Ph.D.

1. Dilemmas in Testing: When accommodations requests and measurement standards collideAHEAD ConferenceOrlando, FloridaJuly 20, 2017 Loring Brinckerhoff, Ph.D. Educational Testing Service Ruth Loew, Ph.D. Educational Testing Service Katie Bugbee, Ph.D. Association of American Medical Colleges Katie Featherston, J.D. ACT, Inc.

2. Agenda • What are “measurement standards”? • Examples of dilemmas: Accommodations vs. measurement standards • Determining supports for English language learners • Assistive technologies – tablet computers, medical tools on smartphones • memory aids • ASL interpretation • Discussion

3. Measurement Standards • Reliability – Does the test give consistent results? • Validity/test construct – Does the test measure what it’s intended to measure? • Test security – protecting the test’s integrity Testing agencies must consider these factors as well as the test taker’s needs in determining accommodations.

4. Supports for English Language Learners

5. The ACT Test: English Learner Supports Research indicates that scores obtained without supports are under predicting college readiness of English learners • Framework for English Learner Accessibility (ACT Whitepaper, 2014) • Programs and Services for High School English Learners in Public Schools: 2015-16 • Non-Regulatory Guidance: English Learners and ESSA

6. General Decision Matrix for Frequently Requested English Learner Supports

7. The ACT Test: English Learner Supports Available supports include: • Extended Time, not to exceed time and half • Approved word-to-word bilingual dictionary (no definitions) • Test directions in the native language • Printed test directions available in Spanish, Arabic, Chinese (Simplified), Chinese (Traditional), French, German, Haitian Creole, Korean, Russian, Somali, Tagalog, Vietnamese • Testing in a familiar environment/small group

8. Assistive Technologies

9. Case #1: • Physical impairment – paralysis secondary to a spinal cord injury • Functional limitation – unable to flex or extend fingers; insufficient dexterity to type on standard keyboard or write with paper/pencil • Accommodation req’d – Ipad with customized stylus • Issue – test security risk • Solution – to be discussed

10. Case #2: • Physical impairment – diabetes • Functional limitation – time & access to supplies/tool necessary for monitoring & managing blood glucose levels • Accommodation req’d – access to continuous blood glucose monitoring app on cell phone • Issue – test security risk • Solution – to be discussed

11. Memory Aids

12. Case #3: • Cognitive impairment – severe learning disability, involving receptive & expressive language processing & auditory memory • Functional limitation – difficulty retaining & processing information • Accommodation req’d – use of memory jogger/formula sheet (compensate for memory weakness, allow for focus on problem solving aspect of task) • Issue – fundamental alteration of exam? • Solution – to be discussed

13. ASL Interpretation as an Accommodation

14. ASL Interpretation Many K-12 state accountability tests offer ASL interpretation of test items as an accommodation. As this population moves up to higher ed., we will probably see more requests. Issues to consider: • Test taker needs an accommodations history • Does the accommodation fundamentally alter test construct? • Is language the construct? Part of the construct? • How will scores be used? • Who are the stakeholders? (State agency? Licensing board?) • Pre-recorded vs. on-the-fly interpretation

15. Discussion

16. Contact Information Loring Brinckerhoff, Ph.D. Director, Office of Disability Policy Educational Testing Service lbrinckerhoff@ets.org Katie Featherston, JD Senior Manager ACT, Inc. katiefeatherston@act.org Ruth Loew, Ph.D. Asst Dir., Office of Disability Policy Educational Testing Service rloew@ets.org Katie Bugbee, Ph.D. MCAT Accommodation Review Association of American Medical Colleges kbugbee@aamc.org