Universally Designed Assessments: Opening the Door to More Appropriate Assessment for All Students

1. Universally Designed Assessments: Opening the Door to More Appropriate Assessment for All Students Martha Thurlow, Sandra Thompson & Christopher Johnstone, National Center on Educational Outcomes University of Minnesota National Center on Educational Outcomes

2. Goals • Define “universally designed” assessments within the current context of large-scale assessments and accountability • Identify elements of universally designed assessments National Center on Educational Outcomes

3. Universally designed assessments: • are designed from the beginning to be accessible and valid for the widest range of students • provide optimal standard assessment conditions National Center on Educational Outcomes

4. Who Benefits? • Universal design does not apply exclusively to people with disabilities or limited English proficiency • It applies to all individuals, with wide ranging characteristics National Center on Educational Outcomes

5. Think about universal design in architecture and tool design • Curb cuts and ramps • Elevators that talk to you • Door handles rather than knobs • Special pen shapes that are easier to hold National Center on Educational Outcomes

6. Proposed Title I Regulations (open to comment at this time) introduce the need for universally designed assessments – [Assessments must be] designed to be accessible and valid with respect to the widest possible range of students, including students with disabilities and students with limited English proficiency. Sec. 200.2(b)(2) National Center on Educational Outcomes

7. Remember this? OFFICIAL BALLOT, PALM BEACH COUNTY, FLORIDA National Center on Educational Outcomes

8. REGULAR PEOPLE GET TRIPPED UP BY THE SIMPLEST THINGS What’s obvious to someone who knows the answer is not always obvious to everyone. National Center on Educational Outcomes

9. It Matters! Ease of Use – National Center on Educational Outcomes

10. Why Might We Want Universally Designed Assessments for Students with Disabilities? • Current wide ranges in use of accommodations – from 8% to 84% of students in latest NCEO survey • Possible misuse (over or under use) of accommodations (in some cases) • Better measurement of students with disabilities National Center on Educational Outcomes

11. Elements of UD Assessments • Inclusive assessment population • Precisely defined constructs • Accessible, non-biased items • Amenable to accommodations National Center on Educational Outcomes

12. Elements of UD Assessments (continued) • Simple, clear, and intuitive instructions and procedures • Maximum readability and comprehensibility • Maximum legibility National Center on Educational Outcomes

13. Element #1: Inclusive Assessment Population • Universally designed assessments are responsive to: • A commitment to serve and be accountable for ALL students • Equitable participation for all students, regardless of Cognitive ability • Cultural background National Center on Educational Outcomes

14. Element #1: Inclusive Assessment Population • Universally designed assessments field-test: • A sample of every type of student expected to participate in the final assessment administration National Center on Educational Outcomes

15. Element #2: Precisely Defined Constructs • Universally designed assessments reflect good measurement qualities: • Actually measure what they are intended to measure • Remove all non-construct-oriented cognitive, sensory, emotional, and physical barriers National Center on Educational Outcomes

16. An Example: Mathematics Tests The reading requirements of a math test often prevent students with marginal reading ability from demonstrating competency in math. National Center on Educational Outcomes

17. Ordering Pizza (Original Item) The cafeteria manager surveyed the students in a middle school to find out if they would buy Brand X pizza on Friday if the manager sold it. She made a circle graph to display the results of her survey. NO YES Based on the results of the survey, answer the following questions: • What fraction of students would buy Brand X pizza on Friday? • What percent of students would buy Brand X pizza on Friday? • There are 1200 students in this school. How many students will buy Brand X pizza on Friday if the manager’s survey is accurate? National Center on Educational Outcomes

18. Ordering Pizza (Revised Item) Maria surveyed the students in her school to find out if they liked pizza on Friday. She made a circle graph to display the results of her survey. NO YES • What fraction of students said “yes”? • What percent of students said “yes”? • There are 1200 students in Maria’s school. How many students said “yes”? National Center on Educational Outcomes

19. The language used in questions on tests that assess subjects other than language needs to become as “transparent” as possible National Center on Educational Outcomes

20. Element #3: Accessible, Non-Biased Items • According to the National Research Council (1999), bias arises when: “Deficiencies in the test itself result in different meanings for scores earned by members of different identifiable subgroups.” National Center on Educational Outcomes

21. Biasincludes anything in an item that could potentially advantage or disadvantage any subgroup of examinees. It takes special thinking and review to ensure that items are not biased for each and every student who will be tested. National Center on Educational Outcomes

22. Universally designed assessments incorporate accessibility as a primary dimension of test specifications • Insist that item developers are trained • Form Bias Review Panels that include individuals who know disability and language issues, as well as cultural, gender, and other issues National Center on Educational Outcomes

23. Element #4: Amenable to Accommodations • Universally designed assessments allow needed accommodations to be used • Plan for students who continue to need accommodations • Facilitate the use of accommodations such as assistive technology National Center on Educational Outcomes

24. No More Accommodations? • universally designed assessments will not eliminate the need for accommodations • universally designed assessments may reduce the need for accommodations • universally designed assessmentswillreduce threats to validity and score comparability when accommodations are used National Center on Educational Outcomes

25. Element #5: Simple, Clear, and Intuitive Instructions and Procedures • Universally designed assessments focus on the knowledge and skills assessed, not on seeing whether the student can figure out how to respond • Applies regardless of experience, knowledge, language skills, or concentration level • Not knowing how to respond can invalidate a student’s test score National Center on Educational Outcomes

26. Element #6: Maximum Readability and Comprehensibility • Universally designed assessments attend to various factors that affect readability • Students’ previous experiences, achievement, and interests • Features such as word and sentence difficulty, organization of materials, and format National Center on Educational Outcomes

27. Sample Readability Guidelines • Use simple, clear, commonly used words, eliminating any unnecessary words • Clearly define any technical terms that are used • Break compound complex sentences into several short sentences. State the most important ideas first • Introduce one idea, fact, or process at a time National Center on Educational Outcomes

28. Element #7: Maximum Legibility • Universally designed assessments use text that enables people to read quickly, effortlessly and with understanding • The physical appearance of text – shapes of letters and numbers – conforms to several dimensions that characterize legible text National Center on Educational Outcomes

29. Legible Text Dimensions Contrast – Black type on matte pastel or off-white paper produces good contrast and reduces eye strain Type Size – Print larger than 12 point increases legibility Spacing – Space between letters and between words in wide National Center on Educational Outcomes

30. Legible Text Dimensions Leading – White space between lines of type (leading) is larger Typeface – Standard typeface, with upper and lower case letters, is better than italic, small caps, or all caps Justification – Unjustified text is easier to read, especially for poor readers National Center on Educational Outcomes

31. Legible Text Dimensions Line Length – Text should be about 40-70 characters, or about 8-12 words per line Blank Space – Space around paragraphs and between columns of type increases legibility National Center on Educational Outcomes

32. Universally designed assessments use non-text materials just as carefully as text materials • Symbols are highly distinguishable • Only essential illustrations are used (ones referred to in text and necessary to answer question) [illustrations for interest often draw attention away from construct being assessed] National Center on Educational Outcomes

33. Universally designed assessments consider the design of the response venue as well as the assessment itself • Large bubbles that avoid most challenges of low vision or difficulty with fine motor skills • Consideration of age of students in selecting format (avoid separate answer sheets for younger students) National Center on Educational Outcomes

34. More information? NCEO Resources • Visit: www.education.umn.edu/nceo • or Search for NCEO • Web site includes: • Topic introduction • Frequently Asked Questions • Online and Other Resources National Center on Educational Outcomes

35. Question: Are All Universally Designed Assessments Computerized? • No, universal design principles apply to all media used for assessments, including the current dominant one – pencil and paper tests. • But, it may be easier to provide an array of options to students through computerized assessments. National Center on Educational Outcomes

36. Computer-Based Assessments • Must maintain each element of universal design • Can be poorly designed and inaccessible in much the same way as paper and pencil assessments • Should be used with great caution unless equity issues have been addressed National Center on Educational Outcomes

37. Equity Issues • Access to computers • Experience using computers • Training and practice with assistive technology devices and software (e.g., screen readers, speech synthesizers) National Center on Educational Outcomes

38. Choice allowed by computer-based testing is a significant benefit for students – they can use the options that are most useful to them! National Center on Educational Outcomes

39. Caution While universally designed assessments can make tests more equitable, producing results that are more valid for all students, they cannot replace instructional opportunity! National Center on Educational Outcomes