Universally Designed English Proficiency and Content Area Assessments

1. Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency Assessments May, 2004

2. Training Goals • Determine why we need universally designed assessments – and for whom • Identify and give examples of essential elements of universally designed assessments • Identify and apply considerations for item review to sample test items

3. A Quick Definition Universally designed assessments are built from the beginning and continually refined to be accessible and valid for the greatest number of students, including English language learners!

4. But, what does that really mean? • Do we want to change the standard of performance? NO • Can we forget about accommodations if we do this? NO • Is this all figured out – for now and forever? NO • Is this something that will benefit only English language learners? NO

5. Think about universal design in architecture and tool design • Curb cuts and ramps • Signage with universal symbols • Door handles rather than knobs • Special pen shapes that are easier to hold

6. The goal is to provide optimal standard assessment conditions for today’s diverse population of students

7. Remember this? OFFICIAL BALLOT, PALM BEACH COUNTY, FLORIDA

8. Analysis of Florida VotesPrecincts with primarily Hispanic, African American, or elderly voters had more than 3 times as many rejected ballots as other precincts.(New York Times, 11/12/01)

9. Design is important in a lot of things – including assessments! http://go.to/funpic

10. Assessments Discussed in this Training • English Language Proficiency Listening/ SpeakingTest - Computer Based • English Language Proficiency Reading and Writing Test -Paper/Pencil • Grade level Academic Content Area Assessment - Computer Based • Grade level Academic Content Area Assessment - Paper/Pencil

11. Universally Designed Assessments Can Increase Reliability and Validity of Assessments for English Language Learners • Analyses of large data files (Abedi, 2002) found language as a source of measurement error that can negatively impact reliability of test results for ELLs. • Language has also been found to be a source of construct irrelevance that can have a negative impact on the validity of the results for ELLs (Abedi, 2002).

12. Universally Designed Assessments Increase Usability for the End Users – STUDENTS! • In the past, emphasis has been placed on developing test items without consideration of the students who are the ultimate end users • The students who are tested have changed and are no longer “even remotely comparable to the designer in skill set, aptitude, expectation, or in almost any attribute that is relevant to the design process”(Rubin, 1994)

13. Student Characteristics Just one of many reasons that we need to be talking about universally designed assessments!

14. Complete these well-known phrases: • A penny saved is . . . • An idle mind is . . . • Don’t bite the hand that . . . • It’s always darkest before . . . • Strike while the . . . • Two’s company, three’s . . . • If at first you don’t succeed, . . .

15. Here’s what the kids say!

16. A penny saved is . . . . . . not much.

17. An idle mind is . . . . . . the best way to relax.

18. Don’t bite the hand that . . . . . . looks dirty.

19. It’s always darkest before . . . . . . Daylight Savings Time.

20. Strike while the . . . . . . bug is close.

21. Two’s company, three’s . . . . . . the Musketeers.

22. If at first you don’t succeed, . . . . . . get batteries.

23. What seems obvious and clear to test developers may not be all that obvious and clear to students. It takes consistent effort and guidance to make sure that test items and tests really are accessible to all students.

24. This is a BIG DEAL! Requirements for universally designed assessments are showing up in federal legislation

25. Title I Regulations introduced the need for universally designed assessments – [Assessments must be] designed to be accessible and valid with respect to the widest possible range of students, including students with disabilities and students with limited English proficiency. Sec. 200.2(b)(2)

26. Increasingly, we need to think about students with a variety of different language and cultural background characteristics

27. The number of English Language Learners in schools has increased dramatically • Nearly 4 million LEP students in grades K-12in 2001-2002 (71.9% increase from 1991-1992) • Represent about 8.4% of all public school students in the U.S. • Enrolled in almost half of all public schools nationwide

28. The number of ELLs with special needs is increasing too! • Most recent estimate is 9% of the total ELL population is in Special Education • Estimated at about 357,000 students • More than 50% of ELLs with disabilities classified as learning disabled

29. The bill for the federal reauthorization of the Individuals with Disabilities Education Act (IDEA) also includes universal design – UNIVERSAL DESIGN- The State educational agency (or, in the case of a district-wide assessment, the local educational agency) shall, to the extent possible, use universal design principles in developing and administering any assessments…. Sec 612, Senate Bill 1248

30. So, when we think about all students, we have to think about students who have a variety of cultural, physical, sensory, and processing characteristics.

31. Caution While universally designed assessments can make tests more equitable, producing results that are more valid for all students, they cannot replace instructional opportunity!

32. Elements of Universally Designed Assessments • Inclusive assessment population • Measures what it intends to measure • Respects the diversity of the assessment population • Clear format for text

33. Elements of Universally Designed Assessments • Clear pictures and graphics • Concise and readable text • Amenable to accommodations

34. Inclusive Assessment Population • Universally designed assessments: • Consider all types of students in the general curriculum from the beginning • Include English language learners and students with disabilities in item tryouts and field testing

35. Example of Field Test Specifications • The statewide field test will include students with disabilities and English Language Learners using the accommodations they would normally use in the large-scale assessment. • There may be need for over sampling of these populations to ensure a valid field test.

36. Measures What it Intends to Measure • Universally designed assessments reflect good measurement qualities: • Reflects the intended content standard (reviewers have information about what is being measured) • Minimize skills required beyond those being measured

37. Suppose a test item requires a student to read an Aesop’s fable What could this item measure? • Decode text? • Comprehend extended passages? • Extent of vocabulary? • Understand the moral or point of the fable? • Discuss the common elements of any fable? • Compare and contrast fables with news reports? • Articulate the relationship between the fable and the overall culture? • Anything else? (National Center on Accessing the Curriculum, 2003)

38. Suggestions by Researchers . . . • Popham (2001) suggests creating an assessment description for each item that concretely describes what is being tested. • Chiu & Pearson (1999) suggest determining non-construct behaviors needed to complete task.

39. Respects the Diversity of the Assessment Population • Accessible to test takers (consider age, gender, ethnicity, disability, socio-economic level) • Avoid content that might unfairly advantage or disadvantage any student subgroup

40. According to the National Research Council (1999), bias arises when: “Deficiencies in the test itself result in different meanings for scores earned by members of different identifiable subgroups.”

41. Consider the Effect of Cultural Bias A. B. A. B. If you measure from the nose to the tail, how much longer is lizard B than lizard A? Use the ruler provided in your testing kit. Give the answer in centimeters.

42. Experience can be crucial! To raise money for a trip to the Wolfridge Environmental Learning Center, sixth graders at Johnson Middle School are selling raffle tickets. The raffle prize is an electric scooter worth more than $300. A total of 500 tickets were sold. You bought two raffle tickets, your sister bought three and your father bought one. What is the probability that someone in your family will win the prize?

43. Clear Format for Text • Universally designed assessments use text that can be read quickly and effortlessly • Standard typeface • Type size at least 12 point • Wide spacing • High contrast • Sufficient leading (Spacing) between lines of text • Staggered right margins (no right justification) • No background graphics behind text

44. Typeface:Standard typeface, with upper and lower case letters, is better than italic, small caps, or all caps Adapted from: Making Text Legible by A. Arditi

45. Type Size All type should be at least 12 point (including captions, keys, labels, and footnotes). Remember that point size differs among typefaces. Adapted from: Making Text Legible by A. Arditi

46. Spacing:Space between letters and between words is wide Adapted from: Making Text Legible by A. Arditi

47. Contrast Text should be printed with the highest possible contrast. Adapted from: Making Text Legible by A. Arditi

48. Leading: Spacing Between Lines of Text Adapted from: Making Text Legible by A. Arditi

49. Staggered Right MarginsText is easier to read when it is justified on the left and unjustified on the right. This prevents pockets of spaces. Text is easier to read when it is justified on the left and unjustified on the right. This prevents pockets of spaces. (APH, 2002) Used with permission

50. Text with background graphics or other media is difficult to read. This is a picture of a starfish in- side a vortex. Of course to you it might look like a giant flower or an octopus in a whirlpool. You might even think it is op- art or a computer rendering of a bad dream. However this photo is to demonstrate to you how difficult it is to read text when it is placed over a busy background. (APH, 2002) Used with Permission