Planning for the Successful Inclusion of Your Students in Computer-Based Tests

1. Planning for the Successful Inclusion of Your Students in Computer-Based Tests NCEO University of Minnesota Martha Thurlow Deb Albus Laurene Christensen Yi-Chen Wu Sheryl Lazarus

2. Background

3. Brief Analysis of State Administered CBTs

4. Search for state CBTs • Searched state department of education Web sites late October/November, 2009 • Verified in February/March, 2010

5. Table 1. Number of States with CBTs Found by Type of Assessment

6. Table 2. Approaches of States Offering Paper Versions of Regular CBT

7. Table 3. Approaches of States Offering Paper Versions of Computer-based End of Course Tests

8. Accommodations Considerations for Computer-based Assessments

9. Current Accommodations Policies • Collecting policy information since 1992 • Synthesis report every two years • Current data is from 2009 • Does not specifically address computer-based assessment • Goal is to describe what is, not to determine whether or not states comply with NCLB/ESEA

10. Analysis Procedures • Obtain publicly available information from state websites (all states plus DC) • Read the participation and accommodation policies and enter data • Verify the data with states • Write the report • Data is made available through a searchable online tool called the Data Viewer (http://data.nceo.info)

11. Relevant Accommodations

12. Future Accommodations on Computer-based Platforms

13. Test Read Aloud Directions-Computer The following information applies only to students assigned the Tests Read Aloud accommodation. Before beginning a test, ensure the following: ● MP3 files are not blocked or restricted on computers used for testing students with this accommodation. Consult the technology coordinator or network administrator, if necessary. ● Headphones are available to students. Students may use personal headphones or ear buds. ● The mute setting is turned off. ● The volume is set to a reasonable level so that other students cannot hear the audio. Although directions for playing the audio and controlling the volume appear on the instructions page at the beginning of each test session, it is important to make sure students understand these tools before testing. Use the information below to prepare students for testing.

14. Test Read Aloud Directions-Computer (continued) Using the Audio Tool: ● Play All, Stop, and Play/Pause buttons are available using the audio tool at the top of the screen (figure 4.2). To hear everything showing on the screen including directions, the test item(s), and answer options, click Play All. The Play/Pause button will alternate between Play and Pause when clicked. To pause, click Pause. To resume audio, click Play. To stop, click Stop. ● The Volume control bar is also located at the top of the screen. To raise the volume, slide the Volume bar to the right. To lower the volume, slide the Volume bar to the left. EOC Test Administration Manual, April/May 2011 21 ● Audio icons appear to the left of each part of a test item (figure 4.2). Click on the icon next to each part of the item to hear the audio file for that part.

15. Considerations for Technology-Based Testing Accommodations • Do technology-based assessments mean that students will no longer need accommodations? • What is the difference between accommodations, embedded features, and good testing practices? • Who makes decisions about what is used during technology-based assessments, and when are those decisions made? • What training is needed for teachers and students? • What should be tracked and documented within the technology-based platform?

16. Do technology-based assessments mean that students will no longer need accommodations?

17. What is the difference between accommodations, embedded features, and good testing practices?

18. Definitions Accommodations are changes in materials and procedures designed to give access to students who need them, and to produce test scores with greater validity for those students. Embedded Features are interactive tools that are part of the test platform and used to customize the assessment forindividual test takers. Good Testing Practices are testing practices and procedures that should be available to all.

19. Who makes decisions about what is used during technology-based assessments, and when are those decisions made?

20. Technology-based accommodations require thinking through logistics!

21. What training is needed for teachers and students?

22. What should be tracked and documented within the technology-based platform?

23. The Preliminary Results for OAKS

24. Background information • Oregon Assessment of Knowledge and Skills (OAKS) • Students can choose they want to take paper or online version • Every student in the state will have up to three opportunities to take the test • One of the highest score will be marked as the “Best score” • Document accommodation “Yes vs. No”

25. Research Questions • How many students with disabilities take a paper version of the OAKS? • Did students with disabilities have less opportunity to take online test compared to students without disabilities? • In which month did students with disabilities generally take OAKS online and how does this compared to students without disabilities? • How many students with disabilities used accommodations for OAKS online? • What are the performance patterns of students with disabilities for OAKS online?

26. How many students with disabilities take a paper version of the OAKS? • Very few students with disabilities took the paper version • None of the students took the paper version more than once

27. Did students with disabilities have less opportunity to take online test? • Reading • No. SPED students have more opportunities to take the test than Non-SPED student • Math

28. In which month did students generally take OAKS online? One opportunity • Results are similar across content area • SPED: more spread out from Nov. to April • Non-SPED : Feb-April

29. In which month did students generally take OAKS online? Two Opportunities • SPED • 1st : Jan & Feb • 2nd : April & May • Non-SPED • 1st : November & Jan • 2nd : April & May

30. In which month did students generally take OAKS online? Three Opportunities • SPED • 1st : Nov., Dec, & Jan. • 2nd : March & April • 3rd: May • Non-SPED • 1st : Oct. &Nov. • 2nd : March & April • 3rd: May

31. How many students with disabilities used accommodations for OAKS online? • Reading • Less than 20% of SPED received accommodations across different opportunities in Reading. • Math • Less than 19% of SPED received accommodations across different opportunities in Math.

32. What are the performance patterns of students with disabilities for OAKS online? • Reading • Most of students took the test once. • For students took the test twice, more than 65% of them got the best score on the second attempt [Non-SPED: 26/(14+26); SPED:25/38].

33. What are the performance patterns of students with disabilities for OAKS online? • Math • Most of students took the test once. • For students took the test twice, more than 69% of them got the best score on the second attempt [Non-SPED: 30/41; SPED:27/39].

34. Conclusions • SWD have more opportunity to take online test • SWD (one opportunity): were more spread out across months. • Less than 20% of students with disabilities used accommodations for OAKS online. • The performance patterns of students with disabilities for OAKS online is similar to students without disabilities.

35. Questions? Comments?

36. . Presenter Contact Information: Martha Thurlow thurl001@umn.edu Deb Albus albus001@umn.edu Laurene Christensen chri1010@umn.edu Yi-Chen Wu wuxx0207@umn.edu Sheryl Lazarus laza0019@umn.edu