What Are We Learning About Including All Students in Assessment and Accountability?

1. What Are We Learning About Including All Students in Assessment and Accountability? Sue RigneyU.S. Department of Education No Child Left Behind: Positives, Obstacles and Solutions University of Connecticut March 19, 2008

2. Assessments Shall Provide for… • Participation of all students • Reasonable adaptations and accommodations for students with disabilities • Inclusion of limited English proficient students with accommodations, including, if practicable, native-language versions of the assessment • Assessment in English of reading/language arts for any student in the US for 3 consecutive school years • (NCLB 1111(b)(3))

3. What Have We Learned About… meeting the assessment needs of English language learners and students with disabilities? • Accommodations • Alternate assessments • Based on alternate achievement standards • Based on modified achievement standards • Impact on accountability

4. LEP Students in the U.S. • 5 Million enrolled in grades PK-12 • 4.5 million ages 3-21 speak a language other than English and speak English less than “very well” • Most were born in the U.S. • Elementary grade ELLs - 24% foreign-born • Secondary grade ELLs – 44% foreign born • Most common languages: Spanish (79.9%), Vietnamese (2.0%), Hmong (1.6%), Cantonese & Korean (1.0% each) • (Source: NCELA)

5. LEP Students in Connecticut • Approximately 4.5% of total enrollment • 148 languages other than English • Spanish - 17,924 students • Portuguese - 1,34 students • Polish - 726 students • Albanian, Chinese, Haitian, Serbo-Croation – approx. 550 each • Bilingual/ESL services authorized in state law • (Source: SERC)

7. Accommodations - LEP • Review of lit yields mixed results • Research mainly examines: simplified English (some students show advantage for ELL) customized English dictionaries or glossaries (depends on format) bilingual dictionary (mixed) glossary extra time (+ELL / +non-ELL) dual-language test booklets (no advantage) native language tests (technical and practical concerns) Source: CRESST Report 731

8. Accommodations - LEP Translations – challenging to create Item translation with verification • Is the item aligned to the same content standard? • Does the item maintain the intended reading level of the item? • Does the item maintain the intended difficulty level of the item by ensuring that the item was not simplified or clarified? • Does the item maintain the essential meaning and style in translation? Back translation Are both translations accurate? Transadaptation (dual development)

9. SC EAGAccommodations - LEP Access-Based Item Development “…a carefully crafted variation of an item using techniques such as plain language, plain formatting, reduced reading load, visuals, and other appropriate item modifications intended to make test content more accessible for ELL and students with certain learning disabilities.” Created by South Carolina Handbook on Developing Accessible Assessment Items for English Language Learners and Students with Learning Disabilities

10. SC EAGAccommodations - LEP Access based item development considers: Contextual factors Culturally broad experiences Clear and explicit instructions Prior learning experiences Structural factors Simple language structures Vocabulary Effective visuals Effective item format Text amount Pattern for demonstrated response Impact of home language

11. Assessing Students with Disabilities Source: NCEO

12. Accommodations - SWD Most students with disabilities are expected to participate in regular statewide assessments • without accommodations • with appropriate accommodations that are consistent with accommodations provided during regular instruction

13. Accommodations for SWD Review of the literature: • Extended time (+ SWD / + non-SWD) • Oral accommodation (+SWD only in math) • Equivalent test structure (supported by data)

14. Accommodations for SWD Steve Sireci says… • Many State accommodation policies not based on research • Research is scarce • Experimental studies recommended

15. Peer Review of State Assessment Systems Found That… • Many states fail to monitor the delivery of accommodations during testing • Many states have not validated results based on accommodations

16. Connecticut has…. • A comprehensive system of monitoring the use of accommodations in testing • Electronic data entry system • District Test Coordinator Training • Annual Accommodation training statewide • State mandated IEP forms • Irregularities file maintained by CSDE • IDEA focused monitoring provides follow-up

17. Connecticut also has…. • Agreed to serve as the lead in a 10-state consortium to investigate the validity of test scores based on accommodations • 10 states – 10 accommodations • Counterbalanced design: special education and general education students • Results to be disseminated nationally

18. Alternate Assessments Should Have… • Clearly defined structure • Guidelines for which students may participate • Clearly defined scoring criteria and procedures • Report format that clearly communicates student performance in terms of the academic achievement standards defined by the State

19. States May Use More Than One Alternate Assessment • Alternate assessment based on grade-level grade-level standards • Alternate assessment based on alternate achievement standards – appropriate only for students with most significant cognitive disabilities • Alternate assessment based on modified achievement standards

20. AA-AAS Communication skills of students with the most significant cognitive disabilities What proportion of this population does not use symbolic language? A. 10% B. 30% C. 50% D. 90%

21. NAAC 4 States Reported

22. NAAC Learner Characteristics Inventory • Represent ~1% or less of the total assessed population • All disability categories were represented but primarily 3 emerge, • Mental Retardation • Multiple Disabilities • Autism • Highly varied levels of expressive/receptive language use • Most students in the population use symbolic communication • Level of symbolic language distribution is similar across grade-bands • Only about 50% of the pre and emerging symbolic language users use ACS • Pre-symbolic expressive language users are more likely to have additional complex characteristics. • Most of the population read basic sight words and solve simple math problems with a calculator. • Lack of skill progression in reading across grade bands (elementary, middle & high) • Skill progression apparent in mathematics across grade bands but still small

23. The LCI in Connecticut Accommodations for CMT or CAPT/ Learner Characteristics Inventory Students who utilize accommodations for the standard Connecticut Mastery Test (CMT) or Connecticut Academic Performance Test (CAPT) as indicated on their Individual Education Program (IEP) must have this information entered on the accommodations website: www.cttestaccommodations.net as soon as possible. This data is collected by our test vendor, Measurement Incorporated, to provide appropriate materials necessary for test administration. Students who will be assessed utilizing the CMT or CAPT Skills Checklist must have their Learner Characteristics Inventory entered on the same accommodations website. These data will register the student to take the Skills Checklist creating a file on the Skills Checklist website for March.

24. AA-MAS • Definition: A modified achievement standard is an expectation of performance that is challenging for eligible students, but may be less difficult than a grade-level academic achievement standard • Emerging research results • RI EAG • GA data

25. RI EAG - Reaching Students in the GapReaching Students in the Gap Who are the students in the gap? Of all students who are not proficient, how can states identify those who are in the assessment gap? What are the attributes of students in the gap and how do these students perform? What issues in the assessments themselves contribute to the gaps? Are there specific aspects of multiple choice items used in state assessments that contribute to the assessment gap?

26. RI EAG - Reaching Students in the Gap Who are the students? Gap 1 – students appear to be proficient in class but not proficient on test Gap 2 – students far below grade level in class and very low scores on assessment Could not distinguish between assessment gap and instruction gap, nor the effects of teacher expectations and content coverage. Full results at www.necompact.org

27. RI EAG - Reaching Students in the Gap Gap 1 • Perform at a proficient level in the classroom but fall below proficient on eighth grade mathematics test • Majority are in general education • Teachers puzzled by test results • Most do not receive any accommodations on the test • Generally taught at grade level • Students with IEPs were under-represented in gap 1 and over-represented in non-gap 1

28. RI EAG - Reaching Students in the Gap Gap 2: (perform well below grade level in the classroom and on the test) • Majority are SwDs, ELLs, and SwD/ELLs • Test scores fall into the lowest achievement level • Teachers not surprised by the test results • Most common accommodations: -alternative settings, especially in small groups with someone other than the general education teacher -scheduling changes, including extended time and breaks -presentation/modality adaptations, such as reading test and/or directions aloud • Generally taught below grade level

29. RI EAG - Reaching Students in the GapThe Item Modification Study(grade 8 mathematics) Purpose • Assess the quality and usefulness of items designed to decompose skills/knowledge required to solve complex problems. • Examine the extent to which students who perform well on the complex items also perform well on the decomposed items. • Examine the extent to which students in the gap are able to succeed on decomposed items while struggling with the complex items.

30. RI EAG - Reaching Students in the Gap Item Modification Results • Effective • Using whole numbers • Using whole numbers & removing the context • Simplifying information in the table (for Gap 1) • Not effective • Changing the table format from vertical to horizontal • Removing the context • Simplifying information in the table (for Gap 2) • Removing the context & changing the numerical sequence from decreasing to increasing

31. Georgia EAGAssessing One and All: A Partnership for Success Who are the students? Persistently low performing - lowest performance level for 3 years - grade 5 cohort & grade 8 cohort Approximately 2% consistently low in both reading and math

32. Georgia EAGCharacteristics of Persistently Low-Performing Students When compared to baseline (all students) • More males 50% vs 60-65% • More black students 40% vs 60-65% • More free/reduced lunch 50% vs 75-80% • More students with disabilities 15% vs 40-55% • More students with mild intellectual disabilities 10% vs 20-30%

33. Georgia EAGMore Demographic Data • A higher proportion of ELL students are persistently low performing in reading (compared to the baseline) • Approximately 2% of students at each grade were identified as persistently low performing in both content areas (Reading and Mathematics) • In grade 8, 77% of the students identified in Reading were also identified in Mathematics

34. Georgia EAGNext steps • Are these students receiving instruction on grade-level standards? • Do these students perform differently in specific test items? Strands? Content standards?

35. AA-MAS in Connecticut Development of New Modified Assessment The United States Department of Education has provided the opportunity for states to develop a new assessment for special education students whose disability has precluded them from achieving grade-level proficiency and whose progress is such that they will not reach grade-level proficiency in the same time frame as other students. Connecticut’s Bureau of Student Assessment has been working closely with the Bureau of Special Education to develop this new alternate assessment. Special education directors have been receiving correspondence about our progress and will be providing valuable information about their districts to help refine eligibility criteria and guide us in assessment development. (Source: The Bureau Blog, January 2008)

36. Students’ Impact on Accountability (SINI)

37. Surprising Results Chancellor Klein Salutes English Language Learners for Academic Gains Date: 06/19/2007 Last Modified: 6/19/2007 10:53:34 PM Press ID: N-61, 2006-2007     Schools Chancellor Joel I. Klein today congratulated English Language Learners (ELLs) for their progress in reading and math during a visit to PS 149 in Queens. ELLs at PS 149 and across the City made significant gains on State Math and English Language Arts (ELA) exams this year, building on progress made since the beginning of the Children First school reforms. The ELA gains are especially significant because a new state regulation, prompted by the federal No Child Left Behind Act, required ELLs with more than one year in the school system to take the test; previously, ELLs were exempt until they had spent at least three years in the system. As a result, more than twice as many ELLs took the exam this year citywide, and 16% met standards, a 5.3 percentage point increase over 2006, and a 12.1 point increase since 2003. In math, 45.1% of ELL students are now proficient, up from 35.8% in 2006 and a 28.4 point increase since 2003.

38. CT successes and challenges

39. CT successes and challenges

40. CT successes and challenges

41. Data Driven Actions – District level • Align curriculum and instruction • Increase professional development opportunities • Provide targeted technical assistance • Provide curriculum and instructional resources • Promote and support collaborative teams of content and student learning specialists • Provide student specific resources e.g., communication systems Source: NAAC 2008

42. Data Driven Actions – School level • Insure that IEP teams are assigning students to the appropriate assessment • Align curriculum and Instruction • Provide professional development opportunities • Promote collaborative content and student specialist teams • Request technical assistance • Insure instructional material and environment accessibility Source: NAAC 2008

43. Data Driven Actions – Student level • Apply participation criteria appropriately • Provide appropriate interpretations of results to parents • Use assessment data in conjunction with other student data to make decisions • Insure all students have a way to communicate • Include academic content on communication systems • Identify individualized curriculum and instructional strategies and supports • Implement instructional data-based decision-making Source: NAAC 2008

44. Our Shared Goal “…to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging State academic achievement standards and state academic assessments” No Child Left Behind