1 / 26

Improving the Validity of Assessment Results for English Language Learners with Disabilities

Improving the Validity of Assessment Results for English Language Learners with Disabilities. Laurene Christensen, Ph.D. Linda Goldstone, M.S. National Center on Educational Outcomes. Overview.

vic
Download Presentation

Improving the Validity of Assessment Results for English Language Learners with Disabilities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving the Validity of Assessment Results for English Language Learners with Disabilities Laurene Christensen, Ph.D. Linda Goldstone, M.S. National Center on Educational Outcomes

  2. Overview • IVARED is an Enhanced Assessment grant that includes a consortium of 5 states: Minnesota, Maine, Michigan, Washington, and Arizona • Goals are to improve our understanding of ELLs with disabilities and to support improved decision-making efforts about how they participate in assessments

  3. Project Activities • Longitudinal look at data • Review of accommodations/technical assistance related to accommodations • Focus groups with educators • Principles and Guidelines for assessment decision-making • Pilot of an online training module

  4. What We Learned from Data • Slightly more boys than girls are ELLs with disabilities in each state • Ethnic diversity and home languages varied widely • In general, the contexts of each state were so different, it is not possible to make comparisons across states

  5. What We Learned about Participation and Accommodations • There are challenges in including some ELLs with low-incidence disabilities in all domains of an English language proficiency assessment • Braille and sign interpretation can be challenging to the construct being measured in some domains of the ELPA • In general, there is an increase in the mention of ELLs with disabilities in accommodations policies

  6. What We Learned from Educator Focus Groups • 35 focus groups across five states • 239 Educators across disciplines; over 500 waitlisted • Over 2,300 pages of qualitative data

  7. Participant job titles

  8. Participant demographics

  9. Participant demographics

  10. Participant’s student grade levels

  11. Focus GroupThemes • Validity of assessment results • Alignment of assessment and instruction • Assessment decision making • Collaboration • Assessment of leadership and structures

  12. Theme #1: Validity of Assessment Results • Bias: -student background -culture • Accessibility: - format - English language level

  13. Theme #2: Alignment of Assessment and Instruction • Student skills vs. grade level standards -multiple assessments • Teaching practices -test preparation -scaffold

  14. Theme #3: Assessment decision making • Assessment participation -state mandates -Individualized education program (IEP) • Accommodations -ESL -implementation

  15. Theme #4: Collaboration • Team -multidisciplinary -inclusive • Constraints -resources -logistics

  16. Theme #5: Leadership • Policies -explicit for ELLDs • Support • -communication -time -resources -training -success

  17. Language or Disability? From Christensen, Shyyan et al., forthcoming

  18. Principles and Guidelines • Focused on ELLs with disabilities • Developed by an expert panel • Second language experts • Special education experts • Assessment experts • Online Delphi process • Started with the topic areas from the regulations for standards and assessments • Included open-ended prompts, and consensus building

  19. Principle 1: Content standards are the same for all students. • Guideline 1A: Include individuals with knowledge of content, second language acquisition, and special education on the team that writes standards. • Guideline 1B: Design standards so they are accessible to all students, including ELLs with disabilities. • Guideline 1C. Provide ongoing professional development on implementation of standards for ELLs with disabilities to ensure high quality instruction and assessment.

  20. Principle 2: Test and item development include a focus on access to the content, free from bias without changing the construct being measured. • Guideline 2A. Understand the students who participate in the assessment, including ELLs with disabilities. • Guideline 2B. Involve people with expertise in relevant areas of test and item development. • Guideline 2C. Use Universal Design principles in test and item development. • Guideline 2D. Consider the impact of embedded item features and accommodations on the validity of assessment results. • Guideline 2E. Include ELLs with disabilities in item try-outs and field testing. • Guideline 2F. Conduct committee-based bias reviews for every assessment through continuous, multi-phased procedures.

  21. Principle 3: Assessment participation decisions are made on an individual student basis by an informed IEP team. • Guideline 3A. Make participation decisions for individual students rather than for groups of students. • Guideline 3B. Make assessment participations in an informed IEP team representing all instructional experiences of the student, as well as parents and students, when appropriate. • Guideline 3C. Provide the IEP team with training on assessment decision making for ELLs with disabilities. • Guideline 3D. Use written policies that specifically address the assessment of ELLs with disabilities to guide the decision-making process.

  22. Principle 4: Accommodations for both English Language Proficiency (ELP) and content assessments are assigned by an IEP team knowledgeable about the individual student’s needs. • Guideline 4A. Provide accommodations for ELLs with disabilities that support their current levels of English proficiency, native language proficiency, and disability-related characteristics. • Guideline 4B. Collect and examine individual student data to determine appropriate accommodations for ELLs with disabilities taking ELP and content assessments. • Guideline 4C. Develop assessment accommodations policies for ELLs with disabilities that account for the need for language-related and disability-related accommodations. • Guideline 4D. Provide decision makers with training on assessment accommodations for ELLs with disabilities.

  23. Principle 5. Reporting formats and content support different uses of large-scale assessment data for different audiences. • Guideline 5A. Use disaggregated data for ELLs with disabilities to account for demographic and language proficiency variables. • Guideline 5B. Highlight districts and schools with exceptional performance to identify characteristics that lead to success of ELLs with disabilities. • Guideline 5C. Provide interpretation guidance to educators about ways in which large-scale assessment data can be interpreted and used for educational planning. • Guideline 5D. Provide different score report formats as guides to parents and students.

  24. Online Training Module • Currently under development • A series of “mini-modules” on key topics related both to the principles and guidelines as well as issues that arose in the focus groups • Will be piloted this spring

  25. Questions? Contact information: Laurene Christensen chri1010@umn.edu Linda Goldstone golds240@umn.edu NCEO nceo.info IVARED ivared.org

More Related