1 / 17

Illustration of a Validity Argument for Two Alternate Assessment Approaches

Illustration of a Validity Argument for Two Alternate Assessment Approaches. Presentation at the OSEP Project Directors’ Conference Steve Ferrara American Institutes for Research August 1, 2006. Goal.

robin-payne
Download Presentation

Illustration of a Validity Argument for Two Alternate Assessment Approaches

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Illustration of a Validity Argument for Two Alternate Assessment Approaches Presentation at the OSEP Project Directors’ Conference Steve Ferrara American Institutes for Research August 1, 2006

  2. Goal • Illustrate planning for the validation process for large-scale assessments using standards-based alternate assessments from two states • Use selected examples from the paper Standards and Assessment Approaches for Students with Disabilities Using a Validity Argument

  3. Over-arching concept • Provide evidence to support intended inferences about students • Consideration of assessment design (i.e., tasks, administration conditions, scoring) • Plans for collecting procedural and empirical evidence • Important specific principles and recommendations in the paper

  4. Examples from the paper • Massachusetts alternate assessment portfolio (Weiner, 2002) • Oregon performance tasks (Tindal et al., 2003) • Mathematics, grades 3-5 • Intended inferences • The assessment adequately reflects the domain of knowledge and skills for the construct • The assessment accurately identifies students’ level of proficiency in mathematics

  5. Procedural evidence • Test design and development process • Quality of the items and tasks • Assemblage of items/tasks/evidence into an assessment • Administration and scoring process

  6. Empirical evidence • Alignment between the alternate content standards and the assessment items/tasks/evidence (and linkage to grade level/band standards) • Item/task functioning • Reliability of scoring and test score interpretations • Internal relations among items and tasks • Response processes • External relations with other measures

  7. Target math standards • Massachusetts standards • Grades three and four standards that focus on number sense (seven objectives) and operations (three objectives) • Oregon mathematics standards • Numbers, Computation and Operations—Grades four to five

  8. MA: Possible assessment strategies and portfolio products • Addressing access skill(s) (skills embedded in academic instruction) • Alice participates in this activity by assembling money envelopes paired with pictures. Alice works with a classmate who counts the money needed for each item and helps Alice place the correct amount into its corresponding envelope. Alice exchanges these envelopes when making a purchase.

  9. Possible portfolio products (cont.) • Teacher note describing the work accomplished by Alice and her classmate • Data collected on Alice’s ability to assemble money envelopes and exchange correct envelopes when making a purchase • Videotape of Alice making a purchase • Alice’s choice of money envelopes selected for her portfolio

  10. Oregon item • Standard: Read, write, order, model, and compare whole numbers up to 1,000,000, common fractions, and decimals up to hundredths. • Practice Item 24: Find the missing number in the pattern. • 2.6 5.2 ___ 20.8 • (A) 7.8 (B) 10.4 (C) 13.0 (D) 15.6 • Alternate Assessment Task 11: Order Numbers • Present the number cards in this order: 3, 1, 8, 6. • Say: Place these numbers in order from smallest to largest.

  11. Summary • (MA) Assemble money envelopes with a classmate, make purchases • Teacher observations of Alice working, videotape of making purchases • (OR) Order numbers from smallest to largest

  12. Test development process

  13. Test administration and scoring

  14. Alignment and construct representation

  15. Rater accuracy and score reliability

  16. Conclusion • The same types of validity questions apply for all (alternate) assessment approaches • How the questions are posed and the evidence relevant to those questions may differ • Intended inferences, corresponding validity questions, and evidence: • Identify during the conceptualization, design, and development process • Pursue during development and as part of implementation

  17. References Tindal, G., McDonald, Tedesco, M., Glasgow, A., & Almond, P., Crawford, L., Hollenbeck, K. (2003). Alternate assessments in reading and math: Development and validation for students with significant disabilities. Exceptional Children, 69(4), 481–494. Wiener, D. (2002). Massachusetts: One state's approach to setting performance levels on the alternate assessment. (Synthesis Report 48). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved Dec. 8, 2005 from http://education.umn.edu/NCEO/OnlinePubs/Synthesis48.html.

More Related