1 / 11

Assessment Task Group

Assessment Task Group. Chicago Meeting Progress Report April 20, 2007. Assessment Task Group. TASK GROUP MEMBERS Camilla Benbow, Susan Embretson, Francis (Skip) Fennell Bert Fristedt, Tom Loveless, and, Sandra Stotsky Ida Kelley, Staff. Assessment Task Group. RESEARCH QUESTION

eitan
Download Presentation

Assessment Task Group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment Task Group Chicago Meeting Progress Report April 20, 2007

  2. Assessment Task Group TASK GROUP MEMBERS Camilla Benbow, Susan Embretson, Francis (Skip) Fennell Bert Fristedt, Tom Loveless, and, Sandra Stotsky Ida Kelley, Staff

  3. Assessment Task Group RESEARCH QUESTION Determine the correspondence of NAEP 4th and 8th grade tests to selected state accountability tests for validity in assessing mathematics proficiency.

  4. Assessment Task Group FOUR ASPECTS OF VALIDITY Aspects are particularly relevant for comparing NAEP and state accountability tests. - Content - Substantive - Consequential - Generalizability

  5. Assessment Task Group POSSIBLE DIFFERENCES BETWEEN NAEP & ACCOUNTABILITY TESTS 1 - The content may be weighted very differently between and within strands. 2 – The cognitive complexity of test items, testing the same content may vary broadly. 3 – The empirical difficulty of items measuring the same content may vary broadly between NAEP and state accountability tests and between different states.

  6. Assessment Task Group POSSIBLE DIFFERENCES BETWEEN NAEP & ACCOUNTABILITY TESTS 4 – Tool Inclusion; several modes of testing may have an impact on assessed proficiency and test validity. 5 – Test delivery mode, particularly computer-based versus paper and pencil tests. 6 – Item formats (true/false, multiple choice, constructed response, worked problems, etc) should be examined for impact on proficiency.

  7. Assessment Task Group COMPARISION VARIABLES 1 - The proportional representation of content, as available from test blueprints, to examine the content aspect of validity. 2 - Cognitive complexity and conceptual skill level of items with the same purported content from an inspection of actual test items, to examine the substantive aspect of validity. 3 - Empirical item difficulty between items with the same content on NAEP and year-end tests.

  8. Assessment Task Group COMPARISION VARIABLES 4 – Tool inclusion. 5 – Test delivery mode. 6 - Item format representation, particularly as crossed with content.

  9. Assessment Task Group

  10. Assessment Task Group • CONSEQUENTIAL VALIDITY • Impact of Varying Test Properties on Observed Proficiency Levels of Significant Groups • - Gender • - Racial-Ethnicity • - English Language Learners • - Various Disabilities

  11. Assessment Task Group • CONSEQUENTIAL VALIDITY • - If available, group differences compared under different test preparations • Research literature should be examined as well

More Related