1 / 16

Describing the Academic Content of International Mathematics and Science Assessments

Describing the Academic Content of International Mathematics and Science Assessments & Their Relationship To State Academic Content Standards & Assessments. Presented at the 2009 IES Conference, June 2009, Washington, DC John Smithson, Ph.D., Wisconsin Center for Education Research.

phillip-whitfield
Download Presentation

Describing the Academic Content of International Mathematics and Science Assessments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Describing the Academic Content of International Mathematics and Science Assessments & Their Relationship To State Academic Content Standards & Assessments Presented at the 2009 IES Conference, June 2009, Washington, DC John Smithson, Ph.D., Wisconsin Center for Education Research

  2. Content Analysis Using a Comprehensive Descriptive Language and a Systematic Set of Procedures to Describe Academic Content A common, neutral language that can be applied to describe any curriculum-based materials or practices. A systematic procedure, utilizing teams of content experts to employ this common, neutral language as a tool for describing the content embedded in the selected materials.

  3. The Language (Taxonomy) • A two-dimensional framework for describing: • WHAT STUDENTS SHOULD KNOW • (Represented by a detailed topic list organized into content areas.) • & • 2) BE ABLE TO DO • (Represented by five categories of student performance expectations.)

  4. The Process (Content Analysis) • Coding Teams of 3-5 Content Experts • Goal: Describe the Assessed Content • Independent Coding by each Analyst w/ Group Discussion • Data Quality: Generalizability not Inter-rater Reliability

  5. The Description (Content Analysis) 101 102 103 104 104E 105 106 B C D E F

  6. Data Collection (Content Analysis) From Coding Form

  7. The Results (Content Description) The 3rd Dimension

  8. Tile Interval = 2.5% Contour Interval = 1% K-12 Science Content PISA 2006 Science Item Pool

  9. Alignment Analysis Identifying similarities and differences between two content descriptions… … in order to answer the question: To what extent (and how) is the PISA similar or different to U.S. curriculum standards and/or other key assessments. Alignment Indices: A summary measure, or set of measures, designed to indicate the extent of similarity & difference between two content descriptions.

  10. K-12 Science Content PISA 2006 2007 TIMSS Gr.8 CG Align: 0.50 FG Align: 0.37 I Memorize II Procedures III Communicate IV Analyze V Apply Concepts I II III IV V I II III IV V

  11. Mathematics Content: All Content Areas Gr. 8 TIMSS Gr. 8 NAEP 0.38 0.59 0.34 PISA 2006 0.62

  12. TIMSS Mathematics Content Alignment

  13. TIMSS Science Content Alignment

  14. PISA Alignment

  15. Alignment Levels to Key Targets Across All Documents Gr.4 TIMSS Math Gr.4 TIMSS Science Gr.8 TIMSS Math Gr.8 TIMSS Science PISA Math PISA Science Total # Documents 267 101 Total # Documents 195 77

  16. www.SEConline.org

More Related