1 / 11

IES Evaluations and Data Collection Instruments

IES Evaluations and Data Collection Instruments. Lauren Angelo National Center for Education Evaluation and Sally Atkins-Burnett Mathematica Policy Research (MPR). Lauren.angelo@ed.gov. SAtkins-Burnett@mathematica-mpr.com. Today’s Presentation. NCEE Studies that Involve Mathematics

virginia-guerrero
Download Presentation

IES Evaluations and Data Collection Instruments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IES Evaluations and Data Collection Instruments Lauren Angelo National Center for Education Evaluation and Sally Atkins-Burnett Mathematica Policy Research (MPR) Lauren.angelo@ed.gov SAtkins-Burnett@mathematica-mpr.com

  2. Today’s Presentation • NCEE Studies that Involve Mathematics • Developing Criteria for Choosing an Instrument • Assessing Teacher Knowledge in the NCEE Math PD Study • Assessing Student Achievement in the NCEE Math PD Study • Assessing Classroom Practice in the NCEE Math Curricula Study

  3. NCEE Studies that Include Mathematics Impact Evaluation of a Professional Development Model in Math Impact Evaluation of Math Curricula Impact Evaluation of Education Technology Impact Evaluation of Academic Instruction for After-School Programs Program Selection and Design for an Evaluation of the Impact of Academically Focused Career and Technical Education (CTE) Evaluation of Promising “Boost Up” Math Programs http://ies.ed.gov/ncee/

  4. The Impact of a Professional Development Model in Math Research Question: What is the impact on teacher knowledge, teacher practice, and student achievement of providing 7th grade teachers with intensive professional development that focuses on common student misconceptions in the areas of fractions, decimals, ratio, and proportion? Intervention:A professional development intervention consisting of a three-day summer institute, five one-day seminars held during the school year, and 10 days of intensive in-school coaching. Targeted approach focused on foundational math concepts and strategies for improving instruction. Design: School-level randomized trial, 16 districts (8 schools per district), approximately 200 7th grade teachers Schedule: August 2005-2010; Currently preparing for implementation and data collection

  5. The Impact of a Professional Development Model in Math Content Knowledge Pedagogical Content Knowledge Student Achievement Math PD Instruction

  6. NCEE Math PD Teacher Knowledge Test Purpose: Assess teacher knowledge of math content and pedagogy • NCEE Math PD Teacher Knowledge Test Criteria • Adequate and even rational number coverage • Math – Common Content Knowledge (CCK) • Pedagogical Content Knowledge (PCK) • Sensitive to growth for our population of teachers • 30- to 40-minute administration • 3 Parallel forms • General Instrument Criteria • Content Coverage • Adequate Psychometrics • Reliability and Validity • Test Properties • Administration Considerations

  7. NCEE Math PD Teacher Knowledge Test • Item Development and Review • Fit the framework • Accurate content • Appropriate for population • Standards of good items (one correct answer, appropriate and plausible distracters) • Piloting • Item difficulty • Biserial correlations • Distracter analysis

  8. Key Math Areas CCK PCK* Total Fractions and Decimals  6  6  12 Ratios, Proportions, and Percents  6  6  12 12 12 24 NCEE Math PD Teacher Knowledge Test Framework Pedagogical Content Knowledge (PCK) * Planning Instruction Delivering Instruction Assessing Understanding Current Status: Piloting Data Collection: To be Completed Spring 2009

  9. NCEE Math PD Student Achievement Test Purpose: Assess student math achievement in rational number content that could assess growth • NCEE MATH PD Student Achievement Test Criteria • Ample coverage of rational number content across various difficulty levels for students in study • Sensitive to growth • Multiple forms, machine scoring, 40 minutes or less • General Instrument Criteria • Content Coverage • Adequate Psychometrics • Reliability and Validity • Test Properties • Administration Considerations

  10. NCEE Math PD Study Student Achievement Test Chose to Use Northwest Evaluation Association (NWEA) Measures of Academic Progress in Mathematics • Item bank • Rational number content • Adaptive test • Computer administered Customization/Development: • Customization to specific parameters • Items supplemented to ensure coverage across difficulty levels for each content area Current Status: Inclusion of new items Data Collection: To begin this fall

  11. Evaluation of Math Curricula Research Question: What is the relative effectiveness of different math curricula on student achievement in early elementary school? Under what condition is each curriculum most effective? What is the relationship between teacher knowledge of math content and pedagogy and the effectiveness of math curricula? Intervention:4 math curricula for elementary school students (grade 1 and 2) Design: Randomized trial, schools are randomly assigned to one of the four curricula (Investigations in Number, Data, and Space (PSF); Math Expressions (HM); Saxon Math (HA); Scott Foresman-Addison Welsey (PSF)) Schedule: September 2005-2009 with an optional year; Curricula currently being implemented (2006-2007) in first-grade classrooms; next year curricula will be implemented in both first and second grades.

More Related