1 / 11

Are we asking the right questions about professional development on national surveys? Insights from cognitive interview

Are we asking the right questions about professional development on national surveys? Insights from cognitive interviews. Laura Desimone Kerstin Carlson Le Floch Susie Ansell James Taylor Meisha Fang American Institutes for Research Vanderbilt University.

tamma
Download Presentation

Are we asking the right questions about professional development on national surveys? Insights from cognitive interview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Are we asking the right questions about professional development on national surveys?Insights from cognitive interviews Laura Desimone Kerstin Carlson Le Floch Susie Ansell James Taylor Meisha Fang American Institutes for Research Vanderbilt University

  2. Improving Surveys of Professional Development • Professional development central to school improvement, yet challenging for surveys • Cognitive interviews is an effective technique for identifying specific problems and possible remedies

  3. Data sources • Cognitive interview data from survey development processes • National Longitudinal Study of NCLB, • National Longitudinal Evaluation of CSR, • Longitudinal Evaluation of the Effectiveness of School Interventions • Approaches in national surveys • NAEP, ECLS, SASS, FRSS, and NELS

  4. What counts as professional development? • Cognitive Interviews: Teachers were unsure of what should be considered professional development – faculty meetings, informal meetings, release time? • National Surveys: Great variation in definitions of what respondents should and should not count as professional development • Implications: Need to acknowledge that teachers receive professional development through formal and informal means, surveys should enable teachers to reflect full range of activities, but better specified.

  5. Recalling professional development activities • Cognitive Interviews: teachers don’t always follow instructions in stem (!) but tend to think about current year; also didn’t recall all activities. • National Surveys: Some questions were anchored to the current school year, others asked about different time periods. • Implications: Research shows it is helpful to anchor questions to a particular event, such as the beginning of the school year

  6. Content vs. Methods

  7. Overlap and Differences by Subject

  8. Impact/Change Everyone knows (especially IES) that to attribute cause, Random assignment is the best! But why can’twe just ask teachers Like we’ve always done before How well did it work… (on a scale of 1 to 4)?

  9. The Concept of a Conceptual Framework Structure/Type Coherence Increase in Teacher Knowledge & Skills & Change in Practice Duration Active Learning Collective Participation Content Focus

  10. More Sophisticated Ideas about Teacher Learning

  11. What are Appropriate Current and Continuing Efforts? • Comparing across surveys • Conducting cognitive interviews • Using a conceptual framework (coincidentally, exactly what we’re doing!)

More Related