1 / 18

Professional Development Activity Log: Comparing Teacher Log and Survey Approaches to Evaluating Professional Developme

Symposium - Evaluating the Quality of Professional Development: Implications for Districts and States. Professional Development Activity Log: Comparing Teacher Log and Survey Approaches to Evaluating Professional Development. AERA Annual Meeting Montreal, April 1 1 , 2004. Kwang Suk Yoon

seth
Download Presentation

Professional Development Activity Log: Comparing Teacher Log and Survey Approaches to Evaluating Professional Developme

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Symposium - Evaluating the Quality of Professional Development: Implications for Districts and States Professional Development Activity Log: Comparing Teacher Log and Survey Approaches to Evaluating Professional Development AERA Annual Meeting Montreal, April 11, 2004 Kwang Suk Yoon Reuben Jacobson American Institutes for Research

  2. Overview • Comparisons between teacher logs with survey • Teachers’ experience with logs • Lessons learned • Next steps

  3. Professional Development Activity Log (PDAL) • The PDAL is a web-based, self-administered, longitudinal data collection tool for teachers to record their professional development experiences in detail with the assistance of a series of structured prompts • Teachers log on to their password-protected web account and fill out their PDAL at regular intervals • Visit www.PDAL.net for more information

  4. PDAL Entries • Name of activity • Number of hours spent on each activity and its duration • Whether the activity is a one-time or continuous event (e.g., recurring over a number of months) • Type of activity (e.g., workshop, summer institute, study group) • Purpose of activity (e.g., strengthening subject matter knowledge) • PD quality features (e.g., active learning, coherence, collective participation) • Content focus (e.g., algebraic concepts: absolute values, use of variables, etc.) • Instructional practice – instructional topics covered in each activity (e.g., use of calculators, computers, or other educational technology)

  5. Why PDAL? • Collects disaggregate information about specific PD activities – Increases the level of specificity of PD data and reduces bias introduced by gross data aggregation • Gathers accurate, reliable, and time-sensitive information – Minimizes recall problem with retrospective reports • Tailors technical assistance to teachers based on their response patterns • Allows teachers to review their own logs – Teachers can reflect on their own PD experiences • Generates context sensitive questions

  6. “Black Box” of Survey Data Hypothetical Data

  7. Fine-grain log-level data on contact hours : Disaggregated by teacher by activity by time Hypothetical Data

  8. Validation of Teacher Logs as an Alternative Data Collection Method • Comparing PDAL and exit survey results • Measurement properties • Correlations • Mean levels • Assessing the relative efficacy of teacher logs and survey

  9. Method • Instruments • PDAL • Conduced over 15 months • Exit Survey • Follow-up questions about PD activities over the same period of 15 months • Questions about teachers’ experiences with PDAL • Sample • 4 Math-Science PartnershipProgram projects • Sample • 476math and science teachers mostly in middle or high schools • Participants • 326 teachers completed at least for a month in PDAL • 165 teachers participated in the Exit Survey

  10. Measurement Properties of PDAL

  11. Agreement between Log & Survey Methods:Correlations between methods

  12. Agreement between Log & Survey Methods:Mean Levels

  13. Teachers’ Experience with or Opinions about PDAL:Results from the PDAL Exit Survey

  14. Unique Benefits of PDAL • Rich, in-depth data with a high level of specificity • Differences in features between PD sponsored by MSP vs. other PD • Topic intensity (i.e., amount of contact hours per topic) • Time-dependent measures • Percent of months with PD • Average contact hours by month • Average span of activity • Can be used for on-going formative evaluation to continuously improve PD • Episode-specific comments and feedback

  15. Average Contact Hours by Month

  16. Lessons Learned & Implications • Complementary uses of logs & surveys for different purposes • Globally estimating the mean level of PD activities • Investigating the variability of specific PD elements and relating it to other outcomes • Improving PD design • Cost and benefits of different data collection methods • Improving survey method: Increase the level of specificity

  17. Next Steps • Final phase of data analysis • Usingboth PDAL and survey data to assess change in teaching practice and assess their relative predictive validity • PDAL users focus group (May 2005) • PDAL usability study • Need for follow-up studies • Such as a new CCSSO-AIR study on improving evaluation of professional development in math and science at state and local levels

  18. Contact Information Kwang Suk Yoon (202) 403-5358 ksyoon@air.org Reuben Jacobson (202) 403-6925 rjacobson@air.org Visit us www.PDAL.netwww.air.org

More Related