Studying the Impact of High Quality Professional Development

1. Studying the Impact of High Quality Professional Development …….on classroom practices and student achievement Katherine P. Divine, Ph.D. Simmie A. Raiford, Ph.D.

2. Why standards-based reform and what do teachers need to know? • Among successful school reform models, most commonly cited key features are: • High, clear, and fair academic standards • Adults who are equipped, empowered, and expected to improve instruction • What do teachers need to do differently?

3. Higher Levels of Accountability • Accountability for outcomes at all levels • Stewardship of resources • millions of $$ spent annually on staff development • school districts (on average) spend 3-5% of operating budget • State level – over past 5 years $30 million; cut this year to $18 million …….Where does it go? …….How is the money spent? …….What is the ROI? …….What is the impact on students?

4. No Child Left Behind “High Quality” • NCLB defines HQ professional development as….that which • uses research-based training & delivery models • is linked to school and district improvement models • uses student assessment data to determine whether there has been an impact on achievement • FLDOE supports this definition by requiring that districts use Professional Development Standards Protocol to design and deliver PD

5. Schultz Center Study • CEPRI noticed that the Schultz Center was moving to accomplish the intent • It took the initiative and made the effort to tackle this problem • Began to monitor participation data in PD, and relate that teacher-level data to the student outcomes achieved by schools

6. A more focused effort to study impact was commissioned……. • Whether developing or evaluating for effectiveness, there has to be a theory of action • A model for how we think things work – should determine choice of program components, and also what is measured

7. Improved or Enhanced Teaching Practices or Methods Increase in Teacher Knowledge and Skills Increase in Student Achievement High Quality Professional Development leads to and and Influence of Professional Development on Student Achievement

8. What was our research design and how did we proceed? • “gold” standard for evaluation and research is use of rigorous designs that allow strong conclusions • would require planning in advance to control for various conditions • Very difficult for school districts to manage • Was not the case for our study either

9. Research Design follows an orderly progression……….. • From less to more rigorous are: • Descriptive (what is happening?) • Causal-comparative (studying causal relationships whether randomization is possible) • Why or How is it happening? (developing and refining a theory)

10. What Students, Teachers, and Schools were included in our study? • Working in retrospect…. • Started with all teachers who had taken courses defined as “standards-based” content • Operationally defined “critical core” as completing certain series of courses • Considered state mandates for intensive reading • Realized that school to school differences and individual differences will occur

11. A Big Concern: was the group representative? • knowing we did not choose or select in advance • knowing that teachers have the prerogative • knowing that organizational and administrative issues at schools played a part Could we be assured that we had not built-in biases?

12. Yes – we were glad to see……… • All regions were represented in a fairly even spread of teachers • Settled on focusing on elementary (due to other mandates and time to implementation issue)

13. What were our results? • We focused on gain scores of students. • Note that gains by grade level differ. • Note that as PD increases, the gains increase. (This was the anticipated pattern.)

14. Status Achievement Levels Moving in right direction – need more data points to determine causation.

15. Where do we go from here? • Next steps proposed for 2005-2006: • Add to PD completion database to increase body of evidence • Validate and refine classroom rubrics and procedures for administering • Develop plan for studying impact of coaching as it relates to how PD becomes ‘imbedded’ • Determine a differentiated plan for how implementation data (classroom to school to school) is part of an aligned system for gauging districtwide progress toward systemic change.