Using Data to Evaluate the Effectiveness of Professional Development

1. Nebraska Department of Education 2012 Data Conference It's More Than Numbers April 2-3, 2012 Using Data to Evaluate the Effectiveness of Professional Development Sue Anderson, Ph.D., ESU 3 Don Johnson, Fort Calhoun Community Schools Bill Menousek, ESU 3

2. Using Data to Evaluate the Effectiveness of Professional Development • Session Topics • Project Rationale • Project Description • District Perspective • Evaluation Design • Results

3. Project Design Rationale Characteristics of high quality professional development: • Offered for a longer duration and greater frequency • Involves teachers directly for more hours in active, engaged learning activities and environments • Focused on a particular content area and allows teachers to gain knowledge on how to teach the content to their students Desimone, L. (2009). How can we best measure teacher’s professional development and its effects on teachers and students? Educational Researcher 38(3): 181-199.

4. Project Design Rationale Characteristics of high quality professional development: • Coherent to teachers’ needs and circumstances • Involves teachers learning from their peers through collective participation Desimone, L. (2009). How can we best measure teacher’s professional development and its effects on teachers and students? Educational Researcher 38(3): 181-199.

5. Project Description Professional Learning Model: Improving Vocabulary and 21st Century Skills

6. Professional Learning Model Overview

7. The 3 Pillars of the Face-2-Face Learning Program TECHNOLOGY CONTENT PEDAGOGY 2. The pedagogy will always include the identification and implementation of 21st century skill(s) in teaching. 1. The content will reference and be clearly focused on the goal of the project. 3. The inclusion of technology will be appropriate to support #1 & #2.

8. Face-2-Face Framework • The goal of the project is reiterated. • The content is delivered and a video of the instruction • is made available on the Fort Calhoun web site. • Support documentation for PLGs is distributed and explained. • Input for videoconference follow-up is gathered. • A brief preview of the next F-2-F learning is presented.

9. Videoconference

10. Project Participants • Fort Calhoun • K-12 teaching staff • Administrators • ESU #3 PDD Staff • Consultants • CIA • Instructional Technology • Data Specialist • Technology Support Specialist

11. School District Perspective

12. Project Evaluation Evaluation Design Findings Conclusions Recommendations

13. Project Evaluation - Design • Data Collection • Surveys (FC staff – one per cycle) • Survey (ESU #3 staff – end of project) • Interviews • Artifacts • Assessment Data – NeSA /NWEA MAP Reading/Vocabulary • Data Analysis • Excel formats (tables, charts, graphs) • Coding for themes

14. Project Evaluation-Design • Survey Item Topics • Confidence in PD content and strategies • Implementation of strategies • Project format • PLGs • Student engagement

15. Findings

16. School District Survey Findings

17. School District Survey Findings

18. School District Survey Findings • *Project Content • Elementary staff more favorable to content • Videoconference sessions not as effective • PLGs not utilized effectively • **Project Format • Preferred face-to-face sessions • Videoconference sessions not as effective • PLGs not utilized effectively

19. ESU #3 Staff Survey Findings

20. ESU #3 Staff Survey Findings

21. Student Test Data Findings • NeSA Reading Vocabulary 2010 and 2011 • Gains in only two grade levels (grades 3 and 6) • Significant decline in Grade 11 • NWEA MAP Reading Vocabulary 2010-2011 • Three test administrations • Overall increases in grades (3,4,5,6,7,9) • Significant decline in Grade 11

22. Conclusions

23. Project Conclusions • Survey/interview data support criteria for effective PD • content reflecting best available research and practice • content is coherent to teachers’ needs • opportunities to learn content based instructional strategies • appropriate technologies to support student learning

24. Project Conclusions • professional learning experiences for a longer duration and greater frequency • Involve teachers in learning from their peers through collective participation • Evaluate the effectiveness of the professional learning experience.

25. Recommendations

26. Project Recommendations • General • Align PD with research-based practices • Fort Calhoun • Continue to focus on vocabulary achievement • Focus on Grade 11 achievement • Review expectations for PLGs

27. Project Recommendations • ESU #3 • Continue to align PD planning with research-based criteria for effectiveness • Utilize collaboration as development and implementation strategies • Incorporate 21st century skills and technology • Expand the analysis of student achievement data • Include evaluation processes • Align PD directly with student learning outcomes (i.e. goals, standards, indicators, etc.)

28. Project Recommendations • General • Align PD with research-based practices • Fort Calhoun • Continue to focus on vocabulary achievement ( Grade 11) • Review expectations for PLGs

29. Project Recommendations • ESU #3 • Continue to align PD planning with research-based criteria for effectiveness • Utilize collaboration as development and implementation strategies • Incorporate 21st century skills and technology

30. Project Recommendations • ESU #3 • Include evaluation processes • Expand the analysis of student achievement data • Align PD directly with student learning outcomes (i.e. goals, standards, indicators, etc.)

31. Comments…Questions