Can an Online Professional Development Project Impact Teacher Knowledge & Student Achievement?

1. Can an Online Professional Development Project Impact Teacher Knowledge & Student Achievement? Vicky Zygouris-Coe, Ph.D., Associate Professor & Principal Investigator, FOR-PD vzygouri@mail.ucf.edu Bonnie Swan, Ph.D., Program Evaluator bswan@mail.ucf.edu Catherine Glass, Director, FOR-PD cc@orion.itrc.ucf.edu Nancy Lewis, Ph.D. nlewis@mail.ucf.edu The 12th Annual Sloan-C International Conference on Asynchronous Learning Networks Orlando, FL November 9, 2006

2. Teacher Quality Matters • While teachers may come to the classroom fully competent to teach, ongoing changes in this new information environment require ongoing, effective professional development—a powerful cornerstone to reading achievement. • This need is even more pronounced in secondary schools.

3. Documenting Change Research Questions: • What impact can online professional development have on teachers’ knowledge of scientifically based reading research; and • What (if any) changes result in their teaching practice?

4. Background

5. MISSION • FOR-PD is a 14-week online course, that was designed to improve Florida K-12 students’ reading achievement. • It functions as a primary delivery mechanism to translate, empower, and support Florida teachers in using scientifically-based reading research and implementing innovative, creative, and effective strategies; and • Provides teachers with ongoing access to abundant, rich, and relevant reading resources.

6. Florida’s Plan for Developing & Supporting Teacher Expertise in Reading Florida Add-on Reading Endorsement Competency 1:  Foundations in Language and Cognition Competency 2:  Foundations of Research-Based Practices Competency 3:  Foundations of Assessment Competency 4:  Foundations of Differentiation Competency 5:  Application of Differentiated Instruction Competency 6:  Demonstration of Accomplishment (Practicum)

7. Background, cont’d • The project, which is housed at the University of Central Florida, was launched in January, 2003; and is • Funded by the Florida DOE Just Read, Florida! Initiative. • FOR-PD was developed collaboratively with • literacy and technology experts, • school districts, • professional organizations, and • teacher educators across the state of Florida. • The free online course is facilitated by reading specialists and other well-qualified educators.

8. Background, cont’d • In a report presented at the U.S.Department of Education Secretary’s No Child Left Behind Leadership Summit FOR-PD was named as an “innovative” e-Learning program for teacher training. Kleiman, 2004, p. 6 Kleiman, G. L. (2004, July). Meeting the need for high quality teachers: E-Learning solutions. White paper written for the U.S. Dept. of Education Secretary’s No Child Left Behind Leadership Summit: Increasing Options through E Learning, Orlando, FL. Retrieved March 6, 2006 from http://www.ed.gov/about/offices/list/os/technology/plan/2004/site/documents/Kleiman-MeetingtheNeed.pdf

9. Background, cont’d • The project is a highly-collaborative endeavor. Currently FOR-PD works with sixty-sevenFlorida districts, seven universities, and recently the project has beenextended through five Community College Educator PreparationInstitutes (EPIs).

10. Large Scale To date (4 yrs) FOR-PD has served almost 25,000 participants statewide.

11. FOR-PD Course Content • Content is based on current scientifically-based research and is updated frequently • 14 Lessons • Online Text • Online Support Materials & Resources • Lessons include online discussions in an interactive environment • Assessment tools include online quizzes and assignments including a Literacy Log which can document evidence of best practices • Face-to-face components are sometimes added on an as-needed basis

12. How do we assess what teachers learn and do in a large-scale online course of professional development?

13. Evaluation Tools

14. FOR-PD Evaluation Tools • Pre- and post-test of reading knowledge (TREKA) • Discussion Boards • Facilitator • Participant • School-level student achievement • Analyze participant work samples • Data-base analysis • Quality Assurance Monitoring

15. Evaluation Tools, cont’dFocus Group and Telephone Interviews • Facilitator and participant focus groups and large group discussions conducted at annual meetings • District contact interviews • Content-area teacher interviews • Facilitator interviews

16. Evaluation Tools, cont’dWeb-based Surveys • Facilitator • End-of-course survey • Training course survey • Participant • Non-completer survey • Implementation survey • End-of-course survey • Help desk • QAC • Facilitator Survey

17. Evaluation Tools, cont’dWork Sample Example: Literacy Log

18. Results of Analysis

19. What FOR-PD Facilitators Are Saying… • I fully believe FOR-PD has made me a more effective facilitator and my students [participants] more successful learners.

20. FOR-PD FacilitatorsResults from End-of-course Surveys

21. Results of AnalysisLiteracy Logs • Analysis of FOR-PD literacy logs have shown: • Teachers are provided with effective avenues toward becoming more reflective in their teaching practice; and • Providing graphic organizers enables teachers to organize their thoughts on a particular topic or lesson and generate ideas for applying their knowledge and strategies at the classroom and school levels.

22. FOR-PD TeachersResults from Classroom Implementation Survey

23. FOR-PD TeachersResults from Classroom Implementation Survey

24. FOR-PD TeachersResults from Classroom Implementation Survey

25. FOR-PD ParticipantsResults from End-of-course Surveys

26. Reading Knowledge Assessment:Results of TREKA Pre to Post, Spring 2006 • Pretest means were similar from one group to another. • Substantial gains on reading knowledge after completing the FOR-PD course: (t(1491) = 43.36, p < .0005). • A large effect size: the eta squared value = .558. • The true mean difference (95% C.I.): 9.4 to 10.3. • Posttest means were similar from one group to another. Participants’Pre and Posttest Mean Scores

27. FOR-PD Teachers–Reading StrategiesResults from Classroom Implementation Surveys

28. FOR-PD Teachers–Reading StrategiesResults from Classroom Implementation Surveys

29. FOR-PD Teachers–Reading StrategiesResults from Classroom Implementation Surveys • Almost all participants (94%) who responded indicated that FOR-PD positively impacted their classroom instruction. • All but 2% of respondents indicated they regularly use FOR-PD strategies with K-12 students. Most (65%) use the strategies they learned 3-or-more times a week. • Most respondents (83%) indicate that they now implement instructional strategies they learned from FORPD before, during, and after reading related instruction on a regular basis. • Differentiated instruction techniques learned in FOR-PD are regularly used by 74% of respondents. • Many (77%) indicated their classroom environment was print-rich, and that they regularly made use a classroom library and word walls. • Sixty-nine percent now use screening diagnosis and progress monitoring to better meet the needs of their students.

30. FOR-PD TeachersResults from End of Course Surveys

31. Focus GroupsParticipants • “I know that everything is data driven. They’re [my students] not taking the FCAT, but their reading level, and I’ve got all the data, has gone up on average of my eleven kids, two grade levels. It is working for my kids.”

32. Focus GroupsFacilitators • “Even though I am not a reading teacher, I needed to have a base understanding of reading so that I could make sure I support the classroom teacher and get the appropriate accommodations for the students in my class. And it helped me do that – get a stronger base of my own knowledge.”

33. FOR-PD TeachersResults from Classroom Implementation SurveysParticipants Report Improvement in Students’ Reading Performance * *See next slide for explanation

34. FOR-PD TeachersResults from Classroom Implementation Surveys Why Some Participants Do Not Believe FOR-PD has a Positive Impact on FCAT Scores Note. Data were obtained from Phase IV Classroom Implementation Survey (n = 430).

35. Summary of Results • Analysis of data from ongoing, rigorous, and well embedded evaluation show that almost all participants use scientifically-based reading strategies for reading instruction taught in FOR-PD; and believed the course helped them to help their students learn.

36. Research has shown that professional development that supports ongoing, collaborative learning is vital for improving student outcomes.

37. QUESTIONS

38. Please visit the FOR-PD website: http://www.itrc.ucf.edu/forpd Thank you!

39. References • Kleiman, G. L. (2004, July). Meeting the need for high quality teachers: E-Learning solutions. White paper written for the U.S. Dept. of Education Secretary’s No Child Left Behind Leadership Summit: Increasing Options through E-Learning, Orlando, FL. Retrieved March 6, 2006 from http://www.ed.gov/about/offices/list/os/technology/plan/2004/site/documents/Kleiman-MeetingtheNeed.pdf • Swan, B., Huh, J., & Ramos, R. (August, 2006). Florida Online Reading Professional Development (FOR-PD) Phase IV Evaluation Results of Surveys. Orlando FL: University of Central Florida