Transforming Teacher Professional Development into Classroom Practices: Using Mixed Methods to Understand Teacher Pract

1. Transforming Teacher Professional Development into Classroom Practices: Using Mixed Methods to Understand Teacher Practice Bridget A. Cotner Kathryn M. Borman Theodore L. Boydston Rheta Lanehart Margaret Karnyski Maressa Dixon Amy Moorer Janelle Christensen University of South Florida In this paper, our research team used mixed methods to understand how teachers in a professional development program, called Teaching SMART, use what they learn and apply it to the classroom.In this paper, our research team used mixed methods to understand how teachers in a professional development program, called Teaching SMART, use what they learn and apply it to the classroom.

2. Research Questions Four guiding research questions: How do teachers perceive their instructional practices? What professional development activities are teachers most engaged in? What types of professional development do teachers report as being the most beneficial? How does professional development influence what is taught in the classroom? There are four research questions guiding this study. Read the four bullets.There are four research questions guiding this study. Read the four bullets.

3. Sample and Data Sources 245 teachers surveyed 4850 students surveyed 5 professional development rubrics 40 classroom observations 95 professional development documents 570 lesson feedback forms 39 teacher interviews The sample and data sources for our analysis presented in this study are: Teachers surveyed: these are teachers who participate in the Teaching SMART professional development program (our treatment teachers) and teachers who are not in the program, our control teachers. Students: these are the students of the teachers who were surveyed. Professional development rubrics, these were completed by the research team members during the professional development training. Classroom observations were conducted with a subset of the 245 teachers that were surveyed. Professional development documents are pieces of paper that teacher were asked to describe what they liked about implementing Teaching SMART. Lesson feedback forms that the treatment teachers were asked to completed after they conduct a Teaching SMART science lesson. Teacher interviews were conducted with a subsample of the teachers who were surveyed. The sample and data sources for our analysis presented in this study are: Teachers surveyed: these are teachers who participate in the Teaching SMART professional development program (our treatment teachers) and teachers who are not in the program, our control teachers. Students: these are the students of the teachers who were surveyed. Professional development rubrics, these were completed by the research team members during the professional development training. Classroom observations were conducted with a subset of the 245 teachers that were surveyed. Professional development documents are pieces of paper that teacher were asked to describe what they liked about implementing Teaching SMART. Lesson feedback forms that the treatment teachers were asked to completed after they conduct a Teaching SMART science lesson. Teacher interviews were conducted with a subsample of the teachers who were surveyed.

4. Method A mixed method approach to data collection: Quantitative Analysis Teacher and student surveys Professional development rubric Classroom observation forms Document analysis Professional development documents Lesson feedback forms Thematic Analysis Teacher interviews We used a mixed method approach for collecting data to help us understand how teachers take what they learn in professional development and apply it to their classrooms. Teachers and students were given surveys to help us understand how science instruction was perceived prior to the implementation of the professional development program. These surveys are re-administered at various time points in the study to document change in attitude or perceptions of practice. We completed a PD rubric during the initial professional development training to identify the activities in the professional development that most engaged the teachers. Three months after the initial professional development, we conducted classroom observations and completed a classroom observation form to determine if there is a difference in science instructional practice between teachers in the treatment program and control teachers. These three methods of data collection were analyzed using quantitative methods. We also collected two types of documents- one from a professional development training that asked the teachers to describe what they liked about implementing Teaching SMART. The second type of document that was the lesson feedback form. This form was completed by treatment teachers after they implemented a Teaching SMART lesson. Lastly, we conducted teacher interviews with a subset of our participating treatment and control teachers to obtain their perspectives on how they teach science. A thematic analysis was conducted to identify the main themes.We used a mixed method approach for collecting data to help us understand how teachers take what they learn in professional development and apply it to their classrooms. Teachers and students were given surveys to help us understand how science instruction was perceived prior to the implementation of the professional development program. These surveys are re-administered at various time points in the study to document change in attitude or perceptions of practice. We completed a PD rubric during the initial professional development training to identify the activities in the professional development that most engaged the teachers. Three months after the initial professional development, we conducted classroom observations and completed a classroom observation form to determine if there is a difference in science instructional practice between teachers in the treatment program and control teachers. These three methods of data collection were analyzed using quantitative methods. We also collected two types of documents- one from a professional development training that asked the teachers to describe what they liked about implementing Teaching SMART. The second type of document that was the lesson feedback form. This form was completed by treatment teachers after they implemented a Teaching SMART lesson. Lastly, we conducted teacher interviews with a subset of our participating treatment and control teachers to obtain their perspectives on how they teach science. A thematic analysis was conducted to identify the main themes.

5. Findings: Baseline Science Instruction Student and Teacher Surveys at Baseline: Analysis of the student and teacher surveys that were administered prior to the professional development training revealed that both treatment and control teachers and their students reported more traditional types of instructional practice in science, such as seatwork. Using science inquiry and science activities were rated low. Likewise, students� perceptions of their teacher�s practice depict a similar picture of science instruction with traditional types of instruction occurring more often than science inquiry.Analysis of the student and teacher surveys that were administered prior to the professional development training revealed that both treatment and control teachers and their students reported more traditional types of instructional practice in science, such as seatwork. Using science inquiry and science activities were rated low. Likewise, students� perceptions of their teacher�s practice depict a similar picture of science instruction with traditional types of instruction occurring more often than science inquiry.

6. Findings: Quantitative Analyses Professional development rubric: Teacher engagement During the initial professional development training, we completed a rubric to rank the engagement level of the teachers. Using Friedman�s chi-square, a higher ranking of engagement level was indicated at times when the teachers were involved in interactive activities, i.e., Balancing Act, Activity Modeling, and the Follow-up discussion of the model lesson. During the initial professional development training, we completed a rubric to rank the engagement level of the teachers. Using Friedman�s chi-square, a higher ranking of engagement level was indicated at times when the teachers were involved in interactive activities, i.e., Balancing Act, Activity Modeling, and the Follow-up discussion of the model lesson.

7. Findings: Quantitative Analyses Teacher survey: End of initial training evaluation At the end of the initial professional development training, the teachers completed a survey. One of the items asked the teachers what they liked best about the training. The teachers responded that they liked the doing the activities and having them modeled to them by the professional development leader as well as having the time to learn and share ideas and strategies for instruction. This finding supports what the finding from the professional development rubric- that both the teachers and the researchers identified the hands-on activities, modeling and discussion as being the most engaging and what the teachers like best.At the end of the initial professional development training, the teachers completed a survey. One of the items asked the teachers what they liked best about the training. The teachers responded that they liked the doing the activities and having them modeled to them by the professional development leader as well as having the time to learn and share ideas and strategies for instruction. This finding supports what the finding from the professional development rubric- that both the teachers and the researchers identified the hands-on activities, modeling and discussion as being the most engaging and what the teachers like best.

8. Findings: Quantitative Analysis Classroom Observation Forms: Synthesis of key indicator ratings for Teaching SMART and control observations Results from our classroom observation forms indicate that the teachers in the Teaching SMART professional development program taught science using more science inquiry and hands-on activities than the control teachers. Thinking back to our baseline survey findings that indicated treatment and control teachers were similar in reported practice, after three months of professional development with Teaching SMART, the treatment teachers are demonstrating higher levels of science inquiry than the control teachers. These classroom observations were conducted 3 months into the implementation of the professional development program, Teaching SMART. Results from our classroom observation forms indicate that the teachers in the Teaching SMART professional development program taught science using more science inquiry and hands-on activities than the control teachers. Thinking back to our baseline survey findings that indicated treatment and control teachers were similar in reported practice, after three months of professional development with Teaching SMART, the treatment teachers are demonstrating higher levels of science inquiry than the control teachers. These classroom observations were conducted 3 months into the implementation of the professional development program, Teaching SMART.

9. Findings: Document Analysis Professional Development Documents: What teachers like about implementing Teaching SMART Teachers were pleased with the Teaching SMART lesson materials provided (microscopes, beakers, owl pellets, etc.) as well as the structure and organization of the Teaching SMART lessons themselves. Many teachers remarked that both they and the students really enjoy the Teaching SMART lessons, that they are fun, and that the students looked forward to those days when they would have a Teaching SMART lesson. Teachers felt that the on-going support provided by the Grant Resource Teachers, made the adoption and implementation of the Teaching SMART lessons easier. In a follow-up professional development session, treatment teachers indicated what they liked about implementing Teaching SMART. Having the materials to do the experiments, knowing that the students enjoyed the lessons, and having the on-going support all help the teachers implement the professional development program in their classrooms.In a follow-up professional development session, treatment teachers indicated what they liked about implementing Teaching SMART. Having the materials to do the experiments, knowing that the students enjoyed the lessons, and having the on-going support all help the teachers implement the professional development program in their classrooms.

10. Findings: Document Analysis Lesson feedback forms: On teacher surveys from the initial training, teachers indicated if they Liked or Liked Very Much a particular lesson. These lessons were the ones that were taught more often than any other lessons. When the treatment teachers teach a Teaching SMART lesson in the classroom, they complete a lesson feedback form. We collected those forms and found that the lessons taught the most were the ones that were presented in the professional development training. When the treatment teachers teach a Teaching SMART lesson in the classroom, they complete a lesson feedback form. We collected those forms and found that the lessons taught the most were the ones that were presented in the professional development training.

11. Findings: Thematic Analysis Teacher interviews: 100% of the treatment teachers have had at least some science professional development compared to only 50% of the control teachers. Among the participating treatment teachers interviewed, one strength of the professional development is doing the activities and/or having them modeled to them in the training. The teachers in the interviews said this allowed them to get first-hand knowledge of how to teach the particular activities in their classrooms.

12. Conclusion When teachers are able to work with instructional materials in professional development, they are more engaged and become better able to implement those activities and strategies in the classroom Both treatment and control teachers and their students perceived science instructional practices to be more traditional (seatwork) before the professional development training. With on-going training and support, the treatment teachers were observed conducting hands-on activities and using science inquiry more than the control teachers Treatment teachers taught lessons that they were exposed to and liked in the training more often than science lessons that they have not seen or experienced Using multiple methods of data collection assists researchers in understanding and documenting change in practice