Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting

1. Comparing the Effects of Two Versions of Professional Development on Science Curriculum Implementation and Scaling-Up Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting Montreal, Canada

2. Introduction and Overview Paul R. BrandonCurriculum Research & Development GroupUniversity of Hawai‘i at Mānoa

3. Project Purpose • A randomized study of the effects of variations in professional development (PD) on • program implementation • student achievement • scale-up • First phase (ending 2/28/06) to prepare for five-year second phase (if funded)

4. FAST • The project compares two versions of teachers’ PD for Foundational Approaches in Science Teaching (FAST). • An award-winning middle-school inquiry-based science program • Shown to positively affect achievement and other student outcomes

5. Phase-I Project Tasks • Prepare the 5-day PD institute with electronic resource and on-line course • Prepare instruments for the second phase: • Teacher questionnaire • Teacher log • Observation procedures • Student assessments

6. Focus on Span of PD • Traditionally, FAST PD is delivered in a 10-day institute. • We will compare it with an alternative 5-day institute followed by an online course, with an electronic multimedia resource.

7. Phase-II Hypothesis The 5-day inquiry-based science training institute, followed by an on-line university-credit course with computer-based multimedia, will have more favorable outcomes than a 10-day institute without the follow-up university course.

8. Examining the effects of PD on: • levels of classroom implementation • intensiveness and extensiveness of long-term use of inquiry-based science (i.e., scale-up) • levels of student achievement

9. Experimental Design • Randomized cluster sample, with schools (N = 80) as clusters • One teacher per school • Even if attrition is 25%, statistical power will be .80.

10. Merit of the Study • Addresses at least five deficits in the literature: • Few studies on effects of PD time span • Few studies about using technology in PD • Few randomized studies • Few studies of the effects of PD on student learning • Few, if any studies, of inquiry science PD

11. Order of Presentations • Gray, Nguyen, & Speitel: Description of PD • Brandon: Log and questionnaire development and questionnaire validation • Taum: Observation guide development • Ayala: Student assessment development • Lawton: Comparison of early effects of the two versions of PD

12. Introduction and Overview Paul R. BrandonCurriculum Research & Development GroupUniversity of Hawai‘i at Mānoa

13. Developing and Implementing an Alternative Version of FAST Professional Development Mary E. Gray ThanhTruc T. Nguyen Thomas W. Speitel University of Hawai‛i at Mānoa

14. FAST • Foundational Approaches in Science Teaching • Inquiry-based, middle school science • Physical, biological, and earth sciences • Exemplary program (USDOE, 2001)

15. FAST Professional Development • All FAST teachers must participate in PD • Certification required to purchase science curricula • Content • Inquiry • Standards • Assessment • Classroom organization and management • Safety

16. Year Round Schedules The Challenge To develop and implement an alternative version of FAST I professional development Teacher Time Resources Standards Licensure

17. Considerations • Current needs of science teachers • Future possibilities (emerging technologies) • Unique challenges (the dynamic nature of inquiry)

18. Review of Literature

19. Review of Literature Professional Development • Strong subject area content (Kennedy, 1999) • Focus on higher-order teaching strategies (Porter, Garet, Desimone, Yoon, & Birman, 2000) • Collaborative, Same subject-grade-school, Active learning, Consistent with teacher goals

20. Review of Literature Professional Development • Quantity linked to improvements (Radford, 1998; Supovitz, Mayer, & Kahle, 2000; Fishman, Marx, Best, & Revital, 2003) • science content knowledge, process skills, and attitudes • Quantity linked to standards based teaching (Supovitz & Turner,2000) • Problems with researching inquiry-based teaching and student learning (Fishman et al., 2003)

21. Review of Literature On-line Learning • 3 million adult learners on-line (Waitts & Lewis, 2003) • Convenience and increased flexibility (Hülsmann,1999) • Evidence of implementation of various pedagogical approaches(Schlager & Schank, 1997)

22. Review of Literature On-line Learning • Quantifiable learning outcomes were not significantly linked to technology adoption (Jones and Paolucci,1998). • Focus on motivation, skills and knowledge, self-directed learning, interactive competence, and technology skills(Kabilan, 2004) • Increased efficacy and self-perception in teachers(Huai, Braden, White, & Elliot, 2003)

23. FASTPro

24. Two-weeks in duration, face-to-face One-week in duration, face-to-face How are the traditional PD and FASTPro different? • Conduct 43 investigations • Conduct 19 investigations • Extensive modeling and practicing time • Electronic resource • WebCT-based learning community

25. Inquiry Teaching and learning science through inquiry is a new experience for many teachers and requires a significant change in attitude and behavior.

26. FASTPro – Addressing Inquiry • Strategies (Loucks-Horsley, Hewson, Love, & Stiles,1998) • Examples • Immersion in inquiry into science • Coaching and mentoring • Technology for professional learning

27. FASTPro – Addressing Inquiry • Change (Hord, Rutherford, Huling-Austin, & Hall,1987; Guskey, 2000) • takes time and persistence • awkwardness and frustration expected • as teacher’s progress through a change process, their needs for support and assistance change • Optimal Mix (Guskey, 2000) • A combination of teacher, program and change agent that will help create a positive relationship for PD to be effective

28. FASTPro

29. FASTStart • One week face-to-face institute • Teachers conduct nearly half of actual student investigations • Use inquiry methodology including modeling, discussion, assessment, and practice

31. Observe interactions of students and teachers View and reflect upon investigations not experienced See detailed step-by-step procedure suggestions Movies and animations Web interface Quicktime and Flash plugins DVD-ROM medium FASTeR

32. Examples from FASTeR

33. Examples from FASTeR • Slideshow 9, Density and the Cartesian Diver • Slideshow 22, Identifying Unknown Substances

35. FASTForward

36. Formative Findings • Suggest that teachers were able to implement successfully and share teaching practice • Added utility, focused and meaningful to teachers’ own environment

37. Infuse technological aspects into the FASTStart face-to-face experience Address credit equivalencies to enable more robust activity requirements Expand to include FAQ’s, indexing, and rich descriptions Expand to include a Website as well as DVD-ROM Automate some computer assisted instruction and feedback mechanisms Future considerations

38. Reflections… computer breakdowns scheduling Months of editing kids fascination with cameras

39. Reflections… • Teamwork and flexibility • Coordination of equipment and resources

40. Future Considerations • Building capacity • Application to other CRDG quality educational programs

41. Mahalo.

42. Developing and Implementing an Alternative Version of FAST Professional Development Mary E. Gray ThanhTruc T. Nguyen Thomas W. Speitel University of Hawai‛i at Mānoa

43. Instrument Development for a Study Comparing Two Versions of Inquiry Science Professional Development Paul R. Brandon Alice K. H. Taum University of Hawai‘i at Mānoa

44. Identifying Constructs • on teaching science with inquiry methods • Reviewed FAST and other inquiry science documents and worked closely with inquiry science experts. • on the context within which inquiry-based science is taught. • Reviewed 55 books and key articles on curriculum indicators and school effectiveness.

45. Developing the Teacher Log • Purpose: To identify the extent to which teachers implement the key features of inquiry-based science. • Reviewed the recent literature on logs. • Kept the instrument short to avoid overburdening teachers (21 items). • Is completed immediately after finishing each student science investigation.

46. Focus of the Teacher Log • Instrument addresses topics such as: • students’ questioning behaviors. • the teacher’s use of questioning strategies. • the teacher’s circulation about the classroom. • teacher-led discussions about variations in the data.

47. Developing the Teacher Questionnaire • Purposes: • To obtain information about implementation that is most appropriately collected once a year. • To collect information on implementation of investigations not covered on logs.

48. Focus of the Questionnaire • 150 items address topics such as: • implementation of key features. • the investigations taught during the entire year. • the adequacy of materials, equipment, etc. • teacher demographics. • teacher attitudes toward science. • teacher participation in science activities outside the classroom

49. Types of Validation Studies to Date • Two types of questionnaire-data validation studies conducted to date: • analyses addressing the relationship between the responses on the log and the questionnaire (“concurrent” validity) • an analysis examining the theoretical model underlying our study (construct validity)

50. Relationship Between Log and Questionnaire Responses • Compared results on total scores for five items that the two instruments had in common • Correlation = .54; effect size = .50