Separating Facts From Fads: The Evidence That Educators Need for Effective Science Instruction and Policy Decisions

1. 1 Separating Facts From Fads: The Evidence That Educators Need for Effective Science Instruction and Policy Decisions Philip M. Sadler, Ed.D. Director of Science Education Harvard-Smithsonian Center for Astrophysics F.W. Wright Senior Lecturer Harvard University Department of Astronomy

2. Abstract U.S. schools are unique in the variety of teaching methods and curricula used for teaching science. Freedom to choose pedagogies and materials are most often vested with the classroom teacher. Because of this natural variation, we have utilized epidemiological methods to mine the backgrounds of college students taking introductory science courses for predictors of performance and persistence while controlling for demographic differences. In surveying thousands of students in randomly selected introductory college biology, chemistry, and physics courses across the U.S., we have put to the test educators' beliefs about the kinds of preparatory experiences and key resources that predict successful performance in college science. I will report on our findings on the value of lab experience, technology, demonstrations, content coverage, block scheduling, class size, Advanced Placement courses, Physics First, project work, and mathematics preparation. We have also gauged the effectiveness of classroom instruction at the middle school level, examining the role of teacher subject matter knowledge and pedagogical content knowledge on student gains. Of particular interest is teacher awareness of common student misconceptions and how differing forms of assessment play out in the measurement of students' conceptual understanding. Confronting the Math and Science Challenge, East Tennessee State University

3. What is so important about science and math? Basis of economic prosperity and national security Knowledge acquired over very long time periods Lots of variety in the educational system Small �hot house� studies are over-generalized Preparation for success in college �gate-keeper� courses

4. Sadler�s Conundrum Teachers� Claim: I prepare students well for success in their next course. Students are not well prepared for success in my course

5. Questions that motivated this study Why do we teach science?

6. Questions that motivated this study Why do we teach science? What do teachers believe works best for preparing students for later study in science?

7. With limited time and money, where do you put your resources? Advanced Placement Block scheduling Labs and demonstrations Assessment Instructional practices Technology Facts vs Concepts Coverage Physics First Mathematics Teacher Knowledge

8. Questions that motivated this study Why do we teach science? What do teachers believe works best for preparing students for later study in science? What evidence do teachers have about what works?

9. FICSS: Factors Influencing College Science Success $3M, 4-year IERI (Inter-agency Educational Research Initiative) study Investigate the kinds of high school experiences that best prepare college students for: introductory courses in biology, chemistry, or physics Drawing hypotheses from teachers, professors, and researchers Sample of 18,000 college students at 55 randomly chosen colleges and universities

10. Is Advanced Placement the Answer?

11. What the public hears �It is better to take a tougher course and get a low grade than to take an easy course and get a high grade.� Clifford Adelman, Senior Research Analyst, U.S. Dept. of Ed.

12. Surprise! AP students often take introductory college courses in science

13. Accounts for half of the observed higher grades

14. Difference in Performance in �102� for Students Who Took AP in High School

15. Conclusions AP students do earn somewhat higher grades in college science Partial proxy for demographic, general scholastic performance, math preparation Will not make up for poor earlier preparation AP exam performance alone is not sufficient support for advanced standing in college Retaking intro courses benefits AP students

16. Block Scheduling

17. Block Scheduling 1/2 schools use block scheduling Allows more flexibility longer labs projects team teaching, fieldwork peer tutoring

18. Block Scheduling Little difference in teaching approaches that are associated with increased learning Hence, no significant difference in overall effectiveness A students +1pt, C students -1pt

19. 19 Changing the Order:Physics First

20. Testing the Physics First Hypotheses Taking more physics will have a positive impact on later learning in chemistry Taking more chemistry will have a positive impact on later learning in biology Control for covariates: SES, verbal and prior achievement

21. Mathematics Effect

22. Physics First Taking courses in the same science helps later No cross-disciplinary effect seen in science Support for strong math preparation 4 years in HS To pre-calculus or calculus

23. Technology

24. Background In 1997, the President�s Council of Advisors on Science and Technology issued a report to President Clinton that widespread federal support should be given to school districts to catalyze and continue instructional technology proliferation in US public schools This report resulted in the formation of the e-Rate Program that subsidized technology infrastructure improvements in US public schools PCAST also cited the dearth of research on best practice in instructional technology use and implementation, and noted that research in this area should be supported

25. Background All totaled, 65 research articles in major peer-reviewed science education journals with 73.8% of these research studies reporting positive outcomes But� None with sample sizes greater than 500 students (can not be considered �nationally representative�) 46 articles did not provide enough information for effect sizes to be calculated, with 28 articles that were purely qualitative studies 12 articles employed a pre/post test design, but did not include a comparison group Meta-analysis, Effect Size: 0.74 SD for all studies 0.26 SD for well-controlled studies

26. Success in First Semester Science Major uses of technology in high school science Computer: graphing and writing Probes: for taking data automatically Simulation: genetics, mechanics Internet: web research and animations

27. Success in First Semester Science Major uses of technology in high school science Computer: graphing and writing Probes: for taking data automatically Simulation: genetics, mechanics Internet: web research and animations Biology � 2683 students at 29 colleges/univ. Chemistry � 3417 students at 31 colleges/univ. Physics � 1792 students at 37 colleges/univ.

28. Three Discipline Summary, N>7000

29. Research Findings indicated that high school science Instructional Technology use was either not significantly or negatively associated with Introductory College Science Performance as measured by final course grades On average, technology implementation does not appear to improve student learning in science

30. 30 Pedagogy and Curriculum

31. 31 Pedagogy and Curriculum

32. What Appears to: Help: Often Analyzed Pictures or Illustrations Often Draw/Interpret Graphs by Hand Emphasis on quantitative problems Labs Addressed Student�s Beliefs More prediction, less demo discussion Testing for facts Focus on key foundational concepts

33. The Impact of Coverage: Depth vs. Breadth

34. The Impact of Coverage: Depth vs. Breadth In teaching my high school science course so that students are well-prepared for college science, I make sure that we cover: All the major topics so that students are familiar with most terms and concepts A few key topics in great depth so that students have mastered a essential foundational concepts

35. The Impact of Coverage: Depth vs. Breadth

36. Conclusions Pedagogy makes a difference Labs that address student ideas Labs with simpler procedures Demonstration with prediction Integrating mathematics, especially graphing Coverage makes a difference Key topics and concepts Avoidance of covering �everything�

37. 37 How effective are we at teaching foundational concepts?

38. Confronting the Math and Science Challenge, East Tennessee State University Psychological Foundations �The unlearning of preconceptions might very well prove to be the most determinative single factor in the acquisition and retention of subject-matter knowledge.� David Ausubel 1978 Some of the philosophy that undergird my beliefs: David Ausubel Karl PopperSome of the philosophy that undergird my beliefs: David Ausubel Karl Popper

39. 5-8 Physical Science: Transfer of Energy � Electrical circuits provide a means of transferring electrical energy when heat, light, sound, and chemical changes are produced.

40. Student Preference

41. 5-8 Physical Science: Motions and Forces � The motion of an object can be described by its position, direction of motion, and speed. That motion can be measured and represented on a graph.

42. Which answers do your students give?

43. Teacher Content and Predictive Knowledge

44. HS Chemistry

45. HS Physics

46. Relationship between Teacher and Student Knowledge

47. Comparison of HS Chemistry Teacher PCK and SMK

48. Comparison of HS Physics Teacher PCK and SMK

49. Yearly Classroom Gain in Middle School Physical Science Courses, N= 172 teachers

50. Confronting the Math and Science Challenge, East Tennessee State University What do we know about conceptual understanding? Misconceptions often unchanged after taking science. Necessary step in learning The standards are hard to master. Teachers are knowledgeable (with some prominent holes in content), but this does not assure student learning. Teachers do not know their students� misconceptions, but should.

51. Patterns in Professional Development Data

52. Which factors predict teacher content knowledge of the curriculum concepts? Grade level Gender Years Teaching Years Teaching science subject Certification in the science subject Degrees (BS, BA, MS, PhD) Grad Courses taken in domain Professional development in science teaching/content

53. Interaction of Years Teaching Subject and Certification

54. 2-Week Institute

55. 2-Week Institute

56. 1-Week Astronomy Institute

57. Comparison of 2 MSP Institutes

58. Patterns in Professional Development Some teacher content weakness at all grade levels: weakest at MS levels Content knowledge grows very slowly for the non-certified teacher Professional development can make a difference in teacher content knowledge Length of program Focus on content knowledge at grade level vs. �science apprenticeships� Activity-based vs. project-based differences Must evaluate the fulfillment of goals Content knowledge at higher levels does not translate to knowledge at lower levels

59. Conclusions

60. Separating Facts from Fads AP is valuable for advanced students, but not a replacement for college Block scheduling is not helpful unless pedagogy changes Changing course sequence does not aid preparation Increase math requirements and integration Technology use must be targeted, should not replace skill development Classroom pedagogy More: math, prediction, discovery labs, testing facts Less: teaching facts,validation labs, lab prep Coverage: fewer topics, more time on misconceptions and key concepts Professional Development: teachers must know both science content and student misconceptions targeted to content at grade level.

61. Publications to date 1997-The Role of High School Physics in Preparing Students for College Physics. The Physics Teacher. 2001- Success in College Physics: The Role of High School Preparation, Science Education. 2001- Gender Differences in Introductory Undergraduate Physics Performance. Int. Journal of Science Education 2005- Factors influencing success in introductory college chemistry. Journal of Research in Science Teaching 2006- Breaking from Tradition: The Unfulfilled Promise of Block Scheduling, The High School Journal 2006- Factors Influencing College Science Success, Journal of College Science Teaching 2006- High school chemistry content background of introductory college chemistry students and its association with college chemistry grades. Journal of Chemical Education.

62. MOSART Websitewww.cfa.harvard.edu/smgphp/mosart

63. Annenberg Channel

64. Acknowledgments Co-investigators: Matthew Schneps, Roy Gould, Robert Tai (University of Virginia) Project Manager: Hal Coyle, Gerhard, Sonnert, Michael Filisky Survey Staff: Jamie Miller, Nancy Cook Smith, Cynthia Crockett, Marc Schwartz (McGill), Annette Trenga, Bruce Ward, Bruce Gregory Video Staff: Yael Bowman, Toby McElheny, Nancy Finkelstein, Alexia Prichard, Alex Griswold Graduate Students: Zahra Hazari, John Loehr Advice Elizabeth VanderPutten, Janice Earle, Joyce Evans, Barry Sloane, Larry Suter of the National Science Foundation, Marcus Leiberman, Responsive Methodologies Financial support Harvard Graduate School of Education, NSF, DoEd, NIH, Annenberg/CPB Foundation. Center for Astrophysics Irwin Shapiro, Susan Roudebush, Judith Peritz.

65. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the  National Science Foundation, National Institutes of Health, U.S. Department of Education

66. Harvard-Smithsonian Center for AstrophysicsScience Education Department 60 Garden Street, MS-71 Cambridge, MA 02138 Phone: 617-496-7598 Fax: 617-496-5405 Email: psadler@cfa.harvard.edu