David McConnell, Geology University of Akron May 2005

1. NAGT Observing and Assessing Workshop Hey, Technology Boy:It’s the Message, Not the Medium David McConnell, Geology University of Akron May 2005

2. Conceptest Group Members David McConnell, University of Akron David Steer, University of Akron Katharine Owens, University of Akron Jeffrey R. Knott, California State University, Fullerton Stephen Van Horn, Muskingum College Walter Borowski, Eastern Kentucky University Jeffrey Dick, Youngstown State University Annabelle Foos, University of Akron, Michelle Malone, Western Washington University Heidi McGrew, University of Dayton

3. Constructivist Teaching Reform • Instructors organize their experiences and observations into patterns or mental models about teaching. • Perceptions of effective teaching shaped by observations as students, teaching assistants, and peers. • It is reasonably easy for instructors to learn something that matches or extends an existing mental model. • Easier to make modest changes tied to existing procedures, e.g., using techniques such as peer instruction or minute paper. • It is difficult for instructors to make substantial changes to an established mental model. • Needs compelling evidence for change – traditional teaching methods ineffective and alternatives show improvement. • Significant changes need sustainable institutional support.

4. Teaching Preconceptions Conceptest Three instructors taught same Physics course in same semester. All professors use the same textbook, cover the same chapters, and receive similar evaluations. Prof A- Lectures emphasized conceptual structure of physics, careful definitions, logical arguments (97 students). Prof B- Demonstrations in lectures, extra time and energy in lecture prep, two teaching awards (192 students). Prof C- Emphasized problem solving, taught by example, solving multiple problems in lectures (70 students). Which class did best on common post-test? 1.A2.B3.C 4.No difference Halloun, I.H. and D. Hestenes, American Journal of Physics, 1985. 53(11): p. 1043-1055.

5. Peer Instruction Pedagogy (Mazur) • Peer instruction (& Conceptests) • Instructors in present short lecture segments (10-20 minutes – limit of student engagement) • Conceptest – conceptual multiple choice question to evaluate student understanding/application of key concept • Students consider conceptest, signal their answer (raised hands, colored cards, classroom performance system) • Based on proportion of correct responses (35-70% optimal), students discuss answers in small groups (2-4), respond • Student or instructor explanation of correct answer Mazur, E., 1997, Peer instruction: A user’s manual: Prentice Hall, 253p.

6. Impact of Alternative Pedagogy Traditional Class Peer Instruction Classes Mazur’s results for Introductory Physics using Force Concept Inventory (FCI) 3 4 2 1 1 Began PI FCI score gain on post-test Refined conceptests 2 3 Changed text FCI pretest score Open ended reading questions 4 n = 117 - 216 Crouch, C.H., Mazur, E., 2001, American Journal of Physics, v. 69, #9, p.970-977

7. Geoscience Conceptest Example Examine the map and answer the question that follows. How many plates are present? a. 3 (26%; 0%) c. 5 (44%; 75%) d. 6 (11%; 7%) b. 4 (19%; 18%) Individual responses Post-discussion responses

8. Impact of Alternative Pedagogy Comparison of difference among pre- and post-test scores for the Geosciences Concepts Inventory (GCI) Geoscience Results • GCI - Standardized measurement instrument • 55% of classes showed no statistical improvement on GCI over the course • No information on teaching strategies for other courses Courses utilizing conceptests Libarkin, J.C., and Anderson, S.W., 2005, Journal of College Science Teaching.

9. Why Peer Instruction Works Seven Principles of Good Practice • Encourages student-faculty contact • Develops cooperation among students • Encourages active learning • Provides prompt feedback • Emphasizes time on task • Communicates high expectations • Respects diverse talents and ways of learning “. . . what the student does is actually more important in determining what is learned than what the teacher does.” Thomas Shuell Chickering & Gamson, AAHE Bulletin, 1987

10. A National Conceptest Database? Chemistry, physics, and astronomy all have extensive conceptest databases • What can we learn from their experience? • Sharing is good – add questions to a common database • Add another step – pre/post response statistics for individual conceptests for comparative assessment – assess student success with questions • Identify bottleneck concepts vs. basic concepts – Which concepts are the most difficult to learn? A geology conceptest question database is available on-line at: http://serc.carleton.edu/introgeo/

11. Impact of traditional instruction Pre- and post-tests of student comprehension were compared for large introductory biology courses for non-majors and majors with comparable class sizes. The majors course presented more content. Mean pre-test score for the non-majors was 29% and the for the majors was 35%. • Predict the post-test score in the courses for non-majors and majors. • 35/41% c. 40/48% • b. 40/41% d. 48/56% Sundberg, M.D., M.L. Dini, and E. Li, Journal of Research in Science Teaching, 1994. 31(6): p. 679-693.

12. Geoscience Conceptest Example Liquid hazardous waste is disposed off by pumping it down injection wells. Which well location would be the most suitable to use for an injection well? Why? A B C

13. Impact of Alternative Pedagogy Student Perceptions of Conceptests Student perceptions of the use of conceptests as a useful teaching and learning tool in an introductory geology lab course at Western Washington University (n = 72) (Data from Michelle Malone) McConnell, D.A., and others., Journal Geoscience Education, 2006

14. The Value of Conceptests Students taught key concepts using one of four methods. Student learning assessed by proportion of correct answers to open ended questions on same concepts on final exam % correct answers Teaching method No demonstration Observation of demonstration w/explanation Prediction prior to demo with a conceptest Prediction prior to demonstration using discussion & a later conceptest 61 70* 77* 82* n = 158-297; * = statistically significant result vs. no demonstration Crouch, C.H., Fagen, A.P., Callan, J.P., & Mazur, E., 2004. American Journal of Physics, v.72 #6, p. 835-838.

15. The Value of Peer Instruction Control Group: Students took test individually. Experimental Group: Students took physics test individually, then again as a pair. Proportion of pairs of students who both got the question wrong on the first test but correct on “paired” test: 29% Students in both groups answered similar questions on a second exam two weeks later. Mean score on second exam for experimental group: 74% Mean score on second exam for control group: 64% Singh, C., 2005. American Journal of Physics, v.73 #3, in press.