Gaining A Geological Perspective Through Active Learning in the Large Lecture Classroom

1. Gaining A Geological Perspective Through Active Learning in the Large Lecture Classroom Jessica Kapp, Randy Richardson, Allister Rees, Jon Patchett, Ross Waldrip University of Arizona Department of Geosciences

2. Traditional Geological Perspective • Tier 1 Gen Ed class for non-science majors • ~1200 students per year (~600 per semester) • ~74% freshman • Four lectures per semester taught by 3 faculty • Lectures are ~150 students each • Optional study groups • Traditionally 8 GTAs and ~10 undergraduate preceptors run study groups • Traditional lecture-style presentation with activities peppered throughout (varies based on faculty member) • Instructional technology varies – animations, movies, demonstrations

3. Goals of Geology Redesign 1. Develop consistency between lectures so that all use an active learning approach 2. Reduce instructor face time and increase time spent peer learning (with other students or preceptors) 3. Reduce faculty time devoted to preparation and implementation of the course 4. Reduce cost per student without compromising learner outcomes and/or student perception of the class 5. Maintain or increase level of rigor of the course while increasing student enthusiasm/interest

4. No doubt we cut costs

5. No doubt we cut costs

6. No doubt we cut costs

7. Key features of redesigned course • Pre-class on-line reading quizzes • In-class activities that promote student interaction and higher-level learning (some graded, some not) • Mandatory break out sessions once a week led by fewer GTAs and more preceptors • Text – The Good Earth (McConnell et al.)

8. Key features of redesigned course • All lectures prepared ahead by J. Kapp, all instructors use same lectures • All lectures include built in activities that are easy to grade • No clickers – several studies are showing they are not as effective at enhancing learning as they are at making it easy to grade stuff

9. Geosciences Concept Inventory 1.0 (Libarkin et al.) • 78 items, administered in three groups • Only the 36 items most related to course were analyzed • Quantitative measure of student learning

10. Examples of GCI questions: • If you could travel back in time to when the Earth first formed as a planet, what would the Earth look like? • A. The Earth would be mostly covered with water • B. The Earth would be mostly molten • C. The Earth would be mostly covered with ice • D. The Earth would be mostly rocky • If you could travel millions of years into the future, how big would the planet Earth be? • A. Smaller than today • B. Larger than today • C. Same size as today • D. We have no way of knowing

11. Pilot semester learning outcomesGeosciences Concept Inventory • GCI Traditional course: • pre-test % correct = 28.67 (SD=11.45, n=96) • post-test % correct = 45.26 (SD=11.76, n=84) • GCI redesigned course: • pre-test % correct = 38.62 (SD=8.42, n=144) • post-test % correct = 48.73 (SD=7.49, n=132) • full implementation post-test % correct = 47.11 (SD = 9.01, n + 117) • Gains from pre-test to post-test are statistically significant, but the difference in post-test GCI scores between the two groups is not. • Interpretation: Students’ knowledge levels, insofar as the GCI can measure, were equivalent in both courses.

12. Other impacts of redesign • Analysis of common essay-style exam questions • Students’ written responses were analyzed in terms of three categories: (1) accuracy and completeness of the response; (2) number of words; (3) numerical points assigned by graders • In all categories, the student-supplied responses from the two courses were indistinguishable

13. Common on-line quiz results • Quiz on Earthquakes • Traditional course: Average = 75% (SD = 17) • Redesigned course: Average = 85% (No SD) This seems significant BUT… • We changed the wording of a few questions • The lecture material changed a bit • These changes were made in response to students’ poor performance on certain questions

14. Class Success • Student success rates (C or higher in the course) were 90% in the traditional course, 87.2% in the pilot semester, and 89.6% in the full redesign. The differences are statistically insignificant. • Withdraw/drop rates were 3% in the traditional course, 0.5% in the pilot semester, and 5.6% in the full redesign. • This last value is skewed, as in our full implementation semester our classes were overenrolled without our consent or knowledge before the semester began, causing what appeared to be an excessive drop rate.

15. Other impacts of redesign • Attitudes Toward Science – no statistical differences • Ratings on course evaluations unchanged • Lecture attendance virtually unchanged • Students positive about usefulness of • In class exercises • Break out sessions • Interview and focus group transcripts reveal that students find the reformed course to be relevant to their lives and educationally satisfying, both of which are only infrequently observed among traditional introductory science survey courses

16. Issues and Findings • Aside from J. Kapp, instructors vary. • Instructors approach material differently. • On-line course management system used in traditional class had to be abandoned due to glitches, lack of tools. Hard to compare results. • Grading still time consuming but students prefer the face time, and in-class assignments get them to class. • Exams – still prefer some form of short answer (grading).

17. Issues and Findings • PRECEPTORS! • Great resource, students like them • Problem 1: High turn over rate. Very few stay for several semesters • Problem 2: Lack of expertise in Geoscience • Problem 3: Lack of confidence leading a break out session • Rare (but real) problem: Some issues with boundaries, authority, etc. We attempted to address #3 with a training workshop for our preceptors. They enjoyed this very much and said they learned a lot about teaching techniques. However, it doesn’t change the fact that they are nervous about answering questions they don’t know the answer to (lack of expertise).

18. And most interesting? • Interviews with students, TAs, preceptors, and faculty. • More than 100 people were involved and 14 hours of transcripts were produced, in addition to the evaluators’ field notes. • All transcripts were inductively analyzed and reanalyzed until repeated themes emerged from the data across all transcripts. • Four recurring themes were prominent across the extensive qualitative data collected (listed by perceived importance to the success and value of the supporting learning in the class) 

19. 1. Structured discussion among students was meaningful • Students and instructors alike described the course as filled with repeated opportunities to absorb, process, and apply lecture information in guided and supportive settings. • Weekly discussion group sections provided time-on-task to “absorb” the ideas. • Discussion group activities focused on real life applications. • Most students understood discussion sessions were not intended to provide new information, but rather to emphasized the important aspects of lecture. • Several students agreed with the comment that, “I’m only doing well in this course because of the discussions.” • GTAs often felt that the cognitive level required of students in the discussion sections, and perhaps the course overall, was too low. • Undergraduate peer mentor instructors felt that the graduate teaching assistants, who were new experts in the field, didn’t have a reasonable opinion of what non-science majoring students needed to understand.

20. 2. The interactive nature of the course engaged students and instructors • Students understood that the interactive, participatory nature of the course was substantively and crucially different than their other courses they were taking. • Students understood how the think-pair-share questions required students to actively process information rather than engaging in passive, and occasionally meaningless, “random note-taking” so often required in other classes. • “If you don’t pay attention you won’t do well, but you should because those are easy points to help your grade.” • Students also felt that the collaborative nature of the class, both in lectures and during the required discussion sessions, helped them build a learning community and that this sense of community led to increased interaction between students in lecture. • “You can’t do this kind of thing with 100 people who don’t know each other.”

21. 3. The non-adversarial nature of the course climate was critical • Applauded the course instructors on the emphasis and repeated importance of clear expectations. • “There are no secrets in this class. They tell us over and over. If you don’t know what it takes to be successful, you aren’t paying attention.” • Students overwhelmingly wanted to comment enthusiastically on their perceived instructors’ attitudes toward the students. The felt that instructors cared about their learning, as evidenced in a number of ways • instructors created a low-risk environment • professors demonstrated respect by learning everyone’s names • professors emailed students and greeted them by name on campus

22. 4. The purposeful instructional activities directly support metacognition • Great value in seeing a variety of teaching modalities used in the class. • Students recognized that professors had a reduced focus on vocabulary. • Having shorter lecture periods punctuated with purposeful interactivity served to “reset” attention spans.

23. Full Implementation • We continue to use consistent, pre-made lectures with built in activities • Continuing effort to enhance lectures with more multimedia material (you tube clips, animations, video) • Exam questions focus more on higher level learning (writing better multiple choice questions) • We continue to use an abundance of preceptors (29 this semester!) • We want to use more GTAs, as the preceptor/TA issue is paramount

24. Contact me Jessica Kapp Department of Geosciences University of Arizona jkapp@email.arizona.edu (520) 626-5701