Assessing multiple research-based transformations in second semester physics.

1. End Notes Assessing multiple research-based transformations in second semester physics. Steven J. Pollock University of Colorado at Boulder (per.colorado.edu) Data Collected Introduction Course Structure • We examine the impact of several proven reforms1,2 in a large course, including Peer Instruction3 and Tutorials4 • Goals: • Study impacts of multiple layered coordinated activities • Investigate necessary features of individual reforms and the dynamics among them. • Improve/examine student learning & expectations/beliefs. • Increased engagement with physics and teaching of physics • Guiding Research Questions: • Which elements most closely associate with learning gains? • What traditionally not-studied factors shape students’ conceptual development, attitudes, beliefs, and success? • What are the impacts of/on undergrad Learning Assistants? Tutorials 50 min/wk, 30 students, 1 grad TA + 1 undergrad Learning Assistant (LA) Weekly prep sessions + LA seminar • Shown here is all new data from Fall ‘04 “Phys 1120” (Physics II) • We have collected data for 2 semesters of reformed Phys I (reported elsewhere) • Pre and post conceptual exams: BEMA8 (Brief E&M Assessment) (Fa ‘04): N=331 matched of 462 students • Pre/post CLASS (Colorado Learning about Science Survey7) • Weekly online Tutorial pretest4 • Posttests given in class (collaborative, with clickers) • Posttests on course exams: 75% mult. choice, 25% long answer. • Attendance (lecture and Tutorial) • Collected but not shown here: • Weekly online attitude, and course surveys/long response • Homework: online and long answer LON-CAPA5 online personalized homework Phys 1120 500+ students 3 lectures/wk (No lab) Text6 HRW Interactive Lectures Peer Instruction with H-ITT personal response system Population Effects Examining Impacts of Reforms Evidence for Conceptual Learning INDIVIDUAL REFORM ELEMENTS EXAMINING LOW OUTCOMES (on BEMA post) LEARNING GAINS: Pre/post (BEMA) 8 . • * Participation is not significantly different:these are not "slackers" • **Weak at-risk markers. Can not easily/uniquely pre identify this population. • ***low gainers not largely D/F students (for whom we rarely get BEMA score) • Statistical uncertainty 1-2% for entries in "tail" column, typically half that for rest. • Bottom quartile => roughly BEMA (post)<50% • CLASS shows “percent favorable” in all categories combined. • “LA” = Undergrad Learning Assistant. “TA” = grad teaching assistant • Data from other institutions shown for comparison at bottom 8 • 44% average gain. Note: BEMA is hard - normalizing to physics grad TA pre-score would yield a normalized class gain of 60%.) • Tutorial attendance is close to 90%, lecture attendance is close to 80% LEARNING GAINS: Tutorials INFLUENCING FACTORS. LEARNING GAINS: GENDER Matrix of correlation coefficients • Correlation coefficients in red are not statistically significant (p>.05) • CLASS => % favorable on all scored categories on pretest • Normalized BEMA gain does not correlate with pretest (c.f. Hake2) • Course grade based heavily (60%) on exams (25% long answer, highly conceptual) covering far more material than BEMA. • No single course reform correlates strongly with BEMA gain • Data are for selected concepts from tutorials. (Results as %, rounded to nearest 5%) • "Post Click" are multiple choice questions answered in class via clickers • Clicker q's encourage peer discussion, but exams are individual. • Female pretest, and gain, on BEMA marginally lower. Final course grades are statistically identical: females outperform on hw and particip, compensating for slightly lower exam scores. Favorably compares to primary implementation of Tutorials * U. Wash. Tutorials on Circuits:9(bulb brightness, current model) => 50% post instruction without Tutorials, 75% with Tutorials. ATTITUDES AND BELIEFS IMPACT ON LEARNING ASSISTANTS. ROLE OF TUTORIALS: Relation to Course Goals Normalized BEMA learning gain for LA’s  45%. Average final score (75%) is statistically indistinguishable from Physics Grad students pre-score. Some quotes from first time “initially tentative” Learning Assistants Fa ‘04: LA #1: ...in the course of this semester I have advanced my conception of teaching far beyond where it was... My model of how classes should be run has also been radically changed... I hardly realized that there were so many options and methods for running a class... Now that I have a base knowledge of techniques and philosophies I believe that through self-evaluation and awareness my teaching can grow immensely. #2: I gain more confidence and ideas in my teaching styles... The Tutorials have helped me determine how to build the correct rapport with my students... I have enjoyed the experience in physics... More importantly, it has showed me teacher preparation is the foundation for student success in learning. #3: I was expecting to learn more about physics as an LA, but had no idea it would be so much... Personally, being an LA has been one of the most rewarding experiences of my college career.... I have found it to be more challenging and involving than I expected, which makes me like it much more... Where before I had considered teaching as a possibility for my future, I have now determined I definitely want it to be part of my life in some fashion or another • A significant fraction of exams cover material beyond Tutorial coverage. • High correlation of grade with Tutorial hw (graded by TA's) => tight coupling with other hw/exam questions/course goals • Bins are course grades (which correlate strongly with BEMA gain). • CLASS7 is a Likert-scale survey of attitudes and beliefs about learning. • Note correlation of final grade with CLASS beliefs at the start • More negative shift in beliefs for lower performing students. Conclusions References McDermott, L.C. and P.S. Shaffer (1992). Am. J. Phys. 60, 994 (1992), andpart II, p.1003 Elby, A. Am. J. Phys69(7), (2001) Redish, Saul, Steinberg, Am. J. Phys. 66 212-224 (1998) Pollock, S. “No Single Cause…”, PERC 2004. Finkelstein, N., Pollock, S., to appear in PRST (2005) • Learning gains (via multiple measures) are high => this is a successful transformed course. (Average posttest BEMA exceeds current CU phys majors at end of their Jr. level E&M.) • Attitudes/beliefs/expectations 10,11 show modest declines (towards “novice”), but less so for higher learning-gain students. (Student evaluations of course are high.) • No single reform element alone appears to account for the measured successes, but the combination of course elements has a large impact 12,13. • LA program is successful (pre/post content gains, LA satisfaction, and interest in teaching) In 3 semesters, 24 STEM majors have been LA’s. Of those, 8 were LA’s twice or more (continuing with more School of Ed courses) and (at least) 3 are headed to K-12 teaching. McDermott, L., Redish, E., American Journal of Physics, 67(9), 755-767, 1999, Redish, E., Teaching Physics with the Physics Suite, Wiley 2003 Hake, R., Am. J. Phys 66, 64-74, (‘98) Hake R., Cons Ecology5(2):28, (‘02) Mazur, E., Peer Instruction, Prentice Hall 1997 McDermott, Shaffer, Tutorials in Introductory Physics, Prentice Hall 02. CAPA homework system: see www.lon-capa.org Halliday, Resnick, Walker, Fundamentals of Physics Wiley 2004 http://cosmos.colorado.edu/phet/survey/fall2003.htmlAdams et al., "Design and Validation of the Colorado Learning About Science Survey", Perkins et al., "Correlating student attitudes with student learning using the CLASS", both in review for Proceedings of the PERC 2004, Sacramento, CA. BEMA: Private communication, R. Chabay and B. Sherwood, see also http://www.ncsu.edu/per/TestInfo.html Acknowledgements Special thanks to PhysTEC (APS/AIP/NSF), NSF STEM-TP, NSF CCLI (DUE #0410744), the CU Physics Dep’t, the entire Physics Education Research at Colorado group, and the Physics Education Group at the Univ. of Washington.