Leveraging Peer Observation for Institutional Change: Spreading Evidence-Based Instructional Practices

1. Spreading Evidence-Based Instructional Practices: Leveraging Peer Observation for Institutional Change University of Portland REFLECT Team: Stephanie Salomone, Math Heather Dillon, Engineering Valerie Peterson, Math Tara Prestholdt, Biology Carolyn James, Math Eric Anctil, Education ASCN Webinar: May 30, 2019 NSF IUSE:EHR 1710735

2. Axiom 1: Active learning is critical* • Significant improvement in grades and pass rates (Freeman, 2014) • Conceptual learning gains (e.g., Kogan & Laursen, 2013; Kwon, Rasmussen, & Allen, 2005; Larsen, Johnson, & Bartlo, 2013) • Reduced disparity between dominant and historically marginalized groups (Kogan & Laursen, 2013; Riordan & Noyce, 2001) • Improved retention (Rasmussen & Ellis, 2013; Seymour & Hewitt 1997)

3. Axiom 2: Changing practices is hard • Lecture still dominates (Johnson et al., 2017; Hora, Ferrare, & Oleson, 2012) • Providing evidence is not enough to motivate change (Foertsch, Millar, Squire, & Gunter, 1997; Reese, 2014; Dancy & Henderson, 2010) • Disseminating curricular materials is insufficient (Henderson, et al., 2011)

4. Participant Poll • Consider a change you would like to enact at your institution. What feels like the biggest barrier to this change? • Limited time/resources for participants • Limited motivation/incentive to change • In conflict with reward structure/institutional culture • General fear or mistrust of change • Other (please share in chat window)

5. Institutional Change Theory Individuals Prescribed Emergent Environments & Structures (Henderson, Beach, & Finkelstein, 2011)

6. Institutional Change Theory Individuals Disseminating curriculum and pedagogy Developing reflective teachers Prescribed Emergent Enacting policy Creating a shared vision Environments & Structures (Henderson, Beach, & Finkelstein, 2011)

7. Participant Poll Individuals • Consider a specific change effort you’ve observed (or are planning). Which quadrant best describes the intended change? B. A. Prescribed Emergent C. D. Environments & Structures E. Two quadrants feature heavily F. Three or four quadrants feature heavily

8. Institutional Change Theory Effective strategies share common features: Align with (or seek to change) beliefs of participants • Involve ongoing interventions • Fit with the culture and structure • of the organization (Henderson, Beach, & Finkelstein, 2011)

9. Institutional Change Theory • CACAOModel of Change (Marker, et al. 2015) Change, Adopters, Change Agents, Organization Allows for balance across “4 quadrants” of institutional change • The “spectrum of adoption” (marketing)

10. The REFLECT Project REFLECT: Redesigning Education For Learning through Evidence and Collaborative Teaching A 3-year, NSF-funded IUSE:EHR project at a private, comprehensive university in the Northwest Context: • ~3500 undergraduates, including School of Engineering • Excellence in teaching is highly valued, traditional models prevail

11. REFLECT Project Goals • Increase awareness of and interest in evidence-based instructional practices (EBIPs) on campus • Create a culture of reflective teaching via faculty peer observation and mentoring • Develop and test assessment tools for reflective teaching • CACAO Guiding Principles: • Early adoptersbuild awareness of EBIPs • Enlist campus leaders (change agents) • Leverage context to shift culture of teaching on campus (organization) • Compensation shows value of people’s work

12. REFLECT Project Components Individuals Summer Institute Exposure to Implementation Peer Observation Training Training and refining Prescribed Emergent Peer Observations Practice and Support Lunches Discuss and Support Environments & Structures (Henderson, Beach, & Finkelstein, 2011)

13. Institute Overview • 3-4 days in May (after semester ends) • Topics include: • Introducing EBIPs • Classroom norms • Student-centered pedagogies • Troubleshooting implementation and pushback • Designing learning objectives and Bloom’s Taxonomy • Data collection and assessment • Technology tools • Work time • Other Institute features: Presidential welcome, guest speakers, food and community, and compensation

14. (Formative) Peer Observation “But as I look at you with my lens, I consider you a mirror. I hope to see myself in you. . . Seeing you allows me to see myself differently and to explore the variables we both use.” (Faneslow, 1990, p 184)

15. Peer Observation Overview • Why peer observation? • to help create more reflective teaching and build faculty networks across campus • to facilitate faculty change by focusing on change in progress: using formative(not summative) tools • to encourage cross-disciplinary collaboration and ultimately establish peer observation as part of the culture of teaching and learning

16. (Formative) Peer Observation • Promotes individual change • Stimulates reflection (Bell, 2001; Cordingley et al., 2005; Cosh, 1999; Reinholz, 2015) • Improves collegial relationships (Carroll & O’Loughlin, 2014; Shortland, 2010; Reinholz, 2017A) • Provides on-going support (Byrne, Brown, & Challen, 2010; Martin & Double, 1998) • Can observation promote institutional change?

17. Peer Observation & Change Peer observation can promote change in beliefs • Peer observation fosters a • long-term, interdisciplinary community of practice Peer observation can address institutional concerns and contexts

18. Participant Poll • There are arguments for and against using faculty peer observation as a measure of instructional quality. Does your institution employ any type of peer observation? • No, student evaluations are the only formal measure of teaching • Yes, but only informally/used by volunteers/not used in promotion and tenure decisions • Yes, peer observation is mandated (to some extent) • Other (please share)

19. Participant Poll • For those who do engage in faculty peer observation (or have at some point), is there any sort of common structure or formal training involved? • No, we’re on our own. • Yes, there’s some structure (common rubric, e.g.) • Yes, there is training. • Yes, there’s structure and training • Other (please share)

20. Peer Observation Overview • 1-day training (end of summer) • Topics include: • Check in and catch up • Fear walk • Intro to protocols, norms • Video examples/practice with protocols • Small cohorts for observation (ongoing)

21. Peer Observation Overview • Bread: Pre-observation conversation and self-assessment • Filling: Select one dimension of instruction • Clarity of instruction • Use of technology • Responding to student thinking • Challenging content • Equity of student engagement • (and others) • Bread: Post-observation conversation

22. Bread: Pre-observation Conversation and Self-Assessment

23. Filling: Challenging Content Students have extended opportunities to engage with topics Concept is routine Concept is extremely challenging Students have limited opportunities to engage with topics

24. Bread: Post-observation

25. Questions for the Audience • How might peer observation be leveraged in your context? • Let’s brainstorm barriers (and mitigation strategies) to effective peer observation… • These observations are formative, not evaluative. What are affordances and constraints in each case? • In your institutional context, what can you imagine in place of or in addition to peer observation?

26. Questions for us? The REFLECT Project www.reflectup.org Contact: Stephanie Salomone salomone@up.edu Thank you!

27. Selected References Active learning and teacher change • Dancy, M., & Henderson, C. (2010). Pedagogical practices and instructional change of physics faculty. American Journal of Physics, 78(10), 1056-1063. • Foertsch, J., Millar, S. B., Squire, L., & Gunter, R. (1997). Persuading professors: A study of the dissemination of educational reform in research institutions. Madison, WI: The LEAD Center. • Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 201319030. http://doi.org/10.1073/pnas.1319030111 • Hora, M. T., Ferrare, J., & Oleson, A. (2012). Findings from classroom observations of 58 math and science faculty. Madison, WI: University of Wisconsin-Madison, Wisconsin Center for Education Research. • Johnson, E., Keller, R., and Fukawa-Connelly, T. (2017). Results from a Survey of Abstract Algebra Instructors across the United States: Understanding the Choice to (Not) Lecture, International Journal of Research in Undergraduate Mathematics Education. • Kogan, M., & Laursen, S. L. (2013). Assessing long-term effects of inquiry-based learning: A case study from college mathematics. Innovative higher education, 1-17. • Kwon, O. N., Rasmussen, C., & Allen, K. (2005). Students' retention of mathematical knowledge and skills in differential equations. School Science and Mathematics, 105(5), 227-239. • Larsen, S., Johnson, E., & Bartlo, J. (2013). Designing and scaling up an innovation in abstract algebra. The Journal of Mathematical Behavior. • Rasmussen, C., & Ellis, J. (2013). Who is switching out of calculus and why. In Proceedings of the 37th Conference of the International Group for the Psychology of Mathematics Education (Vol. 4, pp. 73-80). Kiel, Germany: PME. Education. • Reese, M. (2014). Changing course: The influence of social position on college faculty’s adoption of educational innovations.Ph.D. Dissertation, Johns Hopkins University. • Riordan, J. E., & Noyce, P. E. (2001). The impact of two standards-based mathematics curricula on student achievement in Massachusetts. Journal for Research in Mathematics Education, 368-398. • Seymour, E., & Hewitt, N. M. (1997). Talking about leaving: Why undergraduates leave the sciences (pp. 115-116). Boulder, CO: Westview Press.

28. Selected References Institutional change • Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48(8), 952–984. http://doi.org/10.1002/tea.20439 • Marker, A., Pyke, P., Ritter, S., Viskupic, K., and Moll, A. (2015). Applying the CACAO Change Model to Promote Systemic Transformation in STEM, in G. Weaver, W. Burgess, A. Childress, & L. Slakey (Eds.), Transforming Institutions: Undergraduate STEM Education for the 21st Century (pp. 176-188). West Lafayette, IN: Purdue University Press. Peer observation • Bell, A., & Mladenovic, R. (2008). The benefits of peer observation of teaching for tutor development. Higher Education, 55(6), 735-752. • Byrne, J., Brown, H., & Challen, D. (2010). Peer development as an alternative to peer observation: A tool to enhance professional development. International Journal for Academic Development, 15(3), 215-228. • Carroll, C., & O’Loughlin, D. (2014). Peer observation of teaching: enhancing academic engagement for new participants. Innovations in Education and Teaching International, 51(4), 446-456. • Cordingley, P., Bell, M., Thomason, S., & Firth, A. (2005). The impact of collaborative continuing professional development (CPD) on classroom teaching and learning. Review: How Do Collaborative and Sustained CPD and Sustained but Not Collaborative CPD Affect Teaching and Learning. • Cosh, J. (1999). Peer observation: a reflective model. ELT Journal, 53(1), 22–27. • Martin, G. A., & Double, J. M. (1998). Developing Higher Education Teaching Skills Through Peer Observation and Collaborative Reflection. Innovations in Education & Training International, 35(2), 161–170. • Reinholz, D. L. (2015). The assessment cycle: A model for learning through peer assessment. Assessment & Evaluation in Higher Education, 1–15. • Reinholz, D. (2017A). Graduate Student Instructors learning from peer observations. In Proceedings of the 20thAnnual Conference on Research in Undergraduate Mathematics Education(pp. 846-854).