Some Best Practices for Effective Mentoring of Undergraduates in Science Research CUR/SPS 2006

1. Some Best Practices for Effective Mentoring of Undergraduates in Science ResearchCUR/SPS 2006 Anne-Barrie Hunter, Sandra Laursen, Elaine Seymour Ethnography & Evaluation Research CARTSS, University of Colorado at Boulder

2. Acknowledgments We wish to thank: • NSF-ROLE Program and Howard Hughes Medical Institute for their financial support • The four participating colleges who funded data collection in the early stages of this study. • The site PIs at the four research sites for their advocacy of the project, and their guidance as members of the study’s steering committee: J. Swartz (Grinnell College), S. Wettack (Harvey Mudd College), J. Gentile (Hope College), M. Allen and A. Wolfson (Wellesley College) • The faculty and students at the four colleges for their generosity and openness in giving interviews. We hope that they, and all who have enabled this work, will regard its findings as truthful, useful, and validating

3. Funding agencies and organizations promoting college science education (NSF, NRC, RC, HHMI, etc.) have strongly recommended that institutions of higher education provide greater opportunities for authentic, interdisciplinary and student-centered learning • Promotion of research-based learning as a national objective for science education in research universities (Boyer Commission Report, 1998)

4. What do we know about whatstudents gain from UR? Prior to our work: NOT MUCH: • 9 studies meeting accepted methodological standards Majority of literature largely claims-based by faculty practitioners offering: • descriptive accounts • promotional and discussion articles • histories and reviews These are important, but not evidence-based

5. Study Research Questions • What gains do students make from doing UR immediately following the experience, and in the longer term? • By what processes do these gains come about? How do peers, faculty and departments contribute? • How are career outcomes affected by UR participation? • What are the benefits and costs to faculty of doing UR?

6. Study Design 4 liberal arts colleges: with long history of well-established UR programs (based on apprenticeship model) = “best case scenarios” In-depth, semi-structured interviews with: • comparativestudent andfaculty participant/non-participant samples • college administrators and UR program directors • followed student samples longitudinally Total number of interviews = 367 This analysis: 76 student interviews + 80 faculty interviews =156 interviews

7. Content Analysis of Interview Data Thematic coding= Taking necklaces apart to sort beads and look for patterns in color, shape, size, and frequency (common or rare)

8. Data Analysis Yields: • Findings grounded in faculty and student observations • Frequencies for all observations across the data set (e.g., counts are of observations, not people) • Verbatim comments to illustrate the issues identified Findings help determine nature, range and relative weighting of the issues. Findings are not subject to tests of statistical significance

9. Overview: Comparison of Faculty and Student Positive Observations on Gains from UR

10. 1) 90% of faculty observations and 92% of student observations discuss gains 2) Students and faculty reportobservations on thesame range of gains 3) High degree of agreement between faculty and student observations 4) UR allows students to confirm, refine or clarify pre-existing ideas/intentions 5) Taken together, “Thinking and working like a scientist,”“Becoming a Scientist,” and “Personal-professional Gains,” comprise ~62% of all gains observations Overview of Research Findings on Student Gains from UR

11. Learning theory, communities of practice, and the development of personal and professional identity

12. Social constructivism A model of learning in which knowledge acquisition is a process of integrating new knowledge with prior knowledge: knowledge is continually constructed and reconstructed by the individual. (Vygotsky, 1978)

13. Social constructivism Knowledge acquisition is a “negotiated, social, and contextual process”  Role of the teacher as a “facilitator” of student learning • “meaning making” is a shared, two-way process between the mentor/master and the student/apprentice based upon an activity of mutual interest

14. Social constructivism “Meaning making” is “situated” and takes into account a student’s current level of knowledge Learning objectives aimed at capturing a “zone of proximal development”– a student’s ability to learn and problem-solve beyond his or her current knowledge level through careful guidance from and collaboration with an adult or group of more able peers (Vygotsky, 1978)

15. Social constructivism Lave and Wenger (1991) and Wenger (1998): “communities of practice” “newcomers” are socialized into the practice of the community (i.e., science research) through mutual engagement with, and direction and support from, an “old-timer”

16. Social constructivism Lave and Wenger: professional socialization as an outcome of engaging in a “community of practice” “Legitimate peripheral participation” student’s active participationin the authentic practice of the community • the process by which the novice is inducted into the knowledge and skills of the community’s professional practice under the guidance of the master • the process by which the novice moves from the periphery toward membership in the community

17. Social constructivism “Cognitive apprenticeship” (Brown, Collins & Duguid, 1989; Farmer, Buckmaster & LeGrand, 1992): helps students learn to contend with ambiguity and uncertainty  transmits knowledge about how to deal with situations that are “ill-defined, complex and risky”  prepares the student to negotiate “undefined spaces of learning” where answers are unknown to everyone (novice and master)  opportunities for the student’s self-expression and reflective thinking facilitated by an expert

18. Social constructivism “Epistemological reflection model” Baxter Magolda (1999,2004): links students’ intellectual development to identity development as part of professional socialization process: “ways of knowing” gradually shift from an externally-directed view to one that is internally-directed  process of “self-authorship”

19. Social constructivism “Self-authorship”: • situated learning that takes into account the student’s own perspective • negotiated, shared meaningmaking between the teacher and the student  validates students as capable learners; encourages their epistemological, intellectual, personal, and professional development

20.  Constructivist learning theory and pedagogies, including communities of practice, are manifested in the structure and practice of the apprenticeship-style model of undergraduate research

21. How do these theories on learning, professional socialization and identity development relate to our findings on student gains from participation in UR experiences in the sciences?

22. Interconnectedness of: • cognitive development • personal growth • identity development “Thinking and working like a scientist” “Becoming a scientist”“Personal-professional gains” Intellectual gains  personal and professional growth

23. “Thinking and working like a scientist”Hands-on learning of “how science research is done”  students integrate current and new knowledge  directly apply critical thinking and problem-solving skills to the professional practice of science research  growth in student’s confidence to do science Role of the research advisor as a “facilitator” of student learning: providing guidance and support, helping the student to reflect on their own learning  growth in student’s confidence

24. “Personal-professional gains” Novelty of interacting with faculty as “colleagues” is very powerful for students “Being taken seriously” and respected for their opinions and insights  validates students as capable learners and builds students’ confidence and draws students into the community of science: “I can do science” “I can contribute to science” Student research peers are an extra “resource” and provide camaraderie  contribute to sense of belonging to a community

25. “Becoming a scientist” Students’ reported changes in attitudes toward learning and working as a researcher, but did not project these changes beyond the context of UR • Taking greater care to be accurate, going beyond the minimum expected, greater willingness to work independently, take initiative Faculty research advisors “witness” students’ growth • Increased willingness to work independently, make decisions about “next steps” in the project, more willing to take risks, think creatively  Adoption of attitudes and work norms that indicate professional socialization and integration into the community of practice (though students are not conscious of their professional development)

26. Career clarification • Increased interest and enthusiasm for field of study, or in science, generally, but students largely unclear about future plans at time of first interview • Faculty and students value UR experience as an opportunity to cognitively and affectively assess the appropriateness of a career in science research • Clarified, confirmed and refined previous career intentions, including going on to graduate school • 7 of 76 students discovered “research is not for me”  Personal and professional identity development

27. Enhanced career preparation Students value: UR as “real world work experience” with transferable value; opportunities to network with other scientists, faculty, peers Faculty value UR for enhancing students’ preparation for future work contexts and graduate school • Personal and professional identity development as part of professional socialization  Role of other community of practice members who actively contribute to students’ learning and professional socialization

28. Skills Faculty and students report student gains in technical skills and learning to present: students emphasize transferable value of skill gains • Growth in students’ confidence, understanding of professional practice • Basis of professional socialization and integration into community of practice

29. Some best practices of mentoring students in UR: Research advisor as facilitator of learning “Student-centered” and “situated” UR project: well-defined, but open-ended Meets students at their “level,” but “stretches” intellectual capacity Provides legitimate participation Meaning making is shared and negotiated  “Thinking and working like a scientist” “Becoming a scientist”“Personal-professional gains”

30. Conclusions • Every category of student gains reflects elements of personal growth and professional socialization into the practice of science research. • Research findings show how theory on learning, communities of practice, and personal and professional identity development are manifested in practice.  The importance of UR to students’ education and its contribution to their cognitive, personal and professional development

31. Research Findings:The Benefits to Students ofUR Experiences in the Sciences

32. “Thinking and Working Like a Scientist” Applying critical thinking and problem-solving skills to research Faculty observation: “I tend to go around saying, “Okay. What have you done? What is your analysis?” I can tell that they’re catching on when, as I start discussing possible interpretations with them and I’ll say something and they’ll say, “Oh, but that doesn’t fit with what we did yesterday.” Then you know the science is there.” Student observation: “It really does help you learn to detect your own dumb mistakes. Like, it’s easy to think about something conceptually a little bit wrong, and go with that for about a week. But then you look at what you’ve got, and your spectra don’t make any sense…. Then you realize what the problem is. You learn to recognize things like that quicker and quicker the more you do it.”

33. Summary:“Thinking and Working Like a Scientist” Faculty emphasize more than students, student gains in: • understanding science research • understanding the nature of science Students emphasize more than faculty gains in: • Increases in their knowledge • Appreciating the relevancy of their coursework Student claims in gaining increased understanding of how to frame research questions/develop research design are NOT matched by faculty observations of student gains in this area

34. “Becoming a scientist” Demonstrated gains in attitudes and attributes Faculty observation: Taking “Ownership” of the project/intellectual engagement: “What I look for, and to me, the mark of success in this kind of endeavor is ownership….It’s fun to see….There’s some transformation that occurs, where it suddenly becomes their project. And you see that.” Student observation: Greater intellectual engagement/thinking and working independently: “Just being able to sit down and concentrate on one thing and figure it out and understand…. We work with protein-DNA interactions. And so just for me to look at that and really, really understand it rather than just getting the big overview. And then, actually thinking about the problem critically and creatively and being, ‘Okay. Now what can I change to have this effect and to have this outcome?’ That’s a whole new experience for me.”

35. “Becoming a scientist” Understanding the nature of science/that research requires particular temperament Faculty observation: “I think they learn that science is really boring (laughs). And that’s the key. If they can know that science is boring and still do it, and still stick with it, then they have the makings of a really good scientist.” Student observation: “You can’t get too emotionally distraught over something…I have a tendency, especially when it’s my own dumb mistake that’s caused me to lose several hours of work…. You have to just step back and deal with the facts as they are and say, “Okay. I’ve messed up. I need to correct this. It’ll take a few hours, but then we’ll move on.”

36. Summary: “Becoming a scientist” • Faculty value the development in students of attributes and attitudes important to professional practice, as these are essential if students are going to “become scientists” • Students acknowledge changes in themselves, but do not recognize these as important professional attributes. Rather, students internalize these gains, focusing on the immediate effects of their own self-development.

37. Personal-professional gains Gains in confidence to do science Faculty observation: “You can see it a mile away. When they approach a new piece of equipment, it’s more, “Well, where’s the manual?” (Laughs) “Don’t waste my time teaching me this. Just tell me how to turn it on and I’ll figure it out.” Self-confidence, maturity.” Student observation: “At the beginning, I asked a lot of questions to get a good basis and a good idea when I didn’t really know what I was doing. But by the end of the summer, I didn’t speak to my advisor much, because I would just do it.”

38. Personal-professional gains Establishing a collegial relationship with faculty Faculty observation: “There’s a lawyer in Cedar Rapids that I’ve kept in touch with over the years. He was ’76 class, something like that. And about every other year we get together someplace. We have a lot of mutual friends and we know what each other’s doing. There’s another guy, a faculty member, a mathematician…we see him all the time. He used to baby-sit for us. Their daughter was up a couple of weeks ago.” Student observation: “He said he’s learning as much from us as we are learning from him. …He’ll start taking out his notebook and start writing down things we are saying.… It makes you feel proud when something you said is important enough for someone like him, who has been researching this stuff much longer than we have—especially when it seems like a moment of insight, like something he hadn’t thought of before…. It just feels great when somebody takes me seriously, or takes my work seriously.”

39. Summary:Personal-professional gains Students emphasize more than faculty gains in confidence: • to do research • to contribute to science • in “feeling like a scientist” Students experience the force of these affective gains for themselves Faculty emphasize more than students gains in establishing a collegial relationship Faculty have longer-term evidence of the importance of these relationships (for themselves and for their students)

40. Clarification/Confirmation of Career/Graduate School Intentions Faculty observation: “It’s certainly nice to see them learn over the course of the summer, to see them doing more thinking for themselves, more autonomy, making good choices, making good decisions. It’s nice to see them gain confidence in their role as research collaborators. It’s nice to see them get to a point where they clarify what they do and don’t want to do, because that really does often happen. …It’s nice to see them clarify, ‘Yeah, that was interesting, but it’s not my cup of tea,’ or, ‘Oh, I loved it and this is what I want to do!’” Student observation: “I’ve always been thinking and wanting to go to grad school, ever since I can remember, wanting to get a doctorate, but I actually truly decided, it was this summer when I said, ‘Yes, I’m going to go to grad school. It’s what I want to do.’”

41. Summary:Clarification/Confirmation of Career/Graduate School Intentions Students emphasize gains in: • Assessing “fit” between interests and field of study “research is for me” • Clarifying, refining and confirming previous career/graduate school intentions Faculty emphasize student gains in: • Increased interest

42. Enhanced Career/Graduate School Preparation Provides Relevant Career/Graduate School Preparation Faculty observation: “Some of my M.D.-Ph.D. alums, they come back and say, “Look. I got my Ph.D. because I love being in the lab. I don’t know that I’ll ever become a faculty member. I’m practicing now. But the way of functioning that I got from being in the lab informs everything that I do in general practice.” Student observation: “You’re given a lot of freedom and responsibility to do things so I’m really getting out of it how to go about a professional type job or business, these kinds of things.”

43. Summary:Enhanced Career/Graduate School Preparation The larger number of student comments for all types of observations highlights students position as graduating seniors and their preoccupation with “what comes next” Faculty comments highlight the importance of UR to their own careers: “bean counting” numbers of students, articles, presentations, etc.

44. Summary: Skills Faculty and student observations are well aligned. Both agree greatest gains are in: • Presentation skills • Lab skills Higher ranking of skills for students (3rd) than for faculty (6th) indicate their greater importance to students: • Reflect the steep learning curve of UR: learning new instrumentation at the beginning; meeting the challenges of learning to present at the end • Transferable to other areas in life and as important to future careers or graduate school

45. Conclusions Different observations on the same types of gains reveals how each group sees and values the gains of undergraduate research differently Factors evident in students’ observations: • had just finished their 10-week UR experience: very intensive • were about to start senior year, were uncertain/concerned about future plans • as yet, do not recognize in themselves the development of professional traits that faculty see. Factors evident in faculty members’ observations: • an encompassing view of student gains derived from long experience leading UR • bring own perspective as educators, mentors and professional scientists (particularly noting gains in students seen as necessary if students are going to go on to replace the science profession)