Dr. Angel Hoekstra University of Colorado

1. Effective Pedagogical Use ofStudent Response SystemsMidwestern UniversityCollege of Health Sciences Faculty Retreat – August 23, 2010 Dr. Angel Hoekstra University of Colorado

2. Outline for today’s workshop • Effective Pedagogical Use of Student Response Systems • Question and Answer Session • Break • Small group work: Writing Effective Clicker Questions • Using Student Response Systems to Collect Data for Scholarship of Teaching and Learning Research • Lunch • Small group work: Writing SoTL Clicker Questions • Wrap-up Discussion & Conclusion

3. Socrates proposed that teaching be doneby questioning, not telling… Introduction to Clicker Use • SRS use is not designed to replace lecture, but to act as a supplement • Enhancing student learning through peer teaching, interaction • Peer talk makes conceptual application more engaging • Giving students the ability to ask for help without raising a hand

4. Presentation Outline • Introduction • Pedagogical guidelines for clicker use • Overview of research program • Discussion of data: Effects of clicker use under an active learning pedagogical approach • More guidelines for clicker use • Conclusion

5. Additional Information My background Theoretical & methodological orientation Teaching experience, courses using clickers Recent PhD graduate, currently on the job market Effective pedagogical use of clickers Use should match the instructor’s learning goals Use varies depending on nature of the course material, size of course, composition of the learning community

6. On first glance, a simple process… • Lecture: 5 – 15 minutes • Posting a multiple choice clicker question • Instructor pedagogical decision: Encourage peer discussion? If so, before or after they click in? • Students click in their answers • SRS software gathers, tabulates, displays responses • Class viewing of clicker responses • Instructor explanation • Move on to next part of lecture

7. … is actually not quite so simple. • Clickers are a tool, not a pedagogical strategy • Clickers can be used in a variety of ways • How one “frames” clicker use (instructor talk) strongly influences student behavior in class • E.g. Lifting the “ban on talking,” versus unintentionally reinforcing it, in the Principles of Journalism course • Literature theme: Importance of instructor meta-narrative about why clickers are being used

8. General Purposes for Clicker Use • In-class conceptual application • Fostering critical thinking about course concepts • Encouraging peer talk and social solidarity • Helping students identify common mistakes when applying, calculating or solving • Driving large group discussion • Engaging students more during class

9. Simulating the Effects of Clicker Use • Take a moment to write in answer to the following: What aspect of student response system use most interests you? Put another way, what are you most hoping clicker use may accomplish in your course/s?  Please turn to a neighbor, introduce yourselves, and then discuss your answers.

10. Approaches toGrading Clicker Responses • No credit • Great for first time SRS users • Less motivation for students to click in/come to class • Assigning clicker points “extra credit status” • Motivates students to attend, without requiring them to • Less motivation for students to click in on individual questions • Incorporating clicker points into the course grade • Strongest motivator for student participation in class • Recommended weight: 5-10% of overall course grade

11. Consider the social status of most undergraduate students: Seeking affirmation as adults Recently “on their own,” learning to pay bills, etc. Especially sensitive to instructor behaviors that seem patronizing or seem to reduce the instructor’s workload Problematic instructor behaviors: Using clickers too infrequently Asking clicker questions that are too simplistic, too often Not fully explaining the answer and/or each of the potential answer choices for a question Beware of allowing the SRS to become an attendance monitoring device!

12. Next… A look at the social effects of clicker use for student behavior Considering what motivates students to want to participate in the learning process…

13. The Research Program Mixed Methods Approach, Ethnographic design (2004-2009) • Participant Observation • 6 courses observed in four disciplines, 87 classes observed • Survey Design (N = 3045) • Students surveyed in all 6 courses, plus my own course • Anonymous Free Writes (N = 675) • One-page free writes address types of clicker questions used, potential problems/concerns with clicker use under different pedagogical formats • Semi-structured Interviews (N = 38) • 20 interviews with General Chemistry students (spring 2005) • 8 interviews with General Chemistry students (fall 2005) • 10 interviews with Sociology students (fall 2007 & spring 2008)

14. Disciplines Studied Chemistry Astronomy Journalism Sociology

15. General Chemistry & Stars and Planets Clicker questions interspersed during lecture Peer discussion verbally encouraged in small groups Data generated by SRS technology structurally prompts students to increase their level of attention to their peers Social Statuses chosen by students Regular clicker group members Floaters Loners Self-Testers Effects of Single-vote Pedagogy & Active Learning Approaches

16. Regular Clicker Group Participation • Position preferred by the majority of students: to establish a regular clicker group early in the course and maintain it over time • Vanessa, a diligent reader and regular attendee in General Chemistry, explains her experience in a regular clicker group, I sit by a guy and a girl, and we always work together on clicker questions. I met them in the class… we just sat by each other and now we sit by each other every day and work together. I met them through the clickers, really… a question came up and we just started talking about it, and I definitely am the kind of person where, I will keep to myself before I talk to somebody else, so I was glad that the girl talked to me first… With clickers, I mean, they present the material and then they ask the question, so if you didn’t get it right away, then you are kind of lost. So it helps [to work in a regular clicker group], because then someone explains it to you and you get to think about it again, or in a different way, as you are discussing the question.

17. Benefits of Regular Clicker Group Participation Teams engage in shared activity around common goals • Helping one another understand • Holding one another accountable Survey data: General Chemistry (N= 814) • 75-80% of General Chemistry students said clicker-prompted discussions were at least “somewhat helpful” for learning • 45% said peer talk during clicker questions was “quite” or “extremely” beneficial

18. Student Experiences in Regular Clicker Groups • A male student in Stars and Planets, At first I hated the clicker, I thought it was just a stupid piece of equipment we were being forced to spend our money on, [but] right from the first day, Dr. D said how clickers would be used, and [he] has stuck to his policy. He asks us clicker questions about the reading, or to apply concepts, or [to] express our opinions. I would never really [have] talked to the people next to me if not prompted to through clickers. In other classes I’ve had in this room, I’ve felt completely separate from the class and the material, but thanks to clickers, I feel involved. • Liz, explains her experience in a regular clicker group in General Chemistry, I like the people I am working with a lot… [using clickers] gives us some time to chat in the class and see how each other are. And, there was a week… I had a really hard week, just personally I wasn’t doing so well, and I wasn’t caught up in the class and they noticed it, and they were like, “Hey, is everything going okay? You seem like you are kind of behind.” I was like, “Yeah, I am behind… okay, so by this Friday, I will have read up to this chapter… so you guys ask me on Friday.” And they did [ask her]… so it’s definitely been beneficial, just in accountability.

19. Floaters A term derived from the interview participants: “those willing to work with others but not attached to a regular clicker group” • A small but significant percentage (15-20%) of sample • Floaters tended to sit in the middle/back of the classroom • The extent to which individual students participated in discussions of clicker questions with others depended on: 1) the individual’s preference for social interaction 2) whether s/he consistently did the reading before class • Distinguishing between “floating” and “free-riding”

20. Loners and Self-testers • Both groups contain students who say they prefer to work alone on clicker questions  Loners do, while self-testers work with others • Self-testers comprised a small but significant percentage (<20%) of each sample, while students who consistently worked alone were rare • Two primary reasons for working entirely on one’s own: 1) The student is unprepared and so feels uncomfortable discussing the material with others, and 2) shyness/introversion. • Loners & self-testers tended to sit in the back or on the side of the classroom, and were much more likely to be male • Stereotypes about “who” does science; “who” works well with technology; “who should be able to do things independently”

21. Additional Guidelines • Explain regularly to students how clickers are designed to benefit their learning (they will forget) • Encourage peer talk during clicker questions • You’re challenging years of educational socialization! Keep in mind, most research suggests clicker use is beneficial to learning in a variety of ways!

22. Two minute writing exercise inpreparation for the Q & A… Please reflect on what you have just learned by writing for two minutes. What do you think about all this? What questions do you have?

23. Question & Answer Session Followed by a break… please return at 10:30.

24. Writing Didactic Clicker Questions Individual & Small Group Work 10:30 – 11am

25. Guidelines for Clicker Question Writing • Most questions should range from medium-difficult • Easy clicker questions can be used but not too often • Response categories should offer answers that address common mistakes or misconceptions about the concept • Whenever you explain a clicker question, you should address both the correct and the incorrect answers • Possible response categories with your system: 10 • Students can choose a single answer from 0 – 9 on TurningPoint, or they can rank a set of multiple answers

26. Guidelines for Clicker Question Use • Questions should be used to “break up lecture” into sections of conceptual material… helps keep young adults engaged • Consider not using questions in the last five minutes of class • Consider giving students a few seconds to read the clicker question before peer discussion begins • Explain to your students how the clicker questions they see in class may mirror, or be similar to, your exam questions

27. Writing exercise: Goals for clicker use in your classroom Please list three to five goals that you hope to achieve through clicker use in your course/s. As you write, please take into account the particular nature of the material in your discipline. In addition, please write a little about your anticipated pedagogical approach to clicker use.

28. Small Group Work Please work with your colleagues in your small group to write a few clicker questions together at this time. Introduce yourselves, choose a topic to write a few questions on, and then work through writing them together.

29. Using Student Response Systems to Collect Data for Scholarship of Teaching and Learning ResearchMidwestern UniversityCollege of Health Sciences Faculty Retreat – August 23, 2010 Dr. Angel Hoekstra University of Colorado

30. What is SoTL Research? • “Published work on teaching and learning by college faculty” * • SoTL research focuses on post-secondary teaching & learning, whereas educational research refers to all levels • Professional associations that sponsor this kind of research: Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) American Association of Higher Education (AAHE) International Society of Scholarship of Teaching and Learning (ISSOTL) _______________________________________________________________ * A great resource: Maryellen Weimer, 2006. Enhancing Scholarly Work on Teaching and Learning: Professional Literature that Makes a Difference. Jossey-Bass.

31. The Challenge of Pedagogical Scholarship • Has not traditionally been valued in promotion & tenure • This status is changing as more attention is being paid in academic culture to: • Using active learning to keep students engaged • Holding students accountable for their own learning • Increased focus on confirming learning outcomes through research “Faculty’s continuing fixation with teaching prevents them from seeing that teaching has no purpose unless it can be directly and explicitly connected with learning outcomes… If practice is to improve and college teaching is to develop respect… there must be a viable literature associated with it.” (Weimer 2006, 7)

32. Two basic approaches to SoTL Research • Wisdom of Practice Scholarship: experience-based work, sometimes called the “how-to literature of teaching” • Research Scholarship: addresses pragmatic and applied empirical questions about teaching and learning SoTL scholarship makes some academics anxious because it challenges the traditional model of “discovering new knowledge by empirical means,” but SoTL scholars point out that there are other ways of knowing… ways that come from practice, from experience. These different ways of knowing are still legitimate, especially when paired with good data.

33. Wisdom of Practice Scholarship • Personal accounts of change • Recommended practices reports • Recommended content reports • Personal narratives • Advice giving based on “personal experience or research evidence or both” (Weimer, 40) • Involves reflective and critical analysis of one’s teaching • Content sometimes has an emotional tone, sometimes does not

34. Research Scholarship • Strives to examine teaching practices and learning outcomes by means of rigorous objectivity & rational analysis • This work can be quantitative, qualitative, descriptive, or some combination of all three… but is defined by data collection, analysis • Many “hybrid” approaches are used (e.g. Dr. Hoekstra’s doctoral thesis work) • Handout: List of SoTL Periodicals

35. What characteristics of SoTL research make it credible, rigorous, valued? • Well-designed research program w/appropriate methods • Viable and timely research question/s • Adequate to exceptional literature review • Placement of the phenomenon under study within an appropriate theoretical and/or epistemological context • A well-written and insightful report of the results • Reliability and validity of the data can be established • Author/s explain potential implications of the work

36. Benefits of using clickersto gather data for SoTL research • Clicker questions mimic Likert-style survey question format • Data gathering is efficient, no more hand coding responses • Data can be stored in clicker software by student ID #s • Optional nature of participation is obvious to students • A few survey questions can be asked per class day, rather than distributing an entire set of questions all at once • Handout: our study this fall in three science courses uses a statement encouraging honesty in clicker survey responses

37. Individual work: Reviewing examples Please choose two of the following four exemplary SoTL pieces to read before lunch, and we will discuss them in small groups when we return to the workshop. As you read these, please note in the margins: What are the strengths and weaknesses of each piece?

38. “Walking on Eggs: Mastering the Dreaded Diversity Discussion.” Frederick (1995). Personal narrative, some data Great practical advice for how to manage conflict in discussion “Helping Students Understand Grades.” Svinicki (1998). Nice placement of issue in context Strong practical advice, examples “Using Clickers in Nonmajors- and Majors-Level Biology Courses.” Crossgrove & Curran (2008). Strong literature review Comprehensive approach to research design, data analysis “Active Learning and Cooperative Learning in the Organic Chemistry Lecture Class.” Paulson (1999). Combines personal change account with strong quantitative data Great practical advice, examples Choose one from each category:

39. Small & Large Group Discussions What are the strengths and weaknesses of each of these exemplary SoTL articles? Which did you like better, and why?

40. What other questions do you have? Concluding Comments…

41. Thank you for your participation! Angel.Hoekstra@colorado.edu Many thanks to Dr. Derek Bruff, Dr. Stefanie Mollborn & Dr. Douglas Duncan. Some data in this presentation taken from the following: Mollborn, Stefanie and Angel Hoekstra. 2010. “A Meeting of Minds: Using Clickers for Critical Thinking and Discussion in Large Sociology Courses.”Teaching Sociology, 38(1): 18-27. Hoekstra, Angel. 2008. “Vibrant Student Voices: Exploring Effects of the Use of Clickers in Large College Classrooms.”Learning, Media, and Technology, 33(4): 329-341.